| *** tosky has quit IRC | 00:01 | |
| *** ramishra has quit IRC | 00:40 | |
| *** cdearborn has quit IRC | 00:44 | |
| *** jmasud has quit IRC | 00:55 | |
| *** jmasud has joined #tripleo | 00:56 | |
| *** ysandeep|away is now known as ysandeep | 00:59 | |
| *** Goneri has quit IRC | 01:13 | |
| openstackgerrit | Juan Badia Payno proposed openstack/tripleo-ansible master: BaR Always do MySQL dumps even the services wont be stopped https://review.opendev.org/711458 | 01:14 |
|---|---|---|
| *** rlandy has quit IRC | 01:21 | |
| *** lbragstad__ has joined #tripleo | 01:33 | |
| *** lbragstad_ has quit IRC | 01:36 | |
| *** hjensas_ has joined #tripleo | 01:51 | |
| *** hjensas has quit IRC | 01:54 | |
| openstackgerrit | Emilien Macchi proposed openstack/ansible-role-redhat-subscription master: Improve support for HTTP proxy https://review.opendev.org/714524 | 02:02 |
| *** jmasud has quit IRC | 02:03 | |
| *** jmasud has joined #tripleo | 02:04 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Deprecate Keepalived service https://review.opendev.org/657067 | 02:04 |
| *** ramishra has joined #tripleo | 02:07 | |
| ramishra | weshay|ruck, cloudnull: https://bugs.launchpad.net/tripleo/+bug/1868632 is due to missing missing mistral-extra | 02:08 |
| openstack | Launchpad bug 1868632 in tripleo "overcloud-deploy fails w/ The Workflow errored and no messages were received. " [Critical,Triaged] | 02:08 |
| ramishra | https://review.rdoproject.org/r/#/c/26072/ | 02:08 |
| ramishra | Not sure why it was reverted earlier.. revert does not have enough details | 02:09 |
| ramishra | ykarel|away: ^^ | 02:09 |
| *** lbragstad__ has quit IRC | 02:13 | |
| openstackgerrit | Merged openstack/tripleo-common master: Add mistral workflow module path to mistral ansible config https://review.opendev.org/714492 | 02:14 |
| openstackgerrit | Merged openstack/tripleo-ansible stable/train: Add role/playbook to manage nova image cache https://review.opendev.org/713661 | 02:35 |
| weshay|ruck | thanks ramishra | 02:44 |
| *** rh-jelabarre has quit IRC | 02:49 | |
| ykarel|away | ramishra, looking | 02:50 |
| ykarel|away | ramishra, issue was CentOS8 build job failing, failures likely ignored due to non-voting. | 02:51 |
| ykarel|away | ramishra, issue is mentioned in original patch https://review.rdoproject.org/r/#/c/25280/ | 02:51 |
| ykarel|away | ramishra, recently mistral-extra was added as depenedency for mistral, is this also needed for tripleo-common? | 02:52 |
| ykarel|away | ramishra, with https://review.rdoproject.org/r/#/c/25780/ | 02:53 |
| ramishra | ykarel|away: so we've not packaged it yet? | 02:54 |
| ramishra | " nothing provides python3-mistral-extra" ? | 02:54 |
| ykarel|away | ramishra, checking the logs, but with that mistral patch, that job passed | 02:54 |
| ykarel|away | so possibly some repo issue | 02:54 |
| ykarel|away | looking | 02:54 |
| ramishra | ykarel|away: mistal does not provide mistral-extra, it should be a dep for tripleo-common | 02:56 |
| ykarel|away | ramishra, where u seeing " nothing provides python3-mistral-extra"? | 02:57 |
| ramishra | ykarel|away: https://github.com/openstack/mistral/blob/master/requirements.txt | 02:57 |
| ykarel|away | ramishra, i think mistral guys were involved in this change, and they said it to be added for mistral | 02:57 |
| ramishra | ykarel|away: your build trace in https://review.rdoproject.org/r/#/c/25280/ | 02:57 |
| ramishra | so surely need mistral-extra as a requirement in tripleo-common | 02:57 |
| ykarel|away | ramishra, there issue was due to no space around | 02:58 |
| ykarel|away | nothing provides python3-mistral-extra>= needed by python3-tripleo-common-12.1.1-0.20200224125754.bbb3d19.el8.noarch | 02:58 |
| ykarel|away | missing space around >= | 02:58 |
| ykarel|away | ohhk i think i got issue | 02:59 |
| ykarel|away | let me confirm it | 02:59 |
| ramishra | ykarel|away: ack thanks for looking, but we need that req AFAICT | 03:00 |
| ykarel|away | ramishra, as per logs mistral-extra is installed in container https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-multinode-1ctlr-featureset010-tripleo-master/2031709/logs/undercloud/var/log/extra/podman/containers/mistral_engine/podman_info.log.txt.gz | 03:01 |
| ramishra | if it was the space issue, I've updated https://review.rdoproject.org/r/#/c/26072/ | 03:01 |
| ykarel|away | ramishra, more is missing there | 03:02 |
| ykarel|away | the original patch missed python2 also iirc | 03:02 |
| ykarel|away | but now it can be ignored | 03:02 |
| ramishra | the current issue is surely due to missing mistral-extra | 03:03 |
| ykarel|away | ramishra, i think issue is due to outdated mistral | 03:04 |
| ykarel|away | checking | 03:04 |
| ykarel|away | 10.0.0-0.20200320183531.8a5d35a.el8 | 03:04 |
| ramishra | https://github.com/rdo-packages/mistral-distgit/commit/4322798975ec3305090e7a89ccc31bbfdffdc815 | 03:05 |
| ramishra | we seem to have added it in mistral package | 03:06 |
| ramishra | so yeah, it could be old mistral package | 03:06 |
| ykarel|away | https://github.com/openstack/mistral/commit/8a5d35ac6f7a8677b33354d8a8fe8ea6a3738ea6 is included so mistral is also good | 03:06 |
| weshay|ruck | how are we getting old mistral packages out of consistent? | 03:07 |
| ramishra | ykarel|away: it should have been included in tripleo-common and not mistral.. but whatever | 03:07 |
| ramishra | mistral-extra is 'extra' and not a dep for mistral.. | 03:07 |
| ykarel|away | ramishra, ohhk we can do that but if u see logs it's installed there | 03:08 |
| ykarel|away | so something else is wrong | 03:08 |
| *** ykarel|away is now known as ykarel | 03:09 | |
| *** mmethot has joined #tripleo | 03:09 | |
| *** apetrich has quit IRC | 03:10 | |
| *** mmethot_ has quit IRC | 03:11 | |
| ramishra | ykarel: may be mistal-extra downstream package missing https://github.com/openstack/mistral-extra/commit/bb2af2e2be4ce39a621c222750b2dc7e360fa238 | 03:22 |
| ramishra | I mean rdo packages | 03:24 |
| ykarel | ramishra, ^^ is there in container | 03:24 |
| ramishra | ykarel: the commit is there in the container? | 03:24 |
| ramishra | or the package? | 03:24 |
| ykarel | python3-mistral-extra.noarch 10.0.1-0.20200307102243.b615bd4.el8 | 03:25 |
| ykarel | ^^ is installed in container | 03:25 |
| *** dking_desktop has quit IRC | 03:27 | |
| ykarel | ramishra, do u know since when it started happeing | 03:28 |
| ykarel | i see there is recent promotion for centos8 master | 03:28 |
| ykarel | ok me checks | 03:29 |
| ykarel | https://review.rdoproject.org/zuul/builds?job_name=periodic-tripleo-ci-centos-8-multinode-1ctlr-featureset010-tripleo-master | 03:30 |
| ykarel | last run passed with latest promotion | 03:30 |
| ykarel | but yes good to understand why it happened to avoid further recurrence of it | 03:32 |
| ykarel | strange it failed on 21st with same error, so something basic is wrong in those jobs | 03:37 |
| ramishra | oh, the last promotion was green. Is that the only job which has issues.. | 03:37 |
| ramishra | If the recent runs are green, we can probably update the bug and downgrade it from 'critical ' | 03:39 |
| ykarel | ramishra, from last run it seems good, but still i see something wrong as it failed on 21st with same error | 03:45 |
| ykarel | will try to find what's going on with these | 03:46 |
| ramishra | ykarel: ack, I just updated the bug with the current status.. | 03:46 |
| ykarel | ramishra, and wrt to mistral-extra as dep for tripleo-common | 03:48 |
| ykarel | i don't see mistral as dep for tripleo-common | 03:48 |
| ykarel | so don't get how switch of actions to mistral-extra needs now as dep for tripleo-common | 03:49 |
| ykarel | https://github.com/rdo-packages/tripleo-common-distgit/blob/rpm-master/openstack-tripleo-common.spec | 03:50 |
| ramishra | ykarel: we've mistral-lib that would provide the mistral api and mistral-extra for the actions we use | 03:53 |
| ykarel | ramishra, yes i see mistral-lib | 03:54 |
| ramishra | we've mistral package providing the action entrypoints earlier with mistral service | 03:54 |
| ramishra | not the case when they moved it to mistral-extra | 03:55 |
| ykarel | ramishra, so mistral-extra also needed on host | 03:56 |
| ykarel | outside of container | 03:56 |
| ramishra | nope, in the container | 03:56 |
| ykarel | okk, but it's there already | 03:56 |
| ramishra | ykarel: yeah I can see that.. not sure why it fails at times..I've not yet understood that | 03:57 |
| ykarel | ramishra, okk | 03:57 |
| ramishra | ykarel: we'll get rid of mistral in undercloud soon.. but we've to make sure the packaging is right and the mistral containers have mistral-extra too | 03:59 |
| ykarel | ramishra, okk | 04:11 |
| *** skramaja has joined #tripleo | 04:13 | |
| openstackgerrit | Rabi Mishra proposed openstack/python-tripleoclient master: Create *-messages container when setting deployment status https://review.opendev.org/714567 | 04:27 |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-heat-templates master: Remove most of the mistral related undercloud post config https://review.opendev.org/714354 | 04:29 |
| *** jamesdenton has quit IRC | 04:30 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-heat-templates master: Enable ceph-dashboard on multinode jobs https://review.opendev.org/714568 | 04:30 |
| *** bdodd has quit IRC | 04:52 | |
| *** jmasud has quit IRC | 04:53 | |
| *** jmasud has joined #tripleo | 04:56 | |
| *** ratailor has joined #tripleo | 04:58 | |
| *** jamesdenton has joined #tripleo | 05:03 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Neutron ML2/OVN: Add support to enable IGMP Snooping https://review.opendev.org/714462 | 05:04 |
| *** gchamoul has quit IRC | 05:11 | |
| ykarel | ramishra, i got why it failed in those jobs | 05:13 |
| ykarel | ramishra, mistral-api container don't have latest mistral package | 05:14 |
| ykarel | mistral-engine was fine | 05:16 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Introduce environments/disable-swift.yaml https://review.opendev.org/713747 | 05:17 |
| *** udesale has joined #tripleo | 05:19 | |
| ramishra | ykarel: Ah, ok. sounds resonable | 05:21 |
| ykarel | ramishra, but still its not clear why it didn't got updated | 05:24 |
| ykarel | all container images should be updated same | 05:25 |
| ramishra | ykarel: not sure why engine container use api container packages | 05:25 |
| ykarel | ramishra, actually issue is during mistral-db populate | 05:26 |
| ramishra | may be we bind mount stuff from host.. I can't see anything though.... | 05:26 |
| ykarel | and mistral-db populate usage api image | 05:26 |
| ykarel | and api image is outdated | 05:26 |
| ykarel | from container prepare logs it's missed why it's missing https://logserver.rdoproject.org/openstack-component-tripleo/opendev.org/openstack/tripleo-ci/master/periodic-tripleo-ci-centos-8-multinode-1ctlr-featureset010-tripleo-master/2031709/logs/undercloud/var/log/tripleo-container-image-prepare.log.txt.gz | 05:27 |
| ykarel | there is other bug in logging | 05:27 |
| *** abdysn has joined #tripleo | 05:28 | |
| *** gchamoul has joined #tripleo | 05:29 | |
| *** bfournie has quit IRC | 05:30 | |
| ykarel | ramishra, with logging bug i meant https://bugs.launchpad.net/tripleo/+bug/1866676 | 05:35 |
| openstack | Launchpad bug 1866676 in tripleo "Container updates logs are no longer logged to /var/log/tripleo-container-image-prepare-ansible.log" [Medium,Triaged] | 05:35 |
| *** evrardjp has quit IRC | 05:36 | |
| ramishra | ykarel: if mistral_db_sync is using mistral-api container that should have mistral-extra package.. I think it's missing there | 05:36 |
| *** evrardjp has joined #tripleo | 05:36 | |
| ykarel | ramishra, yup everything outdated there | 05:37 |
| ramishra | I think that's the issue... db_sync should not use api container IMO | 05:37 |
| ramishra | but it's history | 05:38 |
| ykarel | in current situation, all mistral-lib/mistral-extra/mistral should be in sync | 05:38 |
| ykarel | now everything should be ok after recent promotion | 05:38 |
| ramishra | sync in what? | 05:38 |
| ramishra | api does not need those actions | 05:38 |
| ykarel | ramishra, not api, it's about db | 05:38 |
| ykarel | let me paste links | 05:38 |
| ramishra | yes, db_sync should use an image with mistral-extra | 05:39 |
| *** shyamb has joined #tripleo | 05:39 | |
| ykarel | not just mistral-extra, but also need mistral and mistral-lib updated | 05:40 |
| ykarel | https://review.opendev.org/#/c/713873/ from mistral and https://review.opendev.org/#/c/709963/ from mistral-lib needed to get work with mistral-extra | 05:40 |
| shyamb | Hi ykarel: | 05:40 |
| shyamb | Will this document work for osp16 too? | 05:40 |
| shyamb | https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/13/html/advanced_overcloud_customization/sect-registering_the_overcloud | 05:40 |
| shyamb | I want to enable rhosp repos on all overcloud containers | 05:41 |
| shyamb | I saw this rhosp13 doc | 05:41 |
| shyamb | in rhosp16 I am not able to find it | 05:41 |
| ramishra | ykarel: yep, they should be in sync for this kind of change, very difficult to do that you do that when using from source | 05:45 |
| ykarel | shyamb, looks like in 16 method changed https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/16.0/html/advanced_overcloud_customization/ansible-based-registration | 05:46 |
| ykarel | but /me not aware about background, may be ramishra knows ^^? | 05:47 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/stein: ControlPlaneSubnetCidr in net_vip_map_external https://review.opendev.org/713549 | 05:47 |
| openstackgerrit | Merged openstack/python-tripleoclient stable/train: Add --work-dir option to container image build command https://review.opendev.org/714437 | 05:47 |
| openstackgerrit | Merged openstack/tripleo-ci master: Adding dummy epel and epel-modular repo https://review.opendev.org/713676 | 05:47 |
| ykarel | ramishra, yes | 05:47 |
| ykarel | looks like tooling issue | 05:48 |
| ykarel | but without logs it's hard to find what went wrong in those jobs | 05:48 |
| shyamb | ykarel: Okay, thank you | 05:48 |
| shyamb | I will try it | 05:48 |
| ykarel | ack | 05:49 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/train: ControlPlaneSubnetCidr in net_vip_map_external https://review.opendev.org/713548 | 05:54 |
| shyamb | ykarel: document looks perfect | 05:56 |
| shyamb | thank you | 05:56 |
| ykarel | ack | 05:56 |
| *** ykarel is now known as ykarel|afk | 05:58 | |
| *** shyamb has quit IRC | 06:00 | |
| *** abdysn has quit IRC | 06:11 | |
| *** Abdallahyas has joined #tripleo | 06:11 | |
| *** shyamb has joined #tripleo | 06:11 | |
| *** abdysn has joined #tripleo | 06:12 | |
| *** ykarel|afk is now known as ykarel | 06:14 | |
| *** Abdallahyas has quit IRC | 06:16 | |
| *** Abdallahyas has joined #tripleo | 06:16 | |
| *** abdysn has quit IRC | 06:17 | |
| *** saneax has joined #tripleo | 06:20 | |
| *** abdysn has joined #tripleo | 06:22 | |
| *** waleedm has joined #tripleo | 06:23 | |
| *** lmiccini has joined #tripleo | 06:24 | |
| *** Abdallahyas has quit IRC | 06:25 | |
| *** bfournie has joined #tripleo | 06:26 | |
| openstackgerrit | yatin proposed openstack/tripleo-ci master: [WIP] Enable tripleo-image-serve for CentOS8 https://review.opendev.org/713439 | 06:30 |
| openstackgerrit | Rabi Mishra proposed openstack/python-tripleoclient master: Create *-messages container when setting deployment status https://review.opendev.org/714567 | 06:42 |
| *** shyamb has quit IRC | 06:49 | |
| *** marios has joined #tripleo | 06:58 | |
| *** bandini has joined #tripleo | 07:03 | |
| *** marios_ has joined #tripleo | 07:08 | |
| *** pcaruana has joined #tripleo | 07:09 | |
| *** marios has quit IRC | 07:10 | |
| *** marios_ is now known as marios | 07:12 | |
| *** dpawlik has joined #tripleo | 07:16 | |
| openstackgerrit | Merged openstack/tripleo-validations master: Drop eventlet from lower constraints https://review.opendev.org/714283 | 07:24 |
| openstackgerrit | Merged openstack/tripleo-validations master: Don't write playbook stats in binary mode https://review.opendev.org/714280 | 07:35 |
| *** bogdando has joined #tripleo | 07:46 | |
| openstackgerrit | Merged openstack/python-tripleoclient master: Use ansible playbook for plan deploy https://review.opendev.org/713773 | 07:50 |
| Tengu | hjensas_: heya! apparently we want to put a |lower filter for the /etc/hosts generation :). | 07:50 |
| Tengu | although I always thought it WAS case-insensitive, a new BZ says otherwise. probably regression with the move to tripleo-ansible tripleo_hosts_entries | 07:51 |
| Tengu | hjensas_: pushing a test on my lab, we should be fixed shortly. | 07:53 |
| *** jcapitao has joined #tripleo | 07:57 | |
| abdysn | Tengu, hjensas_, good morning, so i am curious about this new patch, why was it not like this since the beginning? why was it render if it is possible to do it directly? | 07:58 |
| *** shyamb has joined #tripleo | 07:58 | |
| Tengu | abdysn: frankly, no idea, I didn't work on that role when it was created | 07:59 |
| Tengu | abdysn: and since I'm not really "fluent" with all the vars in t-h-t, I didn't see that hosts_entry thingy... and didn't think about checking if there was something already there :/. | 08:00 |
| Tengu | this is why I pushed my other changes, before hjensas_ had a look | 08:01 |
| abdysn | Tengu, my point is, maybe there is a point why it is rendered | 08:01 |
| Tengu | we will need to get cloudnull and Luke reviews on it anyway | 08:02 |
| *** cylopez has joined #tripleo | 08:02 | |
| abdysn | Tengu, great, was just curious. thanks for your hard work :D | 08:03 |
| Tengu | np - sorry for the delays though ^^' | 08:03 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-heat-templates master: Include Ansible host key and private key for ceph https://review.opendev.org/714104 | 08:04 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-heat-templates master: Enable ceph-dashboard on multinode jobs https://review.opendev.org/714568 | 08:04 |
| *** tkajinam has quit IRC | 08:05 | |
| *** slaweq has joined #tripleo | 08:06 | |
| shyamb | Hi | 08:06 |
| shyamb | ykarel: | 08:06 |
| shyamb | My provided list of repositories got configured on compute host | 08:07 |
| shyamb | But repositories inside container are different than host | 08:07 |
| shyamb | http://paste.openstack.org/show/791064/ | 08:07 |
| shyamb | What I am missing here? | 08:07 |
| shyamb | I need rhsm repos that I provide in env file during overcloud deployment should be available inside all openstack containers | 08:08 |
| *** apetrich has joined #tripleo | 08:09 | |
| shyamb | Tengu: Overcloud update is failing with this error | 08:14 |
| shyamb | http://paste.openstack.org/show/791065/ | 08:14 |
| shyamb | We are mounting a nfs share inside "/var/lib/nova" | 08:14 |
| shyamb | Do you see any issue from these logs? | 08:15 |
| *** xek has joined #tripleo | 08:15 | |
| shyamb | ""stderr: Error: relabel failed \"/var/lib/nova\": operation not supported"," | 08:15 |
| shyamb | this selinux context we are changing on "/var/lib/nova/triliovault-mounts" | 08:18 |
| shyamb | https://github.com/shyam-biradar/triliovault-cfg-scripts/blob/master/redhat-director-scripts/docker/services/trilio-datamover-osp16.yaml#L167 | 08:18 |
| shyamb | ykarel: | 08:18 |
| Tengu | shyamb: well, there's selinux, of course, and mounting something in the location you pointed will create issues since we have "some" parts in tripleo that ensures we have the right contexts... | 08:21 |
| shyamb | Tengu: This issue looks similar https://bugzilla.redhat.com/show_bug.cgi?id=1727260 | 08:22 |
| openstack | bugzilla.redhat.com bug 1727260 in openstack-tripleo-heat-templates "[OSP15] Deployment with NovaNfsEnabled: True to enabled NFS share for nova ephemeral storage fails" [High,Closed: errata] - Assigned to mschuppe | 08:22 |
| shyamb | in this case user is using NFS as nova backend | 08:23 |
| Tengu | shyamb: well, you want to check if you have https://review.opendev.org/#/c/669317/ | 08:23 |
| Tengu | guess that's it | 08:23 |
| shyamb | Yes, I am checking it | 08:24 |
| shyamb | but we are not using nfs for nova backend | 08:24 |
| shyamb | we are mounting our NFS for trilio backend | 08:24 |
| shyamb | so, I need to check the root cause | 08:24 |
| Tengu | so the fix will probably not apply to you since you're doing it from another place. | 08:24 |
| Tengu | so the nova_nfs_enabled won't be set to true, so the z flag is still applied, so it will fail. | 08:25 |
| Tengu | because NovaNfsEnabled is set to false | 08:25 |
| *** paramite has joined #tripleo | 08:26 | |
| ykarel | shyamb, looking, | 08:26 |
| ykarel | shyamb, so on 13, ur containers repos also got updated when u update repos on host? | 08:26 |
| *** rpittau|afk is now known as rpittau | 08:26 | |
| shyamb | ykarel: In osp13, we have not tried this scenario | 08:27 |
| ykarel | shyamb, okk, i don't think changing repos on host will auto change repos in containers | 08:28 |
| ykarel | but there may be some way available to do this | 08:28 |
| shyamb | ykarel: Okay | 08:29 |
| shyamb | ykarel: I think I can skip this | 08:30 |
| shyamb | I can install packages while building container | 08:30 |
| shyamb | and I can enable subscription there itself | 08:30 |
| shyamb | I needed this for testing on a running container | 08:30 |
| shyamb | Customer should also do not need any repos enabled on container, right? | 08:31 |
| ykarel | shyamb, no idea | 08:31 |
| shyamb | ykarel: okay, np. For now, I am all set | 08:33 |
| shyamb | Thank you | 08:33 |
| ykarel | ack | 08:33 |
| *** pkopec has joined #tripleo | 08:36 | |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/python-tripleoclient master: Limit ansible playbook execution with timeout https://review.opendev.org/713807 | 08:37 |
| shyamb | Tengu: Regarding that nfs issue | 08:39 |
| shyamb | This is the fix | 08:39 |
| shyamb | https://github.com/openstack/tripleo-heat-templates/blob/stable/train/deployment/nova/nova-compute-container-puppet.yaml#L754 | 08:39 |
| shyamb | One doubt I have is, why we are listing the volume two times? | 08:40 |
| shyamb | what is it's meaning | 08:40 |
| openstackgerrit | Daniel Bengtsson proposed openstack/tripleo-common stable/stein: Fix path of log file. https://review.opendev.org/714582 | 08:40 |
| *** amoralej|off is now known as amoralej | 08:40 | |
| Tengu | shyamb: it's not 2 times - there's an "if" just before. | 08:41 |
| *** tosky has joined #tripleo | 08:41 | |
| Tengu | so that's what I just said earlier: the fix won't apply to your case because you're not using the NovaNfsEnabled thing, while still mounting an NFS volume in that location (even sub-dir). | 08:41 |
| Tengu | can't work like that. | 08:41 |
| *** jbadiapa has quit IRC | 08:43 | |
| shyamb | okay | 08:44 |
| shyamb | okay | 08:44 |
| shyamb | so if condition is true then first volume statement will true? | 08:44 |
| shyamb | or second? | 08:44 |
| Tengu | first one | 08:45 |
| shyamb | This is how we mount the "/var/lib/nova" | 08:45 |
| shyamb | https://github.com/shyam-biradar/triliovault-cfg-scripts/blob/master/redhat-director-scripts/docker/services/trilio-datamover-osp16.yaml#L155 | 08:45 |
| Tengu | think about it as ternary | 08:45 |
| shyamb | in the same fashion | 08:45 |
| shyamb | I do not use that z flag | 08:46 |
| shyamb | Do you know what it is for? | 08:46 |
| shyamb | Tengu: | 08:46 |
| *** dtantsur|afk is now known as dtantsur | 08:47 | |
| openstackgerrit | Merged openstack/tripleo-common master: config-download timeout should KILL the process. https://review.opendev.org/713707 | 08:47 |
| *** sshnaidm|pto is now known as sshnaidm | 08:48 | |
| Tengu | shyamb: "z" is for relabelling.... it's even written in the error message you posted | 08:48 |
| *** ccamacho has joined #tripleo | 08:48 | |
| Tengu | and, with NFS volume, you can't relabel anything - nfs just doesn't support it | 08:48 |
| shyamb | Okay | 08:50 |
| shyamb | I got the overall issue | 08:50 |
| shyamb | I will try few things | 08:50 |
| shyamb | thank you | 08:50 |
| *** jmasud has quit IRC | 08:55 | |
| *** jmasud has joined #tripleo | 08:56 | |
| *** suuuper has joined #tripleo | 09:03 | |
| *** jpena|off is now known as jpena | 09:04 | |
| *** jpich has joined #tripleo | 09:06 | |
| *** lucasagomes has joined #tripleo | 09:06 | |
| bogdando | hjensas_, Tengu: hi, do you know if we can migrate scripts/hosts-config.sh in t-h-t to tripleo-ansible? | 09:12 |
| bogdando | not sure why it still sits there | 09:13 |
| bogdando | also afaik passing it via environment var may be limited in bash | 09:13 |
| Tengu | bogdando: I'd say "probably" | 09:13 |
| *** ccamacho has quit IRC | 09:15 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-upgrade master: Missing variable redifition during cleanup. https://review.opendev.org/714585 | 09:21 |
| *** panda|off is now known as panda | 09:21 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-upgrade master: Missed variable redefinition during cleanup. https://review.opendev.org/714585 | 09:23 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-upgrade stable/train: Missed variable redefinition during cleanup. https://review.opendev.org/714587 | 09:24 |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates master: Drop unused hosts-config.sh in deploy steps https://review.opendev.org/714588 | 09:25 |
| *** jfrancoa has joined #tripleo | 09:27 | |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-common master: Remove plan_management workflows and unused actions https://review.opendev.org/714314 | 09:29 |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-common master: Remove swift_backup workbook and some related actions https://review.opendev.org/714315 | 09:29 |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-common master: Remove deployment workbook and some unused actions https://review.opendev.org/714316 | 09:29 |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-common master: Remove unused support.yaml workbook https://review.opendev.org/714317 | 09:29 |
| *** bandini has quit IRC | 09:29 | |
| *** bandini has joined #tripleo | 09:30 | |
| *** avivgt has joined #tripleo | 09:31 | |
| *** shyamb has quit IRC | 09:36 | |
| *** alexmcleod has joined #tripleo | 09:42 | |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates master: Drop unused remnants of the hosts-config bits https://review.opendev.org/714588 | 09:45 |
| bogdando | cjloader, hjensas_: ^^ | 09:53 |
| *** leanderthal has joined #tripleo | 09:53 | |
| bogdando | the next move would be to ditch all the hosts list -> str -> list dances | 09:54 |
| *** leanderthal has quit IRC | 09:54 | |
| bogdando | and just use json all the places | 09:54 |
| bogdando | or a list maybe | 09:54 |
| bogdando | we have like 20 places we do join/split/join | 09:54 |
| bogdando | slagle: ^^ WDYT? | 09:54 |
| *** leanderthal has joined #tripleo | 09:55 | |
| Tengu | a list would be sufficient | 09:56 |
| bogdando | yea | 09:56 |
| Tengu | json for the hosts entries has no real use imho | 09:56 |
| bogdando | ok I'll take a loot that | 09:57 |
| bogdando | look and loot | 09:57 |
| *** ykarel is now known as ykarel|lunch | 09:57 | |
| *** shyamb has joined #tripleo | 09:59 | |
| *** lbragstad__ has joined #tripleo | 10:14 | |
| *** holser has joined #tripleo | 10:19 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-upgrade master: FFWD2 Overcloud upgrade automation. https://review.opendev.org/713528 | 10:19 |
| hjensas_ | bogdando: Tengu: how about storing the hostname/dnsname on the port? (where they belong - https://opendev.org/openstack/neutron-lib/src/branch/master/api-ref/source/v2/samples/ports/port-show-response.json#L17-L18 ? | 10:24 |
| bogdando | hjensas_: unsure how this stacks with the plans to remove neutron from uc | 10:25 |
| *** hjensas_ is now known as hjensas | 10:25 | |
| hjensas | bogdando: why would we ever want to do that? :) | 10:25 |
| bogdando | I think there is a plan for that | 10:25 |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Implement validations-libs List method for tripleoclient https://review.opendev.org/711957 | 10:26 |
| bogdando | and use ansible network automation instead | 10:26 |
| bogdando | hjensas: anyway, the idea is nice | 10:26 |
| bogdando | but would it also work with multiple overclouds? | 10:26 |
| bogdando | custom ports may have arbitrary dns domains or not? | 10:27 |
| hjensas | bogdando: multiple overclouds still have their own ports created? the dns domain and hostname can be set per port. So I don't see why arbitrary dns domains would'nt work. | 10:28 |
| bogdando | ok then | 10:28 |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Implement validations-libs Run method for tripleoclient https://review.opendev.org/711964 | 10:29 |
| bogdando | hjensas: IIRC there had been issues with some dns bits may be only a singletone admin config in ini files | 10:29 |
| bogdando | and users weren't being able even to update that config w/o admin rights for cloud | 10:30 |
| bogdando | in neutron conf files | 10:30 |
| bogdando | was that naneservers? | 10:30 |
| bogdando | nvm | 10:31 |
| hjensas | bogdando: I think that's historical by now. https://opendev.org/openstack/neutron-lib/src/branch/master/api-ref/source/v2/ports.inc#L43-L55 | 10:31 |
| bogdando | nice | 10:31 |
| bogdando | is still a quite new so indeed may be nice for tripleo to catch it up | 10:32 |
| *** udesale_ has joined #tripleo | 10:36 | |
| hjensas | I did'nt look at it a lot in THT context, but putting the data there seems easy. If that can help us reduce some of the list, map, string templating dancing in THT. I would have to look into. | 10:36 |
| *** hjensas_ has joined #tripleo | 10:37 | |
| *** udesale has quit IRC | 10:38 | |
| *** hjensas has quit IRC | 10:41 | |
| *** ykarel|lunch is now known as ykarel | 10:42 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-upgrade master: FFWD2 Overcloud upgrade automation. https://review.opendev.org/713528 | 10:45 |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-common master: Remove mistral dependency for password generation completely https://review.opendev.org/714598 | 10:50 |
| bogdando | hjensas_: wrt missing tht patch for the hosts_entry rework, just drop https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/common/deploy-steps.j2#L393 +3 | 10:57 |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-ansible master: Remove mistral dependency from password rotate module https://review.opendev.org/714599 | 10:57 |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-ansible master: Hide passwords from ansible output https://review.opendev.org/714600 | 10:57 |
| bogdando | and pass only hosts_entry at https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/common/deploy-steps.j2#L449 | 10:57 |
| bogdando | hjensas_: ^^ | 10:58 |
| bogdando | and that patch in tht should require your change in tripleo ansible | 10:58 |
| bogdando | hjensas_: > reduce some of the list, map, string templating dancing in THT | 11:00 |
| bogdando | HostsValue (and VipHosts maybe) are string heat values that ideally should be stored in ansible inventories by config download directly, as lists | 11:00 |
| bogdando | slagle: wdyt^ | 11:00 |
| *** jbadiapa has joined #tripleo | 11:00 | |
| bogdando | that would also affect the export dances documented for external data manipulations in multistack | 11:00 |
| bogdando | internal heat values are not a part of user visible "APIs" so we are free to move them to ansible from heat w/o upgrade impact IMO | 11:01 |
| bogdando | hjensas_: so that could be a good starting point | 11:02 |
| bogdando | same to moving HostsEntryValue from heat to inventory | 11:05 |
| *** s0ru has joined #tripleo | 11:08 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-quickstart-extras master: Set tempest_images path for periodic https://review.opendev.org/714609 | 11:09 |
| bogdando | hjensas_: btw, your current patches with hosts_entry rework, may also let VipHostsEntries go in tht | 11:10 |
| *** s0ru has left #tripleo | 11:10 | |
| bogdando | afaik, it becomes not needed once you drop those 3 unused params | 11:10 |
| bogdando | hjensas_: I hope that's clear. If not, I can propose that patch for you, to complete https://review.opendev.org/#/c/714409/ | 11:11 |
| hjensas_ | bogdando: I think get it, please propose the patch. I'm pre-occupied with the inspection issue in RHEL 8.2 and 16.1 downstream atm. | 11:13 |
| *** hjensas_ is now known as hjensas | 11:13 | |
| *** holser has quit IRC | 11:17 | |
| openstackgerrit | Daniel Bengtsson proposed openstack/tripleo-ansible master: Fix condition tuned profile. https://review.opendev.org/714610 | 11:18 |
| *** holser has joined #tripleo | 11:18 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ci master: Sync vexxhost and rdo cloud settings https://review.opendev.org/711500 | 11:18 |
| *** hjensas has quit IRC | 11:19 | |
| *** shyamb has quit IRC | 11:19 | |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates master: Use global hosts entries to configure /etc/hosts https://review.opendev.org/714611 | 11:19 |
| openstackgerrit | Daniel Bengtsson proposed openstack/tripleo-ansible master: Fix condition tuned profile. https://review.opendev.org/714610 | 11:20 |
| *** hjensas has joined #tripleo | 11:22 | |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates master: Use global hosts entries to configure /etc/hosts https://review.opendev.org/714611 | 11:24 |
| *** shyamb has joined #tripleo | 11:28 | |
| openstackgerrit | Daniel Bengtsson proposed openstack/tripleo-ansible master: Fix condition tuned profile. https://review.opendev.org/714610 | 11:29 |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-heat-templates master: Use global hosts entries to configure /etc/hosts https://review.opendev.org/714611 | 11:31 |
| *** dpawlik has quit IRC | 11:33 | |
| *** dpawlik has joined #tripleo | 11:34 | |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-heat-templates master: Only set haproxy::crl_file once https://review.opendev.org/713593 | 11:38 |
| *** rlandy has joined #tripleo | 11:40 | |
| *** psachin has joined #tripleo | 11:44 | |
| *** waleedm_ has joined #tripleo | 11:46 | |
| *** waleedm has quit IRC | 11:48 | |
| *** shyamb has quit IRC | 11:54 | |
| Tengu | w00t! validation framework are now under TripleO governance! | 11:55 |
| *** derekh has joined #tripleo | 11:56 | |
| *** jcapitao is now known as jcapitao_lunch | 11:59 | |
| *** jpena is now known as jpena|lunch | 11:59 | |
| *** rh-jelabarre has joined #tripleo | 12:00 | |
| *** rfolco has joined #tripleo | 12:02 | |
| *** rh-jelabarre has quit IRC | 12:04 | |
| *** jcoufal has joined #tripleo | 12:04 | |
| *** shyamb has joined #tripleo | 12:07 | |
| *** rh-jelabarre has joined #tripleo | 12:10 | |
| openstackgerrit | Carlos Goncalves proposed openstack/tripleo-ci master: Add Octavia standalone upgrade CI job https://review.opendev.org/710473 | 12:13 |
| openstackgerrit | Carlos Goncalves proposed openstack/tripleo-ci master: Add Octavia standalone upgrade CI job https://review.opendev.org/710473 | 12:15 |
| *** raildo has joined #tripleo | 12:18 | |
| *** ysandeep is now known as ysandeep|brb | 12:23 | |
| *** weshay|ruck changes topic to "CI status: RDO-Cloud CTRL plane going down today. monitoring mirror failures| community irc meeting Tues@1400 UTC - tripleo-ci-community meeting Tues@1330 UTC | https://docs.openstack.org/tripleo-docs" | 12:32 | |
| shyamb | Hi | 12:35 |
| shyamb | owalsh: | 12:35 |
| shyamb | We are facing an issue during upgrade of triliovault on osp16 | 12:35 |
| shyamb | It's related to "/var/lib/nova" relabelling | 12:35 |
| shyamb | Here is the error | 12:36 |
| shyamb | http://paste.openstack.org/show/791078/ | 12:36 |
| shyamb | We mount "/var/lib/nova" in our container | 12:36 |
| shyamb | Here are the mount options we use: https://github.com/shyam-biradar/triliovault-cfg-scripts/blob/master/redhat-director-scripts/docker/services/trilio-datamover-osp16.yaml#L174 | 12:36 |
| shyamb | This bug looks close to our issue: https://bugzilla.redhat.com/show_bug.cgi?id=1727260 | 12:38 |
| openstack | bugzilla.redhat.com bug 1727260 in openstack-tripleo-heat-templates "[OSP15] Deployment with NovaNfsEnabled: True to enabled NFS share for nova ephemeral storage fails" [High,Closed: errata] - Assigned to mschuppe | 12:38 |
| Tengu | shyamb: mention that you have an NFS mount that is managed by trilio in a sub-dir, and neither nova nor trilio knows about it, therefore mount the /var/lib/nova with the "z" flag, calling a relabelling... | 12:44 |
| *** jpich has quit IRC | 12:44 | |
| Tengu | this is the actual issue in your service... | 12:45 |
| Tengu | or, at least, that's what I understood earlier. | 12:45 |
| *** jpich has joined #tripleo | 12:45 | |
| shyamb | Tengu: Okay | 12:46 |
| shyamb | my nova backend is not nfs | 12:46 |
| shyamb | so it mounts the "/var/lib/nova" with z | 12:46 |
| shyamb | shared,z | 12:46 |
| Tengu | yep..... therefore the nfs_enable thing will be False, therefore it uses the "z" flag. but since you have a subdir on NFS there, it fails. | 12:46 |
| Tengu | because z is recursive relabelling | 12:47 |
| Tengu | guess you'll need to override nova template in order to enforce the absence of "z" flag there. | 12:47 |
| shyamb | okay | 12:48 |
| Tengu | UNLESS a new var/param is added in order to override the value in nova template (and in trilio).... | 12:48 |
| Tengu | but adding such a new param will probably raise some questions|concerns, and make the code more complicated for a single specific use-case. | 12:48 |
| Tengu | not sure it has a chance to make through reviews. | 12:48 |
| shyamb | I can share my changes with you for review | 12:49 |
| Tengu | shyamb: guess that nfs share *must* be known by nova? | 12:49 |
| shyamb | This was working fine on osp13, osp14 | 12:49 |
| Tengu | if not, well, move it elsewhere. if yes, well.... you have two proposals that make sense | 12:49 |
| Tengu | yeah, osp-13 and osp-14 were using docker without any selinux support within containers. | 12:50 |
| Tengu | osp-15 and newer moved to podman, and selinux support is fully active. | 12:50 |
| shyamb | okay | 12:50 |
| Tengu | I think this was already pointed in some earlier discussions. | 12:50 |
| Tengu | with docker daemon, an option was passed in order to deactivate some selinux extensions in the daemon - since podman doesn't have such a daemon, it wasn't that easy to deactivate those extensions, and we took the opportunity to port the templates to a full selinux support instead. | 12:51 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-quickstart-extras master: Fix build-images for centos8 jobs set tripleoclient and image-yaml https://review.opendev.org/714627 | 12:51 |
| Tengu | this is also better for customers, better security and all | 12:51 |
| cloudnull | if folks have a moment to review https://review.opendev.org/#/q/starredby:cloudnull+status:open,n,z it would be greatly appreciated. | 12:52 |
| shyamb | Tengu: Okay | 12:53 |
| shyamb | I see | 12:54 |
| *** jmasud has quit IRC | 12:54 | |
| shyamb | What are our options now? | 12:54 |
| Tengu | shyamb: as said: either override the nova templates in your project, or propose a patch adding a new param allowing to NOT set the "z" flag - of course, the default value shouldn't change the default behavior. | 12:55 |
| Tengu | shyamb: unless you can move that shared volume elsewhere, but if I understand, nova must have access to its content, right? | 12:55 |
| shyamb | Tengu: Yes, we can't move it | 12:55 |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Implement validations-libs framework for ALL validator commands https://review.opendev.org/714629 | 12:55 |
| shyamb | We intentionally moved here | 12:56 |
| Tengu | 'k. sooo... I don't see other options than the 2 above. | 12:56 |
| *** lmiccini has quit IRC | 12:56 | |
| shyamb | How can I override the nova template? | 12:56 |
| *** jmasud has joined #tripleo | 12:56 | |
| shyamb | You mean nova-compute heat template? | 12:56 |
| shyamb | Tengu: | 12:56 |
| Tengu | yep | 12:56 |
| shyamb | okay | 12:56 |
| Tengu | basically you'll create a custom service, like trilio-nova-compute and push the needed things there. | 12:57 |
| shyamb | okay | 12:57 |
| Tengu | you can copy-paste the original one and just drop the relevant flag|option|"if" and things | 12:57 |
| Tengu | and ensure all the templates mounting that same nova location are properly overridden as well. | 12:58 |
| owalsh | Tengu, shyamb: /var/lib/nova needs the z flag since it's mounted in multiple containers | 12:58 |
| Tengu | owalsh: n-yeah - but z fails to relabel when it has some subdir on NFS, which is shyamb case... | 12:58 |
| Tengu | it's an unusual thing. | 12:58 |
| Tengu | and a custom template|others. | 12:58 |
| shyamb | owalsh: okay | 12:59 |
| *** lmiccini has joined #tripleo | 12:59 | |
| Tengu | shyamb: MAYBE the right thing to do is to mount that nfs elsewhere on the host, and then bind-mount it to the location. | 12:59 |
| Tengu | that should prevent the relabelling issues. | 12:59 |
| shyamb | owalsh: We are checking the options to solve this one | 12:59 |
| shyamb | Tengu: It's complicated | 12:59 |
| shyamb | We are mounting our NFS share inside our container | 13:00 |
| shyamb | under "/var/lib/nova" | 13:00 |
| *** ekultails has joined #tripleo | 13:00 | |
| shyamb | so that it gets reflected on nova_libvirt and nova_compute | 13:00 |
| shyamb | containers | 13:00 |
| Tengu | chicken, egg, and other things. | 13:00 |
| shyamb | haha | 13:00 |
| Tengu | issue is: with that sub-dir, you'll get relabelling issues, because NFS doesn't know about it. | 13:01 |
| Tengu | (in short: that's the issue you're hitting now) | 13:01 |
| Tengu | and, yes, if you drop that "z" flag, you might have other issues as mentionned by owalsh.... | 13:01 |
| shyamb | Any NFS options that may prevent this situation? | 13:01 |
| *** jpena|lunch is now known as jpena | 13:01 | |
| owalsh | shyamb: does it have to be in /var/lib/nova? | 13:01 |
| shyamb | owalsh: our goal is to make our nfs share available to nova_compute and nova_libvirt container | 13:02 |
| owalsh | so no? | 13:02 |
| shyamb | mounting under "/var/lib/nova" solved the problem | 13:03 |
| shyamb | so we did | 13:03 |
| shyamb | Approved by Redhat dev team | 13:03 |
| Tengu | shyamb: you can mount it in other locations though... there's a whole filesystem in the containers in the end. | 13:03 |
| chandankumar | weshay|ruck, review rdoproject is down | 13:03 |
| weshay|ruck | k.. | 13:03 |
| shyamb | We mount it in our container | 13:04 |
| *** weshay|ruck changes topic to "CI status: RDO-Cloud CTRL plane going down today!!!| community irc meeting Tues@1400 UTC - tripleo-ci-community meeting Tues@1330 UTC | https://docs.openstack.org/tripleo-docs" | 13:04 | |
| shyamb | and it gets available in other two containers through "var/lib/nova" | 13:04 |
| shyamb | which is the only directory shared among three containers | 13:04 |
| shyamb | owalsh: | 13:04 |
| *** skramaja has quit IRC | 13:04 | |
| shyamb | we do not have direct access to nova_compute and nova_libvirt | 13:04 |
| *** skramaja has joined #tripleo | 13:05 | |
| shyamb | owalsh: Do you see any options here? | 13:05 |
| Tengu | shyamb: though you might do specific services for the nova-compute and nova-libvirt in order to mount another volume, to another location, with the right flags (i.e. no z since NFS) | 13:05 |
| owalsh | shyamb: how about NovaComputeOptVolumes | 13:05 |
| Tengu | ah, even better | 13:05 |
| *** psachin has quit IRC | 13:05 | |
| shyamb | owalsh: Okay, I am not aware of that container? | 13:06 |
| shyamb | It's used to mount additional volumes? | 13:06 |
| shyamb | to nova_libvirt as well as nova_compute? | 13:07 |
| *** ysandeep|brb is now known as ysandeep | 13:07 | |
| shyamb | our mount path is not static | 13:07 |
| shyamb | this is one more issue | 13:07 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates stable/train: HA: check before restarting resource on stack update https://review.opendev.org/714634 | 13:08 |
| Tengu | shyamb: ... any way to make it predectible at least? | 13:08 |
| shyamb | We create a hash for a give nfs share and mount under that hash directory | 13:08 |
| shyamb | technically we can predict. but we need to use the algorithm again | 13:09 |
| shyamb | :) | 13:09 |
| shyamb | Tengu: | 13:09 |
| owalsh | shyamb: doubt that matters if the bind mount isn't relabeling | 13:09 |
| Tengu | shyamb: well, so not that dynamic... you can probably generate a param file that will inject the needed info and use that NovaComputeOptVolumes thing.. | 13:11 |
| paramite | EmilienM, Thanks for the review on https://review.opendev.org/#/c/713627 . So I should just move the firewall record from puppet to HAProxy heat template IIUC | 13:12 |
| Tengu | owalsh: apparently that option only works for the nova_compute... ? | 13:12 |
| owalsh | at the moment but... | 13:12 |
| Tengu | :) | 13:12 |
| owalsh | ratailor: are you around? | 13:12 |
| openstackgerrit | Alan Bishop proposed openstack/puppet-tripleo stable/train: Handle ipv6 addresses in etcd and cinder's backend_url https://review.opendev.org/714635 | 13:13 |
| Tengu | yeah, there are other *OptVolumes in there - but not for libvirt. | 13:13 |
| ratailor | owalsh, yea. | 13:13 |
| Tengu | shyamb: maybe https://linux.die.net/man/8/nfs_selinux - not sure though. | 13:14 |
| shyamb | okay | 13:14 |
| Tengu | but it doesn't seem to actually add proper support for relabelling. | 13:15 |
| shyamb | Tengu: owalsh: If we get any nfs selinux option to solve this, then that would be the best way | 13:16 |
| *** lmiccini has quit IRC | 13:17 | |
| owalsh | shyamb: can set the selinux context in the mount options | 13:17 |
| owalsh | ratailor: hey, trying to figure out an selinux vs NFS issue with shyamb. Do you know much about this? | 13:17 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-ci master: lint: enable black https://review.opendev.org/703980 | 13:18 |
| shyamb | owalsh: that would be great | 13:18 |
| owalsh | shyamb: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/selinux_users_and_administrators_guide/sect-security-enhanced_linux-working_with_selinux-mounting_file_systems#sect-Security-Enhanced_Linux-Mounting_File_Systems-Mounting_an_NFS_File_System | 13:18 |
| shyamb | owalsh: Thanks, I will go through this document | 13:19 |
| shyamb | meanwhile if you guys get any specific option please let me know | 13:20 |
| *** amoralej is now known as amoralej|lunch | 13:20 | |
| shyamb | thanks a lot | 13:20 |
| shyamb | This is my end of the day | 13:20 |
| shyamb | owalsh: If you get anything, please send me on shyam.biradar@trilio.io | 13:21 |
| *** jpich has quit IRC | 13:22 | |
| owalsh | shyamb: ack, ttyl | 13:22 |
| marios | fultonj: gfidente|pto: any idea about the defaults at http://codesearch.openstack.org/?q=v4.0.10-stable-4.0-nautilus-centos-7-x86_64&i=nope&files=&repos= tripleo-common | 13:22 |
| fultonj | o/ | 13:22 |
| marios | mainly what should we default to for centos8 fultonj | 13:22 |
| *** jpich has joined #tripleo | 13:22 | |
| marios | (in call with rlandy and chandankumar discussing now) | 13:22 |
| * fultonj looking | 13:22 | |
| marios | & zbr | 13:22 |
| fultonj | marios: so you want to know what ceph-container to use | 13:23 |
| fultonj | marios: unfortunately we don't yet have a cent8 based container. we are waiting for one | 13:23 |
| zbr | i am it out of context | 13:23 |
| openstackgerrit | Flavio Fernandes proposed openstack/tripleo-heat-templates master: Add Octavia OVN Provider configuration https://review.opendev.org/707695 | 13:23 |
| fultonj | marios: i don't see any reason the cent7-based ceph container shouldn't work until it's available | 13:24 |
| fultonj | marios: will that suffice until then we have a cent8-based one? | 13:24 |
| marios | fultonj: ack | 13:25 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates master: Re-enable driver agent for scenario 10 https://review.opendev.org/714639 | 13:25 |
| *** lmiccini has joined #tripleo | 13:25 | |
| marios | fultonj: chandankumar: was poking at this waiting to hear if there was an error running the centos7 container (he will add details here we just got off the phone give couple mins ) | 13:25 |
| *** jcapitao_lunch is now known as jcapitao | 13:25 | |
| *** shyamb has quit IRC | 13:26 | |
| fultonj | chandankumar: i expect the cent7 ceph-container will continue to work | 13:26 |
| chandankumar | marios, fultonj on tobiko testing https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_6d6/683126/35/check/tobiko-tripleo-multinode-containers/6d69a07/logs/undercloud/home/zuul/overcloud_deploy.log | 13:26 |
| chandankumar | marios, fultonj it looked for /usr/share/ceph-ansible/roles/ceph-container-engine/vars/RedHat.yml' | 13:27 |
| chandankumar | and RETRYING: install container and lvm2 packages (3 retries left) | 13:27 |
| chandankumar | and from there I went to container image prepare file and saw c7 image so got worried, looked at the ceph-ansible code base it was correct but not sure why it is coming there | 13:28 |
| *** ykarel is now known as ykarel|away | 13:29 | |
| fultonj | chandankumar: what is the error message? | 13:30 |
| chandankumar | fultonj, FAILED - RETRYING: install container and lvm2 packages (3 retries left). | 13:30 |
| fultonj | chandankumar: we have a tripleo-validation for ceph to ensure lvm2 is installed on overcloud nodes | 13:30 |
| chandankumar | No package python-docker-py available.' | 13:31 |
| chandankumar | fultonj, the question is why it is pulling this file https://github.com/ceph/ceph-ansible/blob/stable-4.0/roles/ceph-container-engine/vars/RedHat.yml | 13:31 |
| *** derekh has quit IRC | 13:32 | |
| ratailor | owalsh, do we have a BZ to know the context ? | 13:32 |
| chandankumar | sorry I think I got the issue | 13:32 |
| fultonj | chandankumar: i agree we don't want to be using that RedHat.yml | 13:33 |
| fultonj | 09:32 chandankumar: sorry I think I got the issue | 13:33 |
| *** mcornea has joined #tripleo | 13:33 | |
| fultonj | chandankumar: you good then? | 13:34 |
| chandankumar | fultonj, https://github.com/ceph/ceph-ansible/blob/stable-4.0/roles/ceph-container-engine/tasks/pre_requisites/prerequisites.yml#L4 | 13:34 |
| chandankumar | I think I need to fix this part | 13:34 |
| chandankumar | fultonj, I need to use something like this {{ ansible_os_family | lower }}-{{ ansible_distribution_major_version | lower }}.yml | 13:35 |
| fultonj | chandankumar: we trigger ceph-ansible with a tag to make it not install any packages | 13:35 |
| chandankumar | fultonj, I need to use something like this {{ ansible_os_family }}-{{ ansible_distribution_major_version | lower }}.yml | 13:35 |
| owalsh | ratailor: not that I know of | 13:35 |
| owalsh | ratailor: IIUC trillovault container bind mounts /var/lib/nova and mounts NFS exports within that | 13:36 |
| fultonj | chandankumar: we've been assuming that the overcloud image should have what it needs (the lvm validation was only to figure out why the predeployed server scenario might have failed to simplify debug for user) | 13:36 |
| openstackgerrit | Rabi Mishra proposed openstack/python-tripleoclient master: Remove usage of swifthelper.SwiftTempUrlAction https://review.opendev.org/714640 | 13:36 |
| fultonj | https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansible/roles/tripleo_ceph_run_ansible/tasks/main.yml#L45 | 13:37 |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-common master: Remove unused support.yaml workbook https://review.opendev.org/714317 | 13:38 |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-common master: Remove mistral dependency for password generation completely https://review.opendev.org/714598 | 13:38 |
| fultonj | chandankumar: https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansible/roles/tripleo_ceph_common/defaults/main.yml#L64 | 13:39 |
| ratailor | owalsh, ack. looks like I need to scroll through discussion above. will do it later. Thanks! | 13:39 |
| chandankumar | fultonj, sorry you mean we can set ceph_ansible_skip_tags: '' to skip the package installation step? | 13:41 |
| fultonj | chandankumar: no | 13:41 |
| chandankumar | or is it not going to check for lvm? | 13:41 |
| fultonj | chandankumar: no | 13:42 |
| fultonj | chandankumar: what i mean is that ceph-ansiuble shouldn't be trying to install anything | 13:42 |
| fultonj | chandankumar: and that we're already passing tags so that it shouldn't | 13:42 |
| fultonj | chandankumar: so... | 13:42 |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Implement validations-libs framework for ALL validator commands https://review.opendev.org/714629 | 13:43 |
| chandankumar | fultonj, ok got it, Now I need to figure out why it is trying to install | 13:43 |
| fultonj | chandankumar: yeah | 13:43 |
| fultonj | chandankumar: by default it shouldn't be trying to install anything | 13:44 |
| fultonj | chandankumar: https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_6d6/683126/35/check/tobiko-tripleo-multinode-containers/6d69a07/logs/undercloud/var/lib/mistral/config-download-latest/ceph-ansible/ | 13:44 |
| openstackgerrit | Flavio Fernandes proposed openstack/tripleo-heat-templates master: Add Octavia OVN Provider configuration https://review.opendev.org/707695 | 13:44 |
| fultonj | chandankumar: the problem is in ceph-ansible | 13:44 |
| fultonj | chandankumar: it simply shouldn't try to install it | 13:44 |
| fultonj | chandankumar: wait... | 13:45 |
| chandankumar | ok | 13:45 |
| fultonj | chandankumar: interesting look at ceph_ansible_command.sh | 13:46 |
| fultonj | chandankumar: so problem isn't in ceph-ansible itself | 13:47 |
| fultonj | chandankumar: because we didn't pass it the tags it needs | 13:47 |
| fultonj | fmount: ^ | 13:47 |
| fultonj | chandankumar: in other wrods... | 13:47 |
| fultonj | https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansible/roles/tripleo_ceph_common/defaults/main.yml#L64 | 13:47 |
| fultonj | defines ceph_ansible_skip_tags with 'package-install,with_pkg' to be skipped | 13:47 |
| fultonj | but i don't see those in https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_6d6/683126/35/check/tobiko-tripleo-multinode-containers/6d69a07/logs/undercloud/var/lib/mistral/config-download-latest/ceph-ansible/ceph_ansible_command.sh | 13:48 |
| fultonj | so that's a bug in the tripleo-ansible role that gfidente|pto fmount and i maintain | 13:48 |
| fultonj | chandankumar: OR your job overrode that var | 13:48 |
| chandankumar | fultonj, let me check the job definition | 13:48 |
| chandankumar | fultonj, https://review.opendev.org/#/c/683126/36/zuul.d/tobiko-tripleo.yaml job definition | 13:49 |
| fmount | fultonj: so the tags we have to skip pkg installation aren't passed to the playbook? | 13:49 |
| fultonj | fmount: yes, exactly | 13:50 |
| chandankumar | fultonj, job definition and playbooks used are ok | 13:51 |
| *** ratailor has quit IRC | 13:52 | |
| chandankumar | fultonj, I am opening a bug | 13:54 |
| fultonj | chandankumar: ok, that's fair | 13:54 |
| fultonj | fmount: i wonder what happended | 13:54 |
| *** chem has quit IRC | 13:55 | |
| Tengu | matbu, gchamoul did you add any reviews requiering some love in the doc? | 13:55 |
| fultonj | fmount: i see we had some commits land in tripleo-ansible/roles/ceph_* so maybe they broke something | 13:55 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/python-tripleoclient master: Switch to ansible for overcloud node (un)provision https://review.opendev.org/708749 | 13:55 |
| *** chem has joined #tripleo | 13:57 | |
| fmount | yeah fultonj I'm taking a look as well | 13:57 |
| *** Goneri has joined #tripleo | 13:57 | |
| fultonj | chandankumar: let me know the bug and fmount or i will take ownership of it | 13:57 |
| *** tkajinam has joined #tripleo | 13:57 | |
| fultonj | thanks fmount | 13:57 |
| fmount | we should have some evidence in the previous job executions | 13:58 |
| * fultonj checks if my local runs in my env had the same issue | 13:58 | |
| *** jdandrea has joined #tripleo | 13:59 | |
| weshay|ruck | #topic agenda | 14:00 |
| weshay|ruck | * Review past action items | 14:00 |
| weshay|ruck | * One off agenda items | 14:00 |
| weshay|ruck | * Squad status | 14:00 |
| weshay|ruck | * Bugs & Blueprints | 14:00 |
| weshay|ruck | * Projects releases or stable backports | 14:00 |
| weshay|ruck | * Specs | 14:00 |
| weshay|ruck | * open discussion | 14:00 |
| weshay|ruck | Anyone can use the #link, #action and #info commands, not just the moderatorǃ | 14:00 |
| weshay|ruck | Hey folks! who's around? | 14:00 |
| ekultails | o/ | 14:00 |
| arxcruz|rover | o/ | 14:00 |
| rlandy | o/ | 14:01 |
| tkajinam | o/ | 14:01 |
| rfolco | o/ | 14:01 |
| pkopec | o/ | 14:01 |
| owalsh | o/ | 14:01 |
| jbadiapa | o/ | 14:01 |
| ysandeep | ysandeep, o/ | 14:02 |
| weshay|ruck | #topic review past action items | 14:02 |
| weshay|ruck | http://eavesdrop.openstack.org/meetings/tripleo/2020/ | 14:02 |
| weshay|ruck | anyone need to raise anything from a previous meeting? | 14:02 |
| weshay|ruck | None. | 14:03 |
| weshay|ruck | #topic one off agenda items | 14:03 |
| weshay|ruck | #link https://etherpad.openstack.org/p/tripleo-meeting-items | 14:03 |
| Tengu | «o/ | 14:03 |
| weshay|ruck | I added two items to the agenda.. | 14:03 |
| weshay|ruck | please add something if you have it.. | 14:03 |
| * weshay|ruck notes that rdo-cloud the 3rd party ci platform is in a scheduled maint window today | 14:03 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Refactorize test_tht_ansible_syntax to welcome new roles/modules https://review.opendev.org/713669 | 14:04 |
| weshay|ruck | the ctrl plane is moving to vexxhost | 14:04 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: standalone/overcloud: enable the HA deployment by default https://review.opendev.org/359060 | 14:04 |
| weshay|ruck | should be back by tomorrow | 14:04 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Deprecate Keepalived service https://review.opendev.org/657067 | 14:04 |
| fultonj | o/ | 14:04 |
| marios | hi | 14:04 |
| weshay|ruck | any questions? | 14:04 |
| weshay|ruck | EmilienM, stop working :) | 14:04 |
| cloudnull | o/ | 14:04 |
| leanderthal | o/ | 14:04 |
| sshnaidm | o/ | 14:04 |
| EmilienM | o/ | 14:04 |
| weshay|ruck | leanderthal, you may want to get the testday on the agenda :) | 14:05 |
| leanderthal | ooOOOooh good call | 14:05 |
| weshay|ruck | ok.. the next topic is around bugs | 14:05 |
| beagles | o/ | 14:05 |
| weshay|ruck | we were up to 60 untriaged bugs last week.. a lot of folks do NOT have access to self triage and that's fine | 14:05 |
| weshay|ruck | but if are a member of the tripleo org in launchpad.. we'd hope that you do follow the triage process | 14:06 |
| sshnaidm | don't we run a meeting bot? | 14:06 |
| weshay|ruck | So Question for the group... | 14:06 |
| weshay|ruck | sshnaidm, yes | 14:06 |
| weshay|ruck | but maybe it's down.. ? | 14:06 |
| weshay|ruck | don't know | 14:06 |
| weshay|ruck | Question... Any objects to closing any bugs in marked "New" ( untriaged ) older than 365 days? | 14:07 |
| weshay|ruck | objection rather | 14:07 |
| marios | weshay|ruck: sounds good to me | 14:07 |
| beagles | k | 14:07 |
| marios | weshay|ruck: but 'latest updated ' date right? | 14:07 |
| weshay|ruck | ya.. I'll account for that... good point | 14:07 |
| marios | weshay|ruck: not filed? maybe doesn't matter 365 is long enough that it either /or | 14:07 |
| chandankumar | fultonj, fmount https://bugs.launchpad.net/tripleo/+bug/1868731 thanks for looking into that | 14:07 |
| openstack | Launchpad bug 1868731 in tripleo "ceph_ansible_skip_tags is getting ignored during multinode overcloud deploy" [Critical,Triaged] | 14:07 |
| beagles | (wrong window) but yeah, that sounds reasonable | 14:07 |
| *** florianf has joined #tripleo | 14:08 | |
| fultonj | chandankumar: thanks | 14:08 |
| weshay|ruck | I'll send an email to the list to have folks reopen if they need it | 14:08 |
| weshay|ruck | ok.. seeing no objections... we'll go ahead and clean it up :) | 14:08 |
| weshay|ruck | #topic one off agenda items | 14:08 |
| weshay|ruck | #link https://etherpad.openstack.org/p/tripleo-meeting-items | 14:08 |
| weshay|ruck | #topic Active Squad status | 14:08 |
| weshay|ruck | ci | 14:08 |
| weshay|ruck | #link https://hackmd.io/IhMCTNMBSF6xtqiEd9Z0Kw?both | 14:08 |
| weshay|ruck | validations | 14:08 |
| weshay|ruck | #link https://etherpad.openstack.org/p/tripleo-validations-squad-status | 14:08 |
| weshay|ruck | ceph-integration | 14:08 |
| weshay|ruck | #link https://etherpad.openstack.org/p/tripleo-integration-squad-status | 14:08 |
| weshay|ruck | transformation | 14:08 |
| weshay|ruck | #link https://etherpad.openstack.org/p/tripleo-ansible-agenda | 14:08 |
| weshay|ruck | mistral-to-ansible | 14:09 |
| weshay|ruck | #link https://etherpad.openstack.org/p/tripleo-mistral-to-ansible | 14:09 |
| *** TrevorV has joined #tripleo | 14:09 | |
| weshay|ruck | any reviews needed by the squads? | 14:09 |
| Tengu | validations: we have our package under TripleO governance! | 14:09 |
| Tengu | it's been merged today. | 14:09 |
| weshay|ruck | rdo governance :) | 14:09 |
| cloudnull | weshay|ruck the following reviews would be great to get some eyes on - https://review.opendev.org/#/q/starredby:cloudnull+status:open,n,z | 14:09 |
| weshay|ruck | thanks cloudnull | 14:10 |
| weshay|ruck | fultonj, I mentioned this earlier.. but ansible 2.9 is being tested along side the latest ceph-ansible.. I don't have a ton of detail on that | 14:10 |
| *** derekh has joined #tripleo | 14:11 | |
| weshay|ruck | ok.. let's move on | 14:11 |
| fultonj | weshay|ruck: ack | 14:11 |
| weshay|ruck | #topic Moderately Active Squads | 14:11 |
| weshay|ruck | Ironic-integration | 14:11 |
| weshay|ruck | https://etherpad.openstack.org/p/tripleo-ironic-integration-squad-status | 14:11 |
| weshay|ruck | upgrade | 14:11 |
| weshay|ruck | #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status | 14:11 |
| weshay|ruck | edge | 14:11 |
| weshay|ruck | #link https://etherpad.openstack.org/p/tripleo-edge-squad-status | 14:11 |
| weshay|ruck | networking | 14:11 |
| weshay|ruck | #link https://etherpad.openstack.org/p/tripleo-networking-squad-status | 14:11 |
| weshay|ruck | probably nothing new here | 14:11 |
| weshay|ruck | #link https://launchpad.net/tripleo/+milestone/ussuri-3 | 14:11 |
| weshay|ruck | 5 New, 28 Incomplete, 4 Invalid, 4 Won't Fix, 4 Confirmed, 385 Triaged, 120 In Progress, 2 Fix Committed, 151 Fix Released | 14:12 |
| weshay|ruck | not bad on the bug counts.. I guess | 14:12 |
| weshay|ruck | Storyboard bugs. | 14:13 |
| weshay|ruck | #link https://storyboard.openstack.org/#!/project_group/76 | 14:13 |
| cloudnull | weshay|ruck we've recently updated the mistral_to_ansible board - https://storyboard.openstack.org/#!/board/208 | 14:13 |
| weshay|ruck | very nice.. | 14:14 |
| weshay|ruck | k.. will include that in the future | 14:14 |
| weshay|ruck | There are two blueprints not started for ussuri | 14:14 |
| weshay|ruck | Mediumtripleo-firewall-rule-reportingNewNot startedussuri | 14:14 |
| weshay|ruck | Mediumremove-puppetDraftingNot startedEmilien Macchiussuri | 14:14 |
| *** amoralej|lunch is now known as amoralej | 14:15 | |
| weshay|ruck | EmilienM, maybe we move that to the next release or future at this point... | 14:15 |
| weshay|ruck | April 20th is m3 I think | 14:15 |
| weshay|ruck | and dsneddon owns https://blueprints.launchpad.net/tripleo/+spec/tripleo-firewall-rule-reporting | 14:15 |
| weshay|ruck | #topic specs | 14:16 |
| weshay|ruck | #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open | 14:16 |
| weshay|ruck | reviews for luke please https://review.opendev.org/#/c/698828/ on | 14:16 |
| weshay|ruck | Add spec for AWX integration. | 14:16 |
| weshay|ruck | #topic open discussion | 14:16 |
| weshay|ruck | Anything else that folks want to bring up to the meeting? | 14:16 |
| EmilienM | weshay|ruck: i'll probably abandon it for now | 14:17 |
| weshay|ruck | k.. thanks | 14:17 |
| weshay|ruck | anyone else ? | 14:17 |
| weshay|ruck | going once | 14:17 |
| weshay|ruck | going twice | 14:17 |
| marios | weshay|ruck: | 14:17 |
| weshay|ruck | #endmeeting tripleo | 14:17 |
| marios | do we wantto consider virtual ptg | 14:17 |
| marios | weshay|ruck: ? | 14:17 |
| weshay|ruck | I'll save the meeting notes | 14:17 |
| marios | weshay|ruck: next time then | 14:17 |
| marios | ... | 14:17 |
| weshay|ruck | marios, that's still not clear | 14:17 |
| weshay|ruck | but most likely | 14:18 |
| cloudnull | +1 for virt ptg | 14:18 |
| marios | weshay|ruck: i think it is clear | 14:18 |
| marios | weshay|ruck: and we should consider how we do it if we do it etc | 14:18 |
| marios | weshay|ruck: or we do things via mailing list | 14:18 |
| sshnaidm | this is final I think, virtual prg | 14:18 |
| EmilienM | +1 | 14:18 |
| marios | nova had some good ideas today on the list | 14:18 |
| Tengu | +42 | 14:18 |
| marios | k weshay|ruck next week's meeting is still early enough for us to organise discuss/if not before then here etc | 14:19 |
| EmilienM | marios: yeah | 14:20 |
| EmilienM | #link http://lists.openstack.org/pipermail/openstack-discuss/2020-March/013555.html | 14:20 |
| EmilienM | check this out ^ pretty good stuffs | 14:20 |
| marios | EmilienM: ack saw some of the discussion go by today but didn't revisit there were some proposals in an equivalent nova thread | 14:20 |
| marios | EmilienM: yes actually this is the one i saw proposal from gibi | 14:21 |
| weshay|ruck | ah.. this is news to me.. | 14:23 |
| weshay|ruck | https://www.openstack.org/events/opendev-ptg-2020/ | 14:23 |
| weshay|ruck | PTG is canceled, will add to the next mtg | 14:23 |
| marios | weshay|ruck: yeah was news for me too this morning | 14:24 |
| leanderthal | well, ptg isn't cancelled, it's virtual. | 14:24 |
| leanderthal | ptg | 14:24 |
| marios | yeah | 14:24 |
| marios | well that's what we should decide formats/organise etc while we have lead time | 14:25 |
| weshay|ruck | March 20 I guess that news came out | 14:25 |
| leanderthal | if you'd like to help organize it, add your name here https://etherpad.openstack.org/p/Virtual_PTG_Planning | 14:25 |
| marios | weshay|ruck: yeah as you know cos you gave me the day off, i was out yesterday :D | 14:25 |
| leanderthal | it came out.... last thursday | 14:25 |
| marios | weshay|ruck: i guess i was working too hard on friday to notice ;) | 14:25 |
| * marios runs | 14:25 | |
| ekultails | A virtual PTG should be interesting. | 14:26 |
| leanderthal | sorry, wednesday lists.openstack.org/pipermail/foundation/2020-March/002854.html | 14:26 |
| weshay|ruck | didn't really hit the list afaict | 14:26 |
| weshay|ruck | odd.. must be a filter issue on my end | 14:27 |
| cloudnull | weshay|ruck did we ever figure out what was going on with mistral / mistral-extras ? -cc apetrich | 14:27 |
| mwhahaha | that didn't got to openstack-discuss | 14:27 |
| mwhahaha | that went to foundation | 14:27 |
| mwhahaha | (it should be forwarded) | 14:28 |
| apetrich | cloudnull, I'm looking into it since this morning but haven't found what can be causing it :( | 14:28 |
| weshay|ruck | cloudnull, ya.. updates in the bug.. yatin and Rabi were working on it last night | 14:28 |
| weshay|ruck | apetrich, thanks | 14:28 |
| *** ysandeep is now known as ysandeep|afk | 14:28 | |
| *** waleedm_ has quit IRC | 14:31 | |
| openstackgerrit | Luke Short proposed openstack/tripleo-common stable/train: config-download timeout should KILL the process. https://review.opendev.org/714659 | 14:33 |
| *** bdodd has joined #tripleo | 14:38 | |
| fultonj | did the meeting end? | 14:39 |
| weshay|ruck | aye | 14:39 |
| weshay|ruck | bot was down | 14:39 |
| fultonj | ok | 14:40 |
| fultonj | chandankumar: https://bugs.launchpad.net/tripleo/+bug/1868731/comments/1 | 14:40 |
| openstack | Launchpad bug 1868731 in tripleo "ceph_ansible_skip_tags is getting ignored during multinode overcloud deploy" [Critical,Triaged] | 14:40 |
| fultonj | chandankumar: standalone jobs are not affected | 14:40 |
| fultonj | chandankumar: is there a multinode job for train or cent7 ? | 14:41 |
| chandankumar | fultonj, fs016 was not used anywhere, we have started using right now for tobiko testing | 14:41 |
| chandankumar | fultonj, let me check train job | 14:41 |
| *** waleedm_ has joined #tripleo | 14:42 | |
| *** skramaja has quit IRC | 14:48 | |
| *** dking_desktop has joined #tripleo | 14:50 | |
| fultonj | chandankumar: https://github.com/openstack/tripleo-heat-templates/blob/master/deployment/ceph-ansible/ceph-base.yaml#L80-L83 | 14:51 |
| *** jbadiapa has quit IRC | 14:52 | |
| chandankumar | fultonj, https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/ci/environments/scenario001-multinode-containers.yaml#L144 | 14:52 |
| fultonj | WTF? | 14:52 |
| fultonj | who did that! | 14:52 |
| chandankumar | CephAnsibleSkipTags: '' | 14:52 |
| fultonj | that's the root cuase | 14:52 |
| fultonj | fmount: ^ fyi | 14:52 |
| fmount | oh | 14:53 |
| fultonj | chandankumar: nice find | 14:53 |
| Tengu | probably some wrong copy-paste :) | 14:53 |
| fultonj | chandankumar: can you just remove that line in your job? | 14:53 |
| fmount | ahahha yeah, so you're overriding the skip-tags w/ no value | 14:54 |
| fmount | yeah, in the standalone jobs we don't have CephAnsibleSkipTags | 14:54 |
| chandankumar | fultonj, yes sure | 14:54 |
| fmount | so the default value is used | 14:54 |
| fultonj | chandankumar: ok, thanks so it was the job definition | 14:54 |
| chandankumar | fultonj, yes | 14:54 |
| chandankumar | EmilienM, https://review.opendev.org/#/c/714104/ please +w it | 14:55 |
| fmount | and by default, in tripleo-ansible the parameter passed to ceph-ansible is https://github.com/openstack/tripleo-ansible/blob/master/tripleo_ansible/roles/tripleo_ceph_common/defaults/main.yml#L64 | 14:56 |
| fmount | chandankumar: ^ | 14:56 |
| fultonj | chandankumar: can you update your patch with Closes-Bug: #1868731 ? | 14:56 |
| openstack | bug 1868731 in tripleo "ceph_ansible_skip_tags is getting ignored during multinode overcloud deploy" [Critical,Triaged] https://launchpad.net/bugs/1868731 | 14:56 |
| *** abdysn has quit IRC | 14:56 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-heat-templates master: Include Ansible host key and private key for ceph https://review.opendev.org/714104 | 14:58 |
| chandankumar | fultonj, ^^ | 14:58 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-operator-ansible master: Add ffwd upgrade prepare role https://review.opendev.org/714669 | 14:58 |
| *** abdysn has joined #tripleo | 14:59 | |
| chandankumar | fultonj, https://review.opendev.org/#/c/714104/3/ci/environments/scenario001-multinode-containers.yaml@155 is it possible to set those values somewhere else | 14:59 |
| chandankumar | fmount, fultonj thanks :-) | 14:59 |
| fultonj | thanks chandankumar | 15:00 |
| chandankumar | fultonj, is there a way ANSIBLE_PRIVATE_KEY_FILE: '/home/zuul/.ssh/id_rsa_tripleo' to remove hardcoded zuul value? | 15:03 |
| fultonj | chandankumar: i'm thinking about that i saw mwhahaha 's comment | 15:03 |
| chandankumar | ok cool, thanks ! | 15:04 |
| fultonj | chandankumar: i think we could make your patch to tht depends-on a tripleo-ansible patch | 15:04 |
| *** tkajinam has quit IRC | 15:04 | |
| fultonj | and in that tripleo-ansible patch we could make it pass --overcloud-ssh-key + --overcloud-ssh-user to the ceph-ansible run | 15:04 |
| fultonj | chandankumar: i'll put a patch up for your to depends on and comment that in the review and we'll see if that's better | 15:05 |
| fultonj | fmount: fyi ^ | 15:05 |
| chandankumar | fultonj, works for me | 15:05 |
| EmilienM | bogdando: have you seen the comment on https://review.opendev.org/714588 ? | 15:09 |
| *** bogdando has quit IRC | 15:11 | |
| openstackgerrit | Merged openstack/tripleo-upgrade master: Missed variable redefinition during cleanup. https://review.opendev.org/714585 | 15:12 |
| *** leanderthal has quit IRC | 15:15 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Add centos/rhel based interface/bridge commands https://review.opendev.org/714208 | 15:17 |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: Clean unit tests from OS::TripleO::Hosts::SoftwareConfig https://review.opendev.org/714677 | 15:19 |
| EmilienM | hjensas: ^ | 15:20 |
| openstackgerrit | Daniel Bengtsson proposed openstack/tripleo-ci master: Fix the markup. https://review.opendev.org/714679 | 15:21 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-operator-ansible master: Add ffwd upgrade prepare role https://review.opendev.org/714669 | 15:35 |
| *** ysandeep|afk is now known as ysandeep | 15:35 | |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Implement validations-libs framework for ALL validator commands https://review.opendev.org/714629 | 15:36 |
| *** abdysn has quit IRC | 15:38 | |
| cloudnull | anyone mind giving https://review.opendev.org/#/c/714520/ a look | 15:39 |
| *** lbragstad__ has quit IRC | 15:40 | |
| *** abdysn has joined #tripleo | 15:41 | |
| weshay|ruck | sshnaidm, matbu FYI.. think I'm hearing opendev/x will be the best home | 15:41 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-operator-ansible master: Add ffwd upgrade prepare role https://review.opendev.org/714669 | 15:42 |
| slaweq | hi, is tripleo standalone working on centos 8 now? | 15:44 |
| *** d0ugal has quit IRC | 15:44 | |
| slaweq | I'm trying to deploy it and I have error like http://paste.openstack.org/show/791091/ | 15:44 |
| *** d0ugal has joined #tripleo | 15:44 | |
| slaweq | and I don't know what exactly am I missing there | 15:44 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-operator-ansible master: Add ffwd upgrade prepare role https://review.opendev.org/714669 | 15:49 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-operator-ansible master: Add overcloud ffwd-upgrade run https://review.opendev.org/714683 | 15:49 |
| mwhahaha | slaweq: if you're internal you need to specify internal ntp servers | 15:53 |
| mwhahaha | it failed trying to query ntp | 15:54 |
| openstackgerrit | Merged openstack/tripleo-common stable/stein: Fix path of log file. https://review.opendev.org/714582 | 15:55 |
| *** ykarel|away is now known as ykarel | 15:58 | |
| *** waleedm_ has quit IRC | 15:58 | |
| ykarel | mwhahaha, slaweq looks like issue is something else not related to ntp server, it failed getting ntp servers from hiera Failed to start Hiera: RuntimeError: Config file /etc/puppetlabs/puppet/hiera.yaml not found | 15:59 |
| ykarel | may be hiera/puppet installed from epel or some other repo? | 15:59 |
| mwhahaha | ykarel: that may not be the fatal error | 16:00 |
| mwhahaha | ntp is fatal | 16:00 |
| ykarel | mwhahaha, looks like it didn't went in the loop due to that hiera error | 16:00 |
| ykarel | https://opendev.org/openstack/tripleo-heat-templates/src/branch/master/validation-scripts/all-nodes.sh#L107 | 16:00 |
| slaweq | ykarel: can be | 16:03 |
| slaweq | I will check | 16:03 |
| *** hjensas_ has joined #tripleo | 16:04 | |
| *** hjensas has quit IRC | 16:07 | |
| *** jcapitao is now known as jcapitao_afk | 16:07 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Make the metalsmith module interactions more resilient https://review.opendev.org/714687 | 16:07 |
| openstackgerrit | Flavio Fernandes proposed openstack/tripleo-heat-templates master: Add Octavia OVN Provider configuration https://review.opendev.org/707695 | 16:08 |
| ykarel | slaweq, ack | 16:09 |
| *** florianf has quit IRC | 16:12 | |
| *** dciabrin has quit IRC | 16:18 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates stable/train: Introduce environments/disable-swift.yaml https://review.opendev.org/714688 | 16:21 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-upgrade master: FFWD2 Overcloud upgrade automation. https://review.opendev.org/713528 | 16:23 |
| *** dtantsur is now known as dtantsur|afk | 16:25 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-operator-ansible master: Add overcloud ffwd-upgrade run https://review.opendev.org/714683 | 16:30 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-operator-ansible master: Add ffwd upgrade converge https://review.opendev.org/714690 | 16:30 |
| *** jcapitao_afk is now known as jcapitao | 16:32 | |
| *** abishop_ has joined #tripleo | 16:33 | |
| *** abishop has quit IRC | 16:33 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-docs master: Introduce Multibackend Storage Documenation https://review.opendev.org/713078 | 16:34 |
| *** artom has quit IRC | 16:40 | |
| *** artom has joined #tripleo | 16:40 | |
| slaweq | ykarel++ it seems that I indeed had hiera from epel repo and that was the issue | 16:42 |
| openstackgerrit | Merged openstack/puppet-tripleo stable/train: pacemaker-remote is broken on CentOS/RHEL < 8 https://review.opendev.org/714107 | 16:43 |
| ade_lee | EmilienM, mwhahaha ever had an undercloud fail to install because of this error in puppet-container-ironic ? | 16:44 |
| ade_lee | uppet-user: Error: /Stage[main]/Ironic::Pxe/File[/var/lib/ironic/tftpboot/ipxe.efi]: Could not evaluate: Could not retrieve information from environment production source(s) file:///usr/share/ipxe/ipxe.efi | 16:44 |
| mwhahaha | nope i've seen that error in bug reports tho | 16:45 |
| * mwhahaha defers to bfournie, hjensas_, stevebaker & co | 16:45 | |
| ade_lee | EmilienM, mwhahaha looks like its failing to set up pxe stuff ... | 16:45 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-operator-ansible master: Fix readme issues https://review.opendev.org/714710 | 16:45 |
| ade_lee | bfournie, stevebaker hjensas_ ^^ hjensas_ this is running the undercloud install in your setup | 16:46 |
| *** abishop_ is now known as abishop | 16:47 | |
| *** ysandeep is now known as ysandeep|away | 16:50 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-operator-ansible master: Add ffwd upgrade converge https://review.opendev.org/714690 | 16:50 |
| *** marios has quit IRC | 16:53 | |
| *** udesale_ has quit IRC | 16:53 | |
| ykarel | slaweq, ack | 16:53 |
| *** ykarel is now known as ykarel|away | 16:53 | |
| *** jmasud has quit IRC | 16:54 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Add centos/rhel based interface/bridge commands https://review.opendev.org/714208 | 16:54 |
| *** jmasud has joined #tripleo | 16:55 | |
| ekultails | Can I get a Workflow nudge on this one? https://review.opendev.org/#/c/707380/ | 17:02 |
| *** lbragstad__ has joined #tripleo | 17:02 | |
| *** lbragstad__ is now known as lbragstad | 17:04 | |
| *** lucasagomes has quit IRC | 17:09 | |
| *** waleedm_ has joined #tripleo | 17:10 | |
| *** sshnaidm is now known as sshnaidm|afk | 17:13 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-operator-ansible master: Add overcloud credentials https://review.opendev.org/714718 | 17:13 |
| *** bandini has quit IRC | 17:14 | |
| *** jpich has quit IRC | 17:16 | |
| *** beagles has quit IRC | 17:16 | |
| *** rpittau is now known as rpittau|afk | 17:18 | |
| openstackgerrit | Merged openstack/tripleo-ansible master: tripleo_container_manage: improve logging for failed containers https://review.opendev.org/714234 | 17:20 |
| openstackgerrit | Merged openstack/tripleo-ansible master: Force container deletion if namespace does not exist in service_kill https://review.opendev.org/714517 | 17:23 |
| *** beagles has joined #tripleo | 17:29 | |
| *** abdysn has quit IRC | 17:31 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-upgrade master: Correct wrong template reference. https://review.opendev.org/714725 | 17:32 |
| *** evrardjp has quit IRC | 17:36 | |
| *** evrardjp has joined #tripleo | 17:36 | |
| *** waleedm_ has quit IRC | 17:36 | |
| *** pkopec has quit IRC | 17:37 | |
| bfournie | ade_lee: sorry, in a meeting, which version are you seeing this on? | 17:43 |
| *** elGamal has quit IRC | 17:45 | |
| *** fultonj has quit IRC | 17:45 | |
| *** jistr has quit IRC | 17:45 | |
| *** yolanda has quit IRC | 17:45 | |
| *** openstackstatus has quit IRC | 17:45 | |
| *** iurygregory has quit IRC | 17:45 | |
| *** dmsimard|off has quit IRC | 17:45 | |
| *** jistr has joined #tripleo | 17:45 | |
| ade_lee | bfournie, master I believe -- not sure if its actually causing a failure -- I tried adding some pxe related packages and errors went away | 17:45 |
| ade_lee | but I still have some error about Mar 24 17:21:03 undercloud.tls.example.com puppet-user[53922]: Error: Facter: error while resolving custom fact "rabbitmq_nodename": undefined method `[]' for nil:NilClass | 17:46 |
| *** yolanda has joined #tripleo | 17:46 | |
| ade_lee | mwhahaha, EmilienM ^^ ya'll seen that one before | 17:46 |
| EmilienM | no | 17:47 |
| mwhahaha | yes | 17:47 |
| mwhahaha | that fact is bad | 17:47 |
| mwhahaha | but it doesn't stop anything | 17:47 |
| *** irclogbot_0 has quit IRC | 17:47 | |
| mwhahaha | fact failures just spit output | 17:47 |
| ade_lee | mwhahaha, ok - then something else is borking things .. | 17:48 |
| *** openstackstatus has joined #tripleo | 17:48 | |
| *** ChanServ sets mode: +v openstackstatus | 17:48 | |
| mwhahaha | https://github.com/voxpupuli/puppet-rabbitmq/blob/master/lib/facter/rabbitmq_nodename.rb | 17:48 |
| mwhahaha | it's failing because rabbitmqctl status doesn't return expected output | 17:48 |
| mwhahaha | but it doesn't impact anything | 17:48 |
| mwhahaha | so if rabbitis installed but not up, it'll spit that out | 17:48 |
| *** elGamal has joined #tripleo | 17:49 | |
| *** irclogbot_1 has joined #tripleo | 17:49 | |
| *** fultonj has joined #tripleo | 17:50 | |
| mwhahaha | weshay|ruck: ade_lee is failing because he's getting centos7 containers on centos8 | 17:55 |
| mwhahaha | we need to better communicate how to get c8 stuff properly setup | 17:55 |
| weshay|ruck | mwhahaha, same setup | 17:55 |
| weshay|ruck | how can I be of service today | 17:55 |
| *** lmiccini has quit IRC | 17:55 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Drop unused remnants of the hosts-config bits https://review.opendev.org/714588 | 17:56 |
| mwhahaha | meh | 17:56 |
| EmilienM | weshay|ruck: hey, I'm working on enabling HA by default in TripleO https://review.opendev.org/#/c/359060 - and it's failing in the minor update (btw ansible-pacemaker issue is solved); when it tries to migrate rabbitmq from non-pacemaker to pacemaker managed containers | 17:56 |
| mwhahaha | weshay|ruck: so current-tripleo is now c8 containers right? | 17:56 |
| weshay|ruck | yes | 17:56 |
| EmilienM | weshay|ruck: at this point I wonder if I should make the job passing or if I should make it non voting, land my stuff and make it voting again | 17:56 |
| mwhahaha | ade_lee: you have centos7 running with centos8 containers, reprovision with centos8 | 17:56 |
| EmilienM | weshay|ruck: btw the jobname is tripleo-ci-centos-8-scenario000-multinode-oooq-container-updates | 17:57 |
| weshay|ruck | EmilienM, updates should remain voting | 17:57 |
| EmilienM | yes, I know but this is a CI situation, not production | 17:58 |
| weshay|ruck | yes.. I'm familiar w/ the reivew | 17:58 |
| weshay|ruck | EmilienM, how so? | 17:58 |
| EmilienM | migration from non-HA to HA isn't something we support AFIK | 17:58 |
| weshay|ruck | aye | 17:58 |
| EmilienM | the update job was designed without Pacemaker | 17:58 |
| EmilienM | and now I'm adding it to have better coverage (see the bug that I found with ansible-pacemaker) | 17:59 |
| weshay|ruck | EmilienM, so let's thread the needle.. I'll put up a patch for tht to mark it non-voting.. we'll mark a depends on the patch to that | 17:59 |
| weshay|ruck | and I'll put up a revert to get it back | 17:59 |
| weshay|ruck | sound ok? | 17:59 |
| EmilienM | weshay|ruck: yes please | 17:59 |
| weshay|ruck | k.. ping you in a few | 17:59 |
| mwhahaha | noooooo | 17:59 |
| mwhahaha | don't be doing ci shenanigans to do this | 18:00 |
| mwhahaha | fix it correctly | 18:00 |
| weshay|ruck | which is? | 18:00 |
| EmilienM | adding update tasks to migrate rabbitmq? | 18:00 |
| mwhahaha | figure out the teps to understandw hat it means to go non-ha to ha | 18:00 |
| mwhahaha | and write upgrade/update tasks | 18:00 |
| mwhahaha | the problem doesn't go away for upgrades | 18:00 |
| *** derekh has quit IRC | 18:01 | |
| EmilienM | good point... | 18:01 |
| EmilienM | but again, just for CI | 18:01 |
| mwhahaha | it doesn't go away | 18:01 |
| EmilienM | non-HA isn't something we support | 18:01 |
| mwhahaha | fix it correctly | 18:01 |
| *** jcapitao has quit IRC | 18:03 | |
| mwhahaha | EmilienM: and non-ha is something we support in the undercloud | 18:07 |
| mwhahaha | EmilienM: which is exactly why this needs to be solved correctly | 18:07 |
| openstackgerrit | Merged openstack/tripleo-ansible master: Add baremetal node providing module https://review.opendev.org/712303 | 18:08 |
| *** xek_ has joined #tripleo | 18:08 | |
| EmilienM | mwhahaha: this patch doesn't affect undercloud, we don't enable pacemaker | 18:09 |
| EmilienM | but I'll solve it correctly | 18:10 |
| *** xek has quit IRC | 18:11 | |
| *** jpena is now known as jpena|off | 18:12 | |
| fultonj | Can anyone tell me how config-download implements --overcloud-ssh-key and --overcloud-ssh-user ? (as passed to 'openstack overcloud deploy') | 18:13 |
| fultonj | I'm tracing my way through the code, but maybe someone can save me some time? | 18:13 |
| fultonj | https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L680-L690 | 18:13 |
| fultonj | Does it end up just setting ANSIBLE_PRIVATE_KEY_FILE before it executes ansible? | 18:14 |
| fultonj | ekultails cloudnull ^ ? | 18:14 |
| mwhahaha | fultonj: it's added to the host via enable_ssh_admin | 18:15 |
| fultonj | mwhahaha: i thought that's how the private key is installed on the host | 18:16 |
| mwhahaha | https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L964-L971 | 18:16 |
| mwhahaha | but that's only if not --config-download | 18:16 |
| fultonj | i'm asking how ansible knows to use what key/user when it executes | 18:17 |
| mwhahaha | if you use --config-download, i think it's assuemd you've set up the keys correctly and we only specifyt eh user | 18:17 |
| mwhahaha | you specify it in ansible.cfg? | 18:17 |
| mwhahaha | or env vars? | 18:17 |
| mwhahaha | aka the normal way ansible knows these things? | 18:17 |
| mwhahaha | or you can pass cli params | 18:17 |
| mwhahaha | so i guess the question is whenyou say how does ansible know, who is executing ansible? | 18:17 |
| fultonj | i know how to do all of that with ansible, that's not what i'm after | 18:18 |
| fultonj | and i'm not talking about --stack-only scenarios where the user runs config-download manually | 18:18 |
| mwhahaha | config-download is overloaded | 18:18 |
| fultonj | https://opendev.org/openstack/tripleo-docs/src/branch/master/deploy-guide/source/deployment/ansible_config_download.rst#user-content-config-download-related-cli-arguments | 18:18 |
| mwhahaha | ok so "in a normal deployment, how does..." | 18:18 |
| fultonj | yeah | 18:18 |
| ekultails | fultonj: TripleO creates it's own internal key-pair that gets setup for the `tripleo-admin` user on the Overcloud nodes. | 18:18 |
| fultonj | yep | 18:19 |
| fultonj | but then... | 18:19 |
| mwhahaha | when folks say config-download, i think --stack-only | 18:19 |
| ekultails | That --overcloud-ssh-{user,key} is only used for the initial connection to setup that tripleo-admin user and key it with the internal key-pair. | 18:19 |
| *** iurygregory has joined #tripleo | 18:19 | |
| fultonj | when mistral executes ansible | 18:19 |
| mwhahaha | mistral uses tripleo-admin | 18:19 |
| mwhahaha | so liek ekultails said, we use it to enable_ssh_admin | 18:19 |
| mwhahaha | https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L964-L971 | 18:20 |
| *** cylopez has quit IRC | 18:20 | |
| ekultails | ^ Yeap | 18:20 |
| fultonj | sorry... i'm having trouble expressing my question | 18:20 |
| ekultails | fultonj: No worries. What exactly are you trying to do / figure out? Do you need to run the deployment as a different user? | 18:21 |
| fultonj | https://review.opendev.org/#/c/714104/ | 18:21 |
| fultonj | my last comment in ^ | 18:21 |
| mwhahaha | so in this case, ceph is having problems with the ssh key but that's an external deploy tasks so it should be inheriting the existing tripleo-admin bits | 18:22 |
| mwhahaha | seems like something else is broken | 18:22 |
| mwhahaha | i think | 18:23 |
| mwhahaha | seems like an issue with the execution of ceph ansible as include in the deployment | 18:23 |
| * mwhahaha steps away | 18:23 | |
| fultonj | well, yes. _that_ is what i'm trying to fix and thus my question | 18:24 |
| fultonj | my initital thought was... | 18:25 |
| fultonj | If `openstack overcloud deploy` passes --overcloud-ssh-key and --overcloud-ssh-user, then pass those the corresponding options and values to the ceph-ansible execution. | 18:25 |
| ekultails | It should be assumed that the you will always connect as the `tripleo-admin` user for deployment tasks. The --overcloud-ssh-{user,key} is only used very temporarily used before the deployment. It's never used again for updates/upgrades/anything. The tripleo-admin key-pair used to be stored in Mistral's database (in all non-master branches). In master I believe we have it stored on the file system somewhere. | 18:26 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Add dynamic concurrency values https://review.opendev.org/714735 | 18:26 |
| fultonj | ekultails: yes, i think that's the first think i wasn't understanding | 18:27 |
| fultonj | The params --overcloud-ssh-key and --overcloud-ssh-user are used to install an ssh for future runs of config-download | 18:27 |
| ekultails | Correct. | 18:27 |
| fultonj | The params --overcloud-ssh-key and --overcloud-ssh-user are used to install an ssh key for a user for future runs of config-download, and not used for each run | 18:28 |
| fultonj | so i don't want to look for those params... | 18:28 |
| ekultails | Yeap, not exactly what you want. | 18:28 |
| fultonj | however, there's something in the config-download run i want to pass to the ceph-ansible run | 18:28 |
| fultonj | i can see this in chandan's job | 18:28 |
| fultonj | export ANSIBLE_PRIVATE_KEY_FILE="/home/zuul/.ssh/id_rsa_tripleo" | 18:29 |
| fultonj | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_5ca/683126/34/check/tobiko-tripleo-multinode-containers/5ca1d8f/logs/undercloud/var/lib/mistral/config-download-latest/ansible-playbook-command.sh | 18:29 |
| fultonj | so i should just be able to use ANSIBLE_PRIVATE_KEY_FILE in my job | 18:30 |
| fultonj | not job | 18:30 |
| fultonj | in my tripleo-ansible/ceph* patch | 18:30 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Dynamically adjust the retires value to fit the deployment https://review.opendev.org/714736 | 18:31 |
| fultonj | https://review.opendev.org/#/c/708252/ was too many vars | 18:31 |
| *** amoralej is now known as amoralej|off | 18:31 | |
| ekultails | You can probably assume the key is always at $HOME/.ssh/id_rsa_tripleo | 18:32 |
| ekultails | And setting it via ANSIBLE_PRIVATE_KEY_FILE should work, yeah. | 18:32 |
| fultonj | https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansible/roles/tripleo_ceph_work_dir/tasks/get_ssh_private_key.yml | 18:33 |
| fultonj | ekultails: ^ | 18:33 |
| fultonj | ekultails: my experience has been that config-download puts the key in its {{ playbook_dir }} and calls it ssh_private_key | 18:35 |
| *** cdearborn has joined #tripleo | 18:35 | |
| fultonj | I can expand thoser tasks to look for $HOME/.ssh/id_rsa_tripleo | 18:35 |
| fultonj | ekultails: are you aware of any changes comeing for {{ playbook_dir }}/ssh_private_key ? | 18:36 |
| cloudnull | fultonj - https://opendev.org/openstack/tripleo-ansible/src/branch/master/tripleo_ansible/roles/tripleo_ceph_work_dir/tasks/get_ssh_private_key.yml#L21 - would be /var/lib/mistral/$CLOUD/ssh_private_key which should exist however, with the removal of the mistral get / store key workflows that location should now be considered unreliable. | 18:37 |
| cloudnull | $HOME/.ssh/id_rsa_tripleo would be better IMO | 18:37 |
| ekultails | fultonj: So I think $HOME/.ssh/id_rsa_tripleo is where the key lives in master. We definitely used to have /var/lib/mistral/overcloud/ssh_private_key and probably still use it (and just copy from ~/.ssh/id_rsa_tripleo). | 18:38 |
| ekultails | cloudnull++ | 18:38 |
| fultonj | cloudnull ekultails right :) | 18:38 |
| fultonj | ok, so that used to work, might still work for a little while more, but i should update to the new way | 18:38 |
| *** saneax has quit IRC | 18:39 | |
| cloudnull | ++ | 18:39 |
| fultonj | ekultails cloudnull i'll update the path to use both but let $HOME/.ssh/id_rsa_tripleo win if both are set | 18:39 |
| fultonj | eventually i could remove the /var/lib/mistral/$CLOUD/ssh_private_key but supporting both should be safe for train backport too | 18:40 |
| fultonj | ekultails cloudnull i'll add you both to the review since you're now familiar with what i'm trying to do. thanks for the advice | 18:40 |
| cloudnull | +1 | 18:40 |
| Madkiss | hi everybody. quick Q: When I want to have networks in network_data.yaml for TripleO collapse, how do I do that? I want to use the same untagged vlan for Ctrlplane, tenant, storage-mgmt and internalapi. | 18:50 |
| fultonj | Madkiss: if you deploy without network isoloation all networks will run on the provisioning network. | 18:51 |
| fultonj | Madkiss: but you're trying to convert an existing deloyment by flattening, then i don't know how to do that | 18:51 |
| *** jcoufal has quit IRC | 18:52 | |
| Madkiss | fultonj: no no, I deploy a new cloud, I kind of need a mixture though | 18:52 |
| *** dpawlik has quit IRC | 18:53 | |
| Madkiss | The basic design looks like this: All machines have two Mellanox Connect-X5 cards with two times 25G (ens1f0/ens1f1 and ens7f0/ens7f1). I want all machines to have bond0 out of ens1sf0/ens1f1 untagged in port 20, which is the management, storage-management, internal api and tenant transport layer. | 18:58 |
| Madkiss | on a subset of nodes, i want ens7f0/ens7f1 to be bond1 and be present in three distinct VLANs (those are the network nodes). and on another subset of nodes, i want the same bond1-based bond to be present untagged in yet another vlan, which is the storag enetwork. | 18:59 |
| Madkiss | i am looking at the documentation, but there are so many options and possibilities that I kind of don't see where to begin. | 18:59 |
| EmilienM | Madkiss: long time no see | 18:59 |
| *** hberaud has quit IRC | 19:01 | |
| Madkiss | hello EmilienM, indeed :-) | 19:02 |
| *** hberaud has joined #tripleo | 19:04 | |
| Madkiss | I am unsure as to whether when I specifiy the same ip rangds for the networks in network_data.yaml, will tripleo recognize that? i do not want a host to have multiple ips on the same interface in the same network. | 19:04 |
| *** alexmcleod has quit IRC | 19:07 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Add dynamic concurrency values https://review.opendev.org/714735 | 19:07 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Dynamically adjust the retires value to fit the deployment https://review.opendev.org/714736 | 19:08 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Add dynamic concurrency values https://review.opendev.org/714735 | 19:09 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Dynamically adjust the retires value to fit the deployment https://review.opendev.org/714736 | 19:09 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Dynamically adjust the retires value to fit the deployment https://review.opendev.org/714736 | 19:10 |
| *** holser__ has joined #tripleo | 19:17 | |
| stevebaker | morning | 19:18 |
| *** holser has quit IRC | 19:18 | |
| ekultails | Hey :-) | 19:19 |
| stevebaker | weshay|ruck, rlandy: I have modified and uploaded a upstream-centos-8-1584907370 which has my key injected in it, and it works! | 19:20 |
| rlandy | stevebaker: k - good news - we uploaded a new image to the openstack-nodepool with the same - however if your image works, that's fine | 19:21 |
| rlandy | stevebaker: we're in a bit of flux today with the rdocloud/vexxhost move | 19:22 |
| stevebaker | rlandy: good luck! | 19:22 |
| *** matbu has quit IRC | 19:23 | |
| *** holser has joined #tripleo | 19:32 | |
| *** holser__ has quit IRC | 19:34 | |
| weshay|ruck | stevebaker, yay! | 19:46 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: standalone/overcloud: enable the HA deployment by default https://review.opendev.org/359060 | 19:50 |
| *** rcernin|brb has quit IRC | 19:57 | |
| *** jfrancoa has quit IRC | 20:02 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-ansible master: Make ceph-ansible run handle missing "{{ playbook_dir }}/ssh_key" https://review.opendev.org/714768 | 20:14 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Deprecate Keepalived service https://review.opendev.org/657067 | 20:16 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Normalize how we run validations https://review.opendev.org/714769 | 20:19 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Normalize how we run validations https://review.opendev.org/714769 | 20:20 |
| openstackgerrit | Merged openstack/tripleo-ansible stable/train: Insert hosts entries to the beginning of the file. https://review.opendev.org/707380 | 20:23 |
| *** abishop has quit IRC | 20:37 | |
| *** abishop_ has joined #tripleo | 20:37 | |
| *** hjensas_ is now known as hjensas | 20:39 | |
| *** TrevorV has quit IRC | 20:47 | |
| *** asbishop has joined #tripleo | 20:49 | |
| *** abishop_ has quit IRC | 20:51 | |
| *** jmasud has quit IRC | 20:54 | |
| *** jmasud has joined #tripleo | 20:56 | |
| openstackgerrit | Flavio Fernandes proposed openstack/tripleo-heat-templates master: Add Octavia OVN Provider configuration https://review.opendev.org/707695 | 20:57 |
| *** raildo_ has joined #tripleo | 20:59 | |
| *** raildo has quit IRC | 21:01 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-operator-ansible master: Add overcloud delete https://review.opendev.org/714776 | 21:02 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-operator-ansible master: Add overcloud delete https://review.opendev.org/714776 | 21:04 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Add centos/rhel based interface/bridge commands https://review.opendev.org/714208 | 21:08 |
| *** rfolco has quit IRC | 21:12 | |
| EmilienM | weshay|ruck: would be nice to get https://review.opendev.org/#/c/705775/ | 21:16 |
| weshay|ruck | ah cool | 21:17 |
| weshay|ruck | mwhahaha++ | 21:17 |
| *** cjloader has quit IRC | 21:18 | |
| *** holser has quit IRC | 21:18 | |
| *** holser has joined #tripleo | 21:19 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: WIP: Remove connections added by Network Manager https://review.opendev.org/714777 | 21:19 |
| *** abishop_ has joined #tripleo | 21:26 | |
| *** dciabrin has joined #tripleo | 21:26 | |
| cloudnull | anyone around mind giving https://review.opendev.org/#/q/topic:bug/1868619+(status:open+OR+status:merged) a review w | 21:27 |
| *** asbishop has quit IRC | 21:28 | |
| openstackgerrit | Merged openstack/tripleo-operator-ansible master: Fix readme issues https://review.opendev.org/714710 | 21:30 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: WIP: Remove connections added by Network Manager https://review.opendev.org/714777 | 21:31 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-ansible master: Fix population of username, public keys https://review.opendev.org/714778 | 21:32 |
| openstackgerrit | Steve Baker proposed openstack/python-tripleoclient master: Switch to ansible for overcloud node (un)provision https://review.opendev.org/708749 | 21:33 |
| openstackgerrit | Steve Baker proposed openstack/python-tripleoclient master: overcloud deploy, use ansible for bm provision https://review.opendev.org/712221 | 21:33 |
| *** holser has quit IRC | 21:36 | |
| Madkiss | where can I see what Ironic and Heat consider "nic1", "nic2" etc for a system? | 21:37 |
| cloudnull | weshay|ruck mwhahaha did either of you ever figure out what was going on with build images and cent8? | 21:37 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-ansible master: Fix population of username, public keys https://review.opendev.org/714778 | 21:37 |
| mwhahaha | no | 21:38 |
| cloudnull | ok :'( | 21:38 |
| mwhahaha | refresh my memory, what was wrong again? | 21:38 |
| mwhahaha | besides everything | 21:39 |
| cloudnull | its just that | 21:39 |
| cloudnull | :D | 21:39 |
| cloudnull | when building images - https://pasted.tech/pastes/b8bf1466a4f64d2cb2eda4cde05e61ea2e09e95b | 21:39 |
| cloudnull | it seems to be failing in this new way ? | 21:39 |
| mwhahaha | odd let me give it a shot | 21:40 |
| cloudnull | though this is the first ive tried it in a week or so | 21:40 |
| cloudnull | this is the complete log - https://pasted.tech/pastes/37fc0b85854200c7ae29711c261cd5d4fc17c33e.raw | 21:41 |
| cloudnull | which I didn't find helpful | 21:41 |
| *** abishop_ has quit IRC | 21:41 | |
| mwhahaha | you're missing the HA repos | 21:41 |
| mwhahaha | 2020-03-24 21:32:03.084 | No match for argument: pacemaker-remote | 21:41 |
| *** abishop has joined #tripleo | 21:41 | |
| cloudnull | i still see "Error: Unable to find a match: pacemaker-remote pacemaker pcs" though idk why ? | 21:42 |
| cloudnull | oh, hum... which repos has that ? | 21:42 |
| mwhahaha | CentOS-HA.repo on my system | 21:42 |
| mwhahaha | or we have a tripleo-centos-highavailability that tripleo-repos provides | 21:42 |
| mwhahaha | if you have uses a recent version of tripleo-repos it should configure it for you on centos8 | 21:43 |
| mwhahaha | if you're on a centos7 env trying to build a centos8, then you're missing it | 21:43 |
| mwhahaha | 2020-03-24 21:32:02.534 | repo: using cache for: centos7-rt | 21:43 |
| mwhahaha | seems odd | 21:43 |
| *** xek_ has quit IRC | 21:43 | |
| cloudnull | im on cent8 , that repo wasn't enabled. will enable it and try again | 21:43 |
| cloudnull | dib should default to building cent8 on cent8 right ? | 21:44 |
| mwhahaha | probably not but who knows | 21:44 |
| mwhahaha | depends on the cli params you're using | 21:44 |
| mwhahaha | we need to update it to default to centos8 now | 21:44 |
| cloudnull | meh , running the things and will report back :D | 21:44 |
| *** raildo_ has quit IRC | 21:45 | |
| cloudnull | Im running `openstack --os-cloud undercloud overcloud image build --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-images-centos8.yaml --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-images-python3.yaml` | 21:45 |
| cloudnull | though i vaguely remember us talking about py3 not being needed? | 21:45 |
| mwhahaha | i usually flip the order | 21:46 |
| mwhahaha | because overcloud-images.yaml is py2 by default | 21:46 |
| mwhahaha | so we have to specify overcloud-images-python3.yaml | 21:46 |
| mwhahaha | to get the python3 opnes | 21:46 |
| cloudnull | ok | 21:46 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-quickstart-extras master: Exit overcloud-deploy.sh on provision failure https://review.opendev.org/714779 | 21:46 |
| Madkiss | do OVS bridges in Ironic support LACP by now without dpdk? | 21:48 |
| mwhahaha | maybe? | 21:50 |
| mwhahaha | https://access.redhat.com/solutions/3417681 seems to indicate yes, though not certain how to express it for os-net-config | 21:51 |
| Madkiss | BondInterfaceOvsOptions seems to be the place for that. | 21:51 |
| *** jcoufal has joined #tripleo | 21:53 | |
| *** jcoufal has quit IRC | 21:54 | |
| ade_lee | mwhahaha, have you tried -- export DIB_YUM_REPO_CONF="/etc/yum.repos.d/delorean*" | 21:56 |
| ade_lee | openstack overcloud image build --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-images-centos8.yaml | 21:56 |
| mwhahaha | yea you need to also specify the python3 image file | 21:57 |
| ade_lee | fails for me with "Problem: The operation would result in removing the following protected packages: kernel-core" | 21:57 |
| mwhahaha | ade_lee: hjensas hit that, if the system that your building on has an older kernel | 21:57 |
| mwhahaha | ade_lee: update the local system to latest and reboot | 21:57 |
| ade_lee | ok | 21:57 |
| mwhahaha | (as you should always do first thing before installing the undercloud) | 21:58 |
| * hjensas needs to update the playbook to reboot ... and we probably want to merge https://review.opendev.org/713988 ? | 21:59 | |
| mwhahaha | i'm not aware of anyone from tripleo who merges stuff in dib | 22:00 |
| mwhahaha | unfortunately | 22:00 |
| mwhahaha | maybe nudge ian | 22:00 |
| * mwhahaha pokes ianw | 22:00 | |
| ianw | umm, i'm happy with that change | 22:01 |
| *** abishop_ has joined #tripleo | 22:02 | |
| *** suuuper has quit IRC | 22:05 | |
| *** abishop has quit IRC | 22:05 | |
| Madkiss | ahum the controlplane that i define in the undercloud -- is that the same one as the former Management network in the overcloud? | 22:09 |
| mwhahaha | Madkiss: it's the provisioning network, a management network is usually different. Though if you consider the provisioning network as maangement (because the undercloud manages the overcloud nodes via it) then yes | 22:10 |
| Madkiss | thing is, I only have one network where all nodes can have their ssh-login-interface (which is equivalent for me to the management network) in. | 22:12 |
| mwhahaha | then it'll need to be that network | 22:12 |
| Madkiss | i have a ctlplane-subnet-entry in undercloud.conf and now I wonder whether I need to define the Management network in network_data.yaml anyway, or whether there are predefined Ctlplane variables | 22:12 |
| mwhahaha | though not certain what network compute uses | 22:12 |
| mwhahaha | so not completely sure that'll work | 22:12 |
| Madkiss | hu? nova-compute requiring its own network would be a new one. | 22:13 |
| mwhahaha | let me check what network it uses for live migration by default | 22:13 |
| weshay|ruck | cloudnull, what's busted? http://zuul.openstack.org/builds?job_name=tripleo-buildimage-overcloud-full-centos-8 | 22:14 |
| weshay|ruck | this review? https://review.opendev.org/#/c/714629/ | 22:15 |
| weshay|ruck | cloudnull, show me what review and I'll look later, have to take my kids to the park | 22:16 |
| mwhahaha | Madkiss: so the libvirt live migration service uses the internal api or storage network by default (which also has an additional ssh). not certain if that conflicts with your bits because it's not the usual login ssh | 22:16 |
| cloudnull | weshay|ruck i didn't have a review in mind. I was attempting to build images locally, which seems to be busted. | 22:17 |
| mwhahaha | all these things are defined int he service net map so you can customize if you needed to | 22:17 |
| Madkiss | internal api and storage are both separate networks I define and that will be present. | 22:17 |
| mwhahaha | right but you said you had teh ssh limitation | 22:17 |
| mwhahaha | is that for managing the network externally? | 22:17 |
| mwhahaha | if so you can define a management network for that ssh seperate from the ctlplane | 22:17 |
| mwhahaha | if the undercloud can ssh to the nodes over the ctlplane still | 22:18 |
| Madkiss | oh, no, sorry, I misexpressed myself. I started with OpenStack in 2011, and back then, the "management network" simply was the main network where all traffic was going except for storage traffic ... | 22:18 |
| mwhahaha | you can move the traffic around to whatever networks you want | 22:18 |
| mwhahaha | the ctlplane is usually just the provisioning/monitoring bits | 22:18 |
| mwhahaha | you can then specify additional networks (internal_api/storage/etc) and move services to them | 22:19 |
| mwhahaha | it's controlled via the service net map in tripleo | 22:19 |
| mwhahaha | you can see the defaults in THT/network/service_net_map.j2.yaml | 22:20 |
| cloudnull | weshay|ruck this is from a recent run - https://pasted.tech/pastes/ae10727972980eefb1f6bfb39b4da2a42c80a637.raw - trying to build local images | 22:20 |
| *** gregwork has quit IRC | 22:21 | |
| cloudnull | weshay|ruck this is the stacktrace - https://pasted.tech/pastes/5dfb26974f02102764438170ac462c1edba1016d | 22:21 |
| Madkiss | mwhahaha: thank you -- perfect! | 22:21 |
| cloudnull | mwhahaha I tried re-running tripleo-repos to make sure all my repos were right, and I reversed the command args `openstack --os-cloud undercloud overcloud image build --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-images-python3.yaml --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-images-centos8.yaml` | 22:22 |
| cloudnull | sadly same result | 22:22 |
| mwhahaha | k let me retry | 22:22 |
| cloudnull | did you happen to have better luck? | 22:22 |
| cloudnull | ok | 22:22 |
| *** slaweq has quit IRC | 22:23 | |
| mwhahaha | your repos are still bad | 22:23 |
| mwhahaha | you can seem them in your log | 22:24 |
| mwhahaha | you forgot to export the repos bits | 22:24 |
| mwhahaha | the botom of https://pasted.tech/pastes/ae10727972980eefb1f6bfb39b4da2a42c80a637.raw shows you only have appstream/baseos/extras/centos7-rt | 22:24 |
| mwhahaha | export DIB_YUM_REPO_CONF="/etc/yum.repos.d/*" | 22:24 |
| mwhahaha | and rerun | 22:25 |
| Madkiss | one other question: documentation says "ctlplane interface is always nic1", but i want the ctlplane to be on a bond from ens1f0 and ens1f1. can I redefine that somewhere? | 22:25 |
| mwhahaha | Madkiss: you'd define that in your network config for the host | 22:25 |
| mwhahaha | i cannot help you with network nic configs | 22:25 |
| * mwhahaha avoids them like the plague | 22:25 | |
| Madkiss | mwhahaha: you mean the nic configs for the individual roles? | 22:26 |
| mwhahaha | yea | 22:26 |
| Madkiss | gotcha, thank you | 22:26 |
| * mwhahaha handwaves https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/features/network_isolation.html | 22:26 | |
| Madkiss | some light at the end of the tunnel after i had to find out that my beloved director UI has died. | 22:26 |
| ade_lee | mwhahaha, so my image build completed .. | 22:27 |
| cloudnull | mwhahaha running nw | 22:27 |
| *** rh-jelabarre has quit IRC | 22:27 | |
| ade_lee | mwhahaha, specifically -- openstack overcloud image build --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-images-centos8.yaml | 22:28 |
| mwhahaha | if only there was some sort of ansible collection that wrapped all this stuff for you | 22:28 |
| mwhahaha | and had an example playbook | 22:28 |
| mwhahaha | https://opendev.org/openstack/tripleo-operator-ansible/src/branch/master/playbooks/overcloud-images.yml | 22:28 |
| * mwhahaha coughs | 22:28 | |
| mwhahaha | line 55 needs tweaking tho | 22:28 |
| mwhahaha | cause it needs python3 | 22:29 |
| ade_lee | mwhahaha, cool - I | 22:30 |
| ade_lee | I'll try it -- I was following along what hjensas was doing .. | 22:30 |
| mwhahaha | i got the missing pacemaker bits cause it didn't find my ha repo either | 22:30 |
| mwhahaha | because i didn't specify /etc/yum.repo.d/* | 22:31 |
| ade_lee | I tried -- openstack overcloud image upload though -- and get the error -- Required file "./ironic-python-agent.initramfs" does not exist. | 22:31 |
| mwhahaha | if you specify only /etc/yum.repo.d/delorean* you get missing ha repos | 22:31 |
| ade_lee | ah .. | 22:32 |
| mwhahaha | that means you only build the overcloud image and not the ipa | 22:32 |
| ade_lee | well actually it created ironic-python-agent.qcow2 | 22:32 |
| mwhahaha | it shouldn't create a qcow2 for that | 22:33 |
| mwhahaha | i've got ironic-python-agent.{initramfs,kernel,vmlinuz} | 22:33 |
| ade_lee | hmm .. and yet it did .. | 22:34 |
| ade_lee | ronic-python-agent.d ironic-python-agent.log ironic-python-agent.qcow2 overcloud-full.d overcloud-full.log overcloud-full.qcow2 | 22:34 |
| * mwhahaha shrugs | 22:34 | |
| ade_lee | is whats in my images directory | 22:34 |
| mwhahaha | this is one of the oldest processes we have | 22:35 |
| mwhahaha | hasn't changed in years :D | 22:35 |
| mwhahaha | my images ar ebuild so we'll see | 22:35 |
| mwhahaha | are building | 22:35 |
| *** lbragstad has quit IRC | 22:38 | |
| ade_lee | yup - I see .. | 22:38 |
| ade_lee | 2020-03-24 22:16:06.303 | Converting image using qemu-img convert | 22:38 |
| ade_lee | 2020-03-24 22:17:15.633 | Image file ./ironic-python-agent.qcow2 created... | 22:38 |
| ade_lee | at the end of my build | 22:38 |
| cloudnull | hum... https://pasted.tech/pastes/0f5f928716f7245b720c162cdf4fc12512a13787.raw - it ran longer, but still seemed to fail :\ | 22:39 |
| ade_lee | mwhahaha, gotta dinner - but I'll check back in a bit -- thanks | 22:40 |
| mwhahaha | cloudnull: 2020-03-24 22:36:00.917 | Problem: The operation would result in removing the following protected packages: kernel-core | 22:40 |
| mwhahaha | cloudnull: update system and reboot | 22:40 |
| * mwhahaha broken record | 22:40 | |
| cloudnull | ? | 22:40 |
| cloudnull | I need to update the local system and reboot ? | 22:41 |
| mwhahaha | yes | 22:41 |
| cloudnull | is this the normal joys of DIB ? | 22:41 |
| mwhahaha | well you should always update your undercloud before you install after you configure the repos so you don't hit this | 22:41 |
| mwhahaha | but sure | 22:41 |
| mwhahaha | if you see teh conversation from about an hour ago with ade_lee, you can see why | 22:42 |
| openstackgerrit | Merged openstack/diskimage-builder master: Use rpm -e instead of dnf for cleaning old kernels https://review.opendev.org/713988 | 22:42 |
| mwhahaha | specifically because -^ | 22:43 |
| cloudnull | this was an already deployed cloud, which I was trying to build new images on just for grins. | 22:43 |
| cloudnull | is there a proceedure to get updated images without having to update/reboot the undercloud ? | 22:43 |
| mwhahaha | yea it's a bug with dnf | 22:43 |
| ade_lee | cloudnull, mwhahaha though in my case, my undercloud was already updated, but I was running an old kernel. had to modify grub to boot into newer one. | 22:43 |
| cloudnull | oh ok | 22:43 |
| mwhahaha | it popped up in fedora a long while ago but never hit centos7 | 22:44 |
| mwhahaha | because yum vs dnf | 22:44 |
| cloudnull | TIL | 22:44 |
| mwhahaha | https://bugs.launchpad.net/diskimage-builder/+bug/1623409 | 22:44 |
| openstack | Launchpad bug 1623409 in diskimage-builder "Error: The operation would result in removing the booted kernel" [High,Fix released] - Assigned to Harald Jensås (harald-jensas) | 22:44 |
| mwhahaha | Lucas Alvares Gomes (lucasagomes) on 2016-09-14 | 22:44 |
| mwhahaha | so yea takes about 4 years to fix stuff | 22:44 |
| cloudnull | lol | 22:44 |
| *** rcernin has joined #tripleo | 22:45 | |
| mwhahaha | also you usually wouldn't build iamges w/o having updated your undercloud | 22:45 |
| mwhahaha | the process should be update undercloud to latest and then build | 22:46 |
| mwhahaha | especially when dealing with upstream because dlrn repos may go away | 22:46 |
| mwhahaha | so you'd want the latest ones | 22:46 |
| cloudnull | OIC - this makes sense | 22:46 |
| *** tkajinam has joined #tripleo | 22:53 | |
| mwhahaha | yea worked for me | 22:56 |
| mwhahaha | don't have an ipa .qcow2 either | 22:56 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: WIP: Remove connections added by Network Manager https://review.opendev.org/714777 | 22:56 |
| mwhahaha | 99-ramdisk-create should ahve generated all the ramdisk bits | 22:56 |
| mwhahaha | so the key is: export DIB_YUM_REPO_CONF="/etc/yum.repos.d/*" | 22:58 |
| mwhahaha | openstack overcloud image build --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-images-python3.yaml --config-file /usr/share/openstack-tripleo-common/image-yaml/overcloud-images-centos8.yaml | 22:59 |
| mwhahaha | ade_lee: you might try flipping the config-file ordering to ensure python3 is specified first followed by centos8 | 22:59 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: Update image URL links for CentOS 8 https://review.opendev.org/714790 | 23:03 |
| rlandy | weshay|ruck: ^^ | 23:03 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Add centos/rhel based interface/bridge commands https://review.opendev.org/714208 | 23:04 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: WIP: Remove connections added by Network Manager https://review.opendev.org/714777 | 23:05 |
| openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient master: Switch default images to py3 https://review.opendev.org/714791 | 23:07 |
| cloudnull | mwhahaha thanks for the help . I learned a bit more and it works now. :D | 23:09 |
| mwhahaha | cloudnull: k you're now the expert | 23:09 |
| * mwhahaha runs away | 23:10 | |
| * cloudnull hold my beer | 23:10 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-operator-ansible master: Fix image build for python3 https://review.opendev.org/714793 | 23:13 |
| * mwhahaha fixes the defaults and runs away | 23:13 | |
| *** chem has quit IRC | 23:29 | |
| *** lbragstad has joined #tripleo | 23:46 | |
| *** mcornea has quit IRC | 23:47 | |
| openstackgerrit | Merged openstack/python-tripleoclient master: Add mistral workflow module path to ansible-runner https://review.opendev.org/714493 | 23:48 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: DCN/Edge: Handle ipv6 address for local glance endpoint https://review.opendev.org/714482 | 23:48 |
| *** rlandy has quit IRC | 23:57 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!