| cloudnull | mwhahaha stevebaker sshnaidm|afk is there a doc on deploying nova-less ? | 00:00 |
|---|---|---|
| * cloudnull would like to setup a local test env | 00:01 | |
| mwhahaha | I think it's in the docs | 00:01 |
| stevebaker | cloudnull: https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/provisioning/baremetal_provision.html | 00:01 |
| EmilienM | it sounds like a friday project :D | 00:01 |
| EmilienM | Tengu: and a patch in your tripleo-lab :P | 00:02 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/train: Force facts cache refreshing after OS upgrade. https://review.opendev.org/708343 | 00:02 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/train: Add swiftoperator role on ceph-rgw template https://review.opendev.org/708627 | 00:08 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/train: tripleo_container_manage: set tripleo_container_manage_check_puppet_config https://review.opendev.org/708619 | 00:09 |
| openstackgerrit | Merged openstack/tripleo-upgrade master: Configure undercloud_enable_paunch when set https://review.opendev.org/708241 | 00:09 |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Disable paunch on fs050 https://review.opendev.org/708256 | 00:09 |
| *** ade_lee has joined #tripleo | 00:11 | |
| *** jmasud has quit IRC | 00:11 | |
| *** jmasud has joined #tripleo | 00:14 | |
| *** stevebaker has quit IRC | 00:36 | |
| *** ekultails has left #tripleo | 00:56 | |
| openstackgerrit | Merged openstack/python-tripleoclient master: Remove mistral from the list_available_roles deploy workflow https://review.opendev.org/708276 | 01:01 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Remove mistral from the create_container deploy workflow https://review.opendev.org/708279 | 01:01 |
| *** stevebaker has joined #tripleo | 01:15 | |
| cloudnull | stevebaker ping - i've been looking into https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/provisioning/baremetal_provision.html - but I don't see where I'd define node ipmi setting. where might that go? | 01:21 |
| cloudnull | and I was looking at the logs https://logserver.rdoproject.org/58/707658/5/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/7734c27/logs/undercloud/home/zuul/ | 01:21 |
| cloudnull | and i didn't see anything that seemed to standout | 01:22 |
| stevebaker | cloudnull: isn't that something which is determined for the ironic node during node import? At node provision time, all nodes should be in an available state with things like ipmi already set up | 01:25 |
| cloudnull | oh , so we still need the instackenv ? | 01:25 |
| cloudnull | and need to do node import and introspection ? | 01:26 |
| openstackgerrit | Merged openstack/tripleo-common master: Set PROFILE_TASKS_TASK_OUTPUT_LIMIT https://review.opendev.org/706550 | 01:29 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/stein: Swap tasks to avoid non-existing variable error. https://review.opendev.org/705650 | 01:29 |
| stevebaker | cloudnull: yes, this just adds an extra provision step to boot the nodes, instead of having it done by heat->nova->ironic | 01:29 |
| cloudnull | ah i see | 01:29 |
| cloudnull | cool , thanks! | 01:29 |
| stevebaker | cloudnull: that doc page is at the end of the baremetal section, which all still applies https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/provisioning/index.html | 01:30 |
| cloudnull | and the ssh key, you said that's default to ~/.ssh/id_rsa | 01:30 |
| cloudnull | ? | 01:30 |
| cloudnull | ok this makes more sense now, thanks | 01:30 |
| stevebaker | cloudnull: yes, unless you specify openstack overcloud node provision --overcloud-ssh-key ... | 01:31 |
| cloudnull | in the most recent job log -https://logserver.rdoproject.org/58/707658/5/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/7734c27/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz I see the key being set to : | 01:31 |
| cloudnull | 2020-02-19 22:46:11 | Using ssh user "heat-admin" for initial connection. | 01:31 |
| cloudnull | 2020-02-19 22:46:11 | Using ssh key at "/home/zuul/.ssh/id_rsa_tripleo" for initial connection. | 01:31 |
| cloudnull | which would be the key used when none is defined and its auto-discovered | 01:31 |
| cloudnull | looks like we just run - openstack overcloud node provision --stack overcloud --output /home/zuul/overcloud-baremetal-deployed.yaml /home/zuul/overcloud_baremetal_deploy.yaml | 01:32 |
| stevebaker | cloudnull: oh. the provision command needs the exact same auto-discover logic, whatever that is | 01:33 |
| cloudnull | before that was just pulling from mistral , so we'll need to update that | 01:34 |
| stevebaker | parser.add_argument( | 01:35 |
| stevebaker | '--overcloud-ssh-key', | 01:35 |
| stevebaker | default=os.path.join( | 01:35 |
| stevebaker | os.path.expanduser('~'), '.ssh', 'id_rsa_tripleo'), | 01:35 |
| stevebaker | help=_('Key path for ssh access to overcloud nodes.') | 01:35 |
| stevebaker | ) | 01:35 |
| openstackgerrit | Merged openstack/tripleo-ansible master: tripleo_container_manage: disable logging for tasks using container_data https://review.opendev.org/708739 | 01:36 |
| cloudnull | so that default is the key used when the node is provisioned ? | 01:37 |
| cloudnull | https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L603-L625 | 01:38 |
| cloudnull | which looks like its used there? | 01:38 |
| stevebaker | cloudnull: I think so yes. Also the provision command defaults to a public key, but the deploy command uses the private. So thats wrong too | 01:39 |
| cloudnull | so really, os.path.expanduser('~'), '.ssh', 'id_rsa_tripleo'), just needs to be set to the pub key ? | 01:40 |
| cloudnull | id_rsa_tripleo.pub | 01:41 |
| stevebaker | cloudnull: I'm assuming the other way around, the provision command needs to default to the private id_rsa_tripleo | 01:41 |
| cloudnull | oh their different - https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_node.py#L619-L623 | 01:43 |
| cloudnull | looks like this is the function being used | 01:43 |
| *** rfolco has quit IRC | 01:44 | |
| cloudnull | and it seems that switch is used for different purposes across commands | 01:44 |
| cloudnull | both private and public keys, in different methods | 01:45 |
| cloudnull | but the use of baremetal.deploy_roles across modules, seems to expect different things ? | 01:46 |
| cloudnull | https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_node.py#L635-L639 vs https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L614-L618 | 01:48 |
| cloudnull | the parsed arg between the two modules for parsed_args.overcloud_ssh_user is different | 01:48 |
| stevebaker | cloudnull: yes they are. what does get_hosts_and_enable_ssh_admin do with parsed_args.overcloud_ssh_key? | 01:52 |
| cloudnull | it pulls a list of hosts and then runs the enable ssh admin workflow | 01:53 |
| cloudnull | which creates the tripleo-admin user | 01:53 |
| stevebaker | cloudnull: using that key for initial access? | 01:54 |
| cloudnull | yes | 01:54 |
| stevebaker | cloudnull: ok, here is what I think needs to happen. 1) provision command needs to default to id_rsa_tripleo.pub https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_node.py#L621 | 01:55 |
| stevebaker | cloudnull: 2) overcloud deploy _provision_baremetal needs to append '.pub' to parsed_args.overcloud_ssh_key (this code path is *only* hit if you want to do provision and deploy in one go. we're calling provision seperately) | 01:56 |
| stevebaker | https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L611 | 01:56 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: placement: Remove Nova to Placement extraction step https://review.opendev.org/703804 | 01:59 |
| cloudnull | ok | 02:00 |
| stevebaker | cloudnull: heh, in my zuul generated node id_rsa and id_rsa_tripleo are the same key | 02:00 |
| cloudnull | yea they should be one in the same | 02:00 |
| cloudnull | which is confusing | 02:00 |
| openstackgerrit | Steve Baker proposed openstack/python-tripleoclient master: Use correct default key file for baremetal provision https://review.opendev.org/708781 | 02:04 |
| stevebaker | cloudnull: its a needed fix, but it may not fix your problem | 02:04 |
| cloudnull | you beat me to it :D | 02:06 |
| cloudnull | do we need to set that pub key entry here too https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L611-L612 | 02:06 |
| cloudnull | https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_node.py#L632-L633 | 02:06 |
| cloudnull | looks like the same code is in two palces. | 02:06 |
| cloudnull | **places | 02:06 |
| stevebaker | cloudnull: for that command the argument is documented as being the public key, I've change the default to id_rsa_tripleo.pub instead | 02:08 |
| cloudnull | ok | 02:08 |
| cloudnull | seems off that the same switch would have a different value and use the same underlying functions? | 02:09 |
| cloudnull | https://pasted.tech/pastes/20e4bb6026b7158afc110ece2dd893ddf5758636.raw | 02:10 |
| cloudnull | would normalize the options, and should have the desired effect? | 02:10 |
| cloudnull | OK i gotta unplug for the evening. thanks for you help on this stuff stevebaker , i'll give this metalsmith things a better spin tomorrow | 02:14 |
| stevebaker | cloudnull: yes, given that provision is a new command I'm fine with switching to a private key to make it consistent with other commands. A subset of that change would need to be backported to stable/train though, which is fine | 02:14 |
| cloudnull | ++ | 02:14 |
| stevebaker | cloudnull: thanks very much, I'll abandon my change for yours | 02:14 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/python-tripleoclient master: Use correct default key file for baremetal provision https://review.opendev.org/708781 | 02:15 |
| cloudnull | ^ i just pushed to yours | 02:16 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-ansible master: Improve execution and add a port check https://review.opendev.org/707658 | 02:17 |
| cloudnull | ^ see if that makes fs001 happier | 02:17 |
| cloudnull | ok, now I go away for real. | 02:17 |
| stevebaker | sweet | 02:17 |
| stevebaker | bye | 02:17 |
| cloudnull | have a great night :D | 02:17 |
| stevebaker | still day here :) | 02:17 |
| cloudnull | thanks again , I appreciate you answering my n00b questions. | 02:17 |
| cloudnull | expect more soon :D | 02:17 |
| openstackgerrit | Steve Baker proposed openstack/python-tripleoclient master: Use correct default key file for baremetal provision https://review.opendev.org/708781 | 02:20 |
| stevebaker | i just updated the commit message | 02:21 |
| *** mschuppert has quit IRC | 02:29 | |
| *** Goneri has quit IRC | 02:34 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-upgrade stable/train: Configure undercloud_enable_paunch when set https://review.opendev.org/708790 | 02:47 |
| *** openstackstatus has joined #tripleo | 02:56 | |
| *** ChanServ sets mode: +v openstackstatus | 02:56 | |
| *** rlandy|bbl is now known as rlandy | 03:03 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: pip install contextlib2 https://review.opendev.org/708791 | 03:13 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: ensure sudo is not called in ci.centos https://review.opendev.org/708264 | 03:14 |
| openstackgerrit | David Vallee Delisle proposed openstack/os-net-config master: We need to use the MAC instead of PCI Address for Mellanox cards https://review.opendev.org/706429 | 03:16 |
| *** psachin has joined #tripleo | 03:33 | |
| *** ykarel|away is now known as ykarel | 03:34 | |
| *** lbragstad has quit IRC | 03:39 | |
| *** stevebaker has quit IRC | 03:40 | |
| *** evrardjp has quit IRC | 03:42 | |
| *** evrardjp has joined #tripleo | 03:45 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/python-tripleoclient master: Use correct default key file for baremetal provision https://review.opendev.org/708781 | 03:54 |
| openstackgerrit | Sagi Shnaidman proposed openstack/python-tripleoclient master: Use correct default key file for baremetal provision https://review.opendev.org/708781 | 03:58 |
| sshnaidm|afk | cloudnull, oops.. | 03:58 |
| openstackgerrit | Sagi Shnaidman proposed openstack/python-tripleoclient master: Use correct default key file for baremetal provision https://review.opendev.org/708781 | 04:00 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: Improve execution and add a port check https://review.opendev.org/707658 | 04:01 |
| sshnaidm|afk | cloudnull, returned back your patch | 04:02 |
| *** jmasud has quit IRC | 04:12 | |
| *** jmasud has joined #tripleo | 04:14 | |
| *** udesale has joined #tripleo | 04:20 | |
| *** ramishra has joined #tripleo | 04:29 | |
| *** rlandy has quit IRC | 04:41 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ci master: DNM: test SA on c8 dry run https://review.opendev.org/708800 | 04:48 |
| *** ramishra has quit IRC | 04:58 | |
| *** saneax has joined #tripleo | 05:01 | |
| *** ramishra has joined #tripleo | 05:06 | |
| openstackgerrit | Rabi Mishra proposed openstack/tripleo-common master: Move plan container creation to utils https://review.opendev.org/708802 | 05:11 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ci master: Add centos8 dry run job for CI https://review.opendev.org/708803 | 05:12 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ci master: Add centos8 dry run job for CI https://review.opendev.org/708803 | 05:15 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ci master: DNM: test SA on c8 dry run https://review.opendev.org/708800 | 05:17 |
| *** ccamacho has quit IRC | 05:17 | |
| *** mmedvede has quit IRC | 05:17 | |
| *** mmedvede has joined #tripleo | 05:18 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart master: Handle pip being installed as /usr/bin/pip3 https://review.opendev.org/706979 | 05:21 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Use tripleo-operator-ansible for standalone https://review.opendev.org/705775 | 05:26 |
| *** evrardjp has quit IRC | 05:34 | |
| *** evrardjp has joined #tripleo | 05:34 | |
| openstackgerrit | Merged openstack/tripleo-ansible stable/train: tripleo_container_manage: disable logging for tasks using container_data https://review.opendev.org/708754 | 05:38 |
| *** ade_lee has quit IRC | 06:00 | |
| *** ade_lee has joined #tripleo | 06:00 | |
| *** udesale_ has joined #tripleo | 06:03 | |
| *** ratailor has joined #tripleo | 06:04 | |
| *** ratailor has quit IRC | 06:05 | |
| *** udesale has quit IRC | 06:06 | |
| *** ratailor has joined #tripleo | 06:07 | |
| *** marios|ruck has joined #tripleo | 06:08 | |
| *** lmiccini has joined #tripleo | 06:19 | |
| *** yolanda has joined #tripleo | 06:22 | |
| *** ramishra has quit IRC | 06:32 | |
| *** ramishra has joined #tripleo | 06:32 | |
| *** ahosam has joined #tripleo | 06:33 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: Use CI mirrors for molecule jobs https://review.opendev.org/708808 | 06:47 |
| *** jfrancoa has joined #tripleo | 06:49 | |
| *** abdysn has joined #tripleo | 06:51 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: DNM: test deps https://review.opendev.org/708809 | 06:53 |
| sshnaidm|afk | marios|ruck, fyi https://review.opendev.org/#/c/708803/ | 06:58 |
| marios|ruck | ack sshnaidm|afk | 07:00 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-ci master: [WIP] Improve build images role for CentOS8 https://review.opendev.org/708599 | 07:15 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: Use CI mirrors for molecule jobs https://review.opendev.org/708808 | 07:21 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: Use CI mirrors for molecule jobs https://review.opendev.org/708808 | 07:22 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: DNM: test deps https://review.opendev.org/708809 | 07:22 |
| *** dpawlik has joined #tripleo | 07:31 | |
| *** paramite has joined #tripleo | 07:36 | |
| *** ratailor has quit IRC | 07:40 | |
| *** ratailor has joined #tripleo | 07:41 | |
| *** slaweq has joined #tripleo | 07:51 | |
| *** jpena|off is now known as jpena | 07:51 | |
| *** jbadiapa has joined #tripleo | 07:52 | |
| *** ykarel is now known as ykarel|lunch | 08:01 | |
| *** holser has joined #tripleo | 08:02 | |
| *** tesseract has joined #tripleo | 08:05 | |
| *** jmasud has quit IRC | 08:12 | |
| *** jmasud has joined #tripleo | 08:14 | |
| Tengu | hello there | 08:25 |
| Tengu | EmilienM: wow. yep, indeed, that's a Friday Patch for tripleo-lab I think :). | 08:25 |
| *** rpittau|afk is now known as rpittau | 08:26 | |
| *** tosky has joined #tripleo | 08:29 | |
| *** iurygregory has joined #tripleo | 08:31 | |
| *** ccamacho has joined #tripleo | 08:32 | |
| *** bogdando has joined #tripleo | 08:35 | |
| *** tesseract has quit IRC | 08:38 | |
| *** tesseract has joined #tripleo | 08:39 | |
| openstackgerrit | Juan Badia Payno proposed openstack/tripleo-ansible master: BaR Rename the mysql_short_bootstrap_name https://review.opendev.org/708819 | 08:39 |
| *** amoralej|off is now known as amoralej | 08:40 | |
| *** gfidente has joined #tripleo | 08:41 | |
| *** jpich has joined #tripleo | 08:51 | |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/tripleo-common master: Move away from "ss" and drop default verbose mode https://review.opendev.org/708339 | 08:54 |
| *** avivgt has joined #tripleo | 08:55 | |
| *** pkopec has joined #tripleo | 09:01 | |
| *** shardy has joined #tripleo | 09:04 | |
| *** holser has quit IRC | 09:08 | |
| *** mschuppert has joined #tripleo | 09:08 | |
| *** holser has joined #tripleo | 09:08 | |
| openstackgerrit | Juan Badia Payno proposed openstack/tripleo-ansible master: BaR Rename the mysql_short_bootstrap_name https://review.opendev.org/708819 | 09:09 |
| *** lucasagomes has joined #tripleo | 09:09 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-common master: Added overcloud images yaml for CentOS-8 https://review.opendev.org/708606 | 09:12 |
| *** ykarel|lunch is now known as ykarel | 09:13 | |
| *** suuuper has joined #tripleo | 09:16 | |
| *** tkajinam has quit IRC | 09:19 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-quickstart-extras master: Revert "Add tempest.api.compute.images test_delete_saving_image to skip" https://review.opendev.org/706817 | 09:25 |
| *** derekh has joined #tripleo | 09:29 | |
| chandankumar | cgoncalves, thanks for working on c8 support in dib | 09:39 |
| chandankumar | https://review.opendev.org/#/c/684308/ | 09:39 |
| chandankumar | cgoncalves, I was looking for that | 09:39 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-ci master: [WIP] Improve build images role for CentOS8 https://review.opendev.org/708599 | 09:40 |
| *** xek_ has joined #tripleo | 09:46 | |
| *** StevenK has quit IRC | 09:52 | |
| *** StevenK has joined #tripleo | 09:52 | |
| openstackgerrit | Juan Badia Payno proposed openstack/tripleo-ansible master: BaR Rename the mysql_short_bootstrap_name https://review.opendev.org/708819 | 09:54 |
| *** tesseract has quit IRC | 09:58 | |
| *** jaosorior has joined #tripleo | 09:59 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-common master: Added overcloud images yaml for CentOS-8 https://review.opendev.org/708606 | 10:02 |
| openstackgerrit | Abdallah Yasin proposed openstack/os-net-config master: Adding IPoIB pkeys interfaces to the os-net-config https://review.opendev.org/707331 | 10:03 |
| *** alexmcleod has joined #tripleo | 10:07 | |
| *** tesseract has joined #tripleo | 10:11 | |
| openstackgerrit | Francesco Pantano proposed openstack/tripleo-heat-templates master: Fix dcn-hci resource mapping path https://review.opendev.org/708838 | 10:19 |
| openstackgerrit | Francesco Pantano proposed openstack/tripleo-heat-templates master: Fix dcn-hci resource mapping path https://review.opendev.org/708838 | 10:38 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-ci master: Pin kolla venv and package install workaround py2 queue bug https://review.opendev.org/708846 | 10:38 |
| marios|ruck | ykarel: chandankumar: yuck ^^ | 10:38 |
| marios|ruck | ykarel: chandankumar: testing https://review.rdoproject.org/r/#/c/25080/ | 10:41 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: Use CI mirrors for molecule jobs https://review.opendev.org/708808 | 10:43 |
| *** pbandark has joined #tripleo | 10:43 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: DNM: test deps https://review.opendev.org/708809 | 10:43 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-ci master: [WIP] Improve build images role for CentOS8 https://review.opendev.org/708599 | 10:44 |
| ykarel | marios|ruck, no need to pin package | 10:44 |
| ykarel | rpm | 10:44 |
| ykarel | it will be get from whatever is in repo | 10:44 |
| ykarel | for repo u already proposed pin in rdoinfo, that will take care of it | 10:44 |
| openstackgerrit | Merged openstack/tripleo-ci master: Fix scenario010 triggers https://review.opendev.org/708722 | 10:45 |
| marios|ruck | ykarel: k i thought we wanted to pin there too.. will update after the test runs. also can make the git chckout conditional for py2 | 10:46 |
| marios|ruck | ykarel: chandankumar: thanks | 10:46 |
| openstackgerrit | Martin Kopec proposed openstack/ansible-role-collect-logs master: Dynamically generate a list of files to collect for rsync https://review.opendev.org/703765 | 10:50 |
| *** pbandark has quit IRC | 11:01 | |
| *** shardy has quit IRC | 11:02 | |
| *** udesale_ has quit IRC | 11:03 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-ci master: [WIP] Improve build images role for CentOS8 https://review.opendev.org/708599 | 11:03 |
| *** shardy has joined #tripleo | 11:03 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-ci master: Pin kolla venv install workaround py2 queue import error https://review.opendev.org/708846 | 11:04 |
| *** rpittau is now known as rpittau|bbl | 11:20 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: WIP: try podman driver https://review.opendev.org/708851 | 11:23 |
| *** sshnaidm|afk is now known as sshnaidm | 11:23 | |
| *** ykarel is now known as ykarel|afk | 11:24 | |
| *** jaosorior has quit IRC | 11:30 | |
| *** ansmith has joined #tripleo | 11:33 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: WIP: try podman driver https://review.opendev.org/708851 | 11:38 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ci master: DNM: c8 test https://review.opendev.org/708855 | 11:41 |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-ci master: [WIP] Improve build images role for CentOS8 https://review.opendev.org/708599 | 11:42 |
| *** ansmith has quit IRC | 11:44 | |
| openstackgerrit | Michal Pryc proposed openstack/tripleo-ansible master: DNM - check ansible-playbook-3 on rhel8 https://review.opendev.org/708856 | 11:48 |
| *** dtantsur|afk is now known as dtantsur | 11:51 | |
| *** rfolco has joined #tripleo | 11:52 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: NodeDataLookup utility should rely on python env https://review.opendev.org/708715 | 11:54 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: WIP: try podman driver https://review.opendev.org/708851 | 11:54 |
| *** jpena is now known as jpena|lunch | 12:00 | |
| *** amoralej is now known as amoralej|lunch | 12:01 | |
| chandankumar | cgoncalves, Hello, around? | 12:07 |
| *** ccamacho has quit IRC | 12:14 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: WIP: try podman driver https://review.opendev.org/708851 | 12:24 |
| *** ratailor has quit IRC | 12:26 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ci master: DNM: c8 test https://review.opendev.org/708855 | 12:26 |
| openstackgerrit | Francesco Pantano proposed openstack/tripleo-heat-templates master: Fix dcn-hci resource mapping path https://review.opendev.org/708838 | 12:28 |
| *** udesale has joined #tripleo | 12:34 | |
| *** beagles is now known as beagles_pto | 12:35 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: Improve logic in introspection module https://review.opendev.org/707286 | 12:36 |
| *** social has joined #tripleo | 12:45 | |
| *** numans has quit IRC | 12:49 | |
| *** psachin has quit IRC | 12:49 | |
| *** rlandy has joined #tripleo | 12:51 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-upgrade stable/train: Ensure bc package is installed for l3 ping tests. https://review.opendev.org/708862 | 12:51 |
| *** shardy has quit IRC | 12:55 | |
| *** shardy has joined #tripleo | 12:56 | |
| abdysn | Tengu: hi, it seams https://review.opendev.org/#/c/705634/ is not added to the gate? does it need a recheck? | 12:57 |
| Tengu | abdysn: hmmm.... depends-on has merged, should have been triggered | 12:57 |
| *** morazi has joined #tripleo | 12:58 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-heat-templates stable/stein: Force facts cache refreshing after OS upgrade. https://review.opendev.org/708863 | 12:58 |
| *** raildo has joined #tripleo | 12:58 | |
| Tengu | hmm... running recheck might drop the +1 from zuul in case we hit any weird things. | 12:59 |
| Tengu | comme on gate...... | 12:59 |
| Tengu | ah | 12:59 |
| Tengu | abdysn: starting gate. | 12:59 |
| Tengu | abdysn: I just added a second +W to trigger it. | 12:59 |
| *** sshnaidm has quit IRC | 13:00 | |
| *** sshnaidm has joined #tripleo | 13:01 | |
| *** rh-jelabarre has joined #tripleo | 13:03 | |
| abdysn | Tengu: Great :) | 13:04 |
| *** rpittau|bbl is now known as rpittau | 13:07 | |
| *** artom has quit IRC | 13:08 | |
| abdysn | Tengu: oh right, do you by chance know anyone who can provide the specs to build the haproxy container? i mean the haproxy version and configuration? | 13:09 |
| Tengu | abdysn: hm, that's all in kolla iirc | 13:09 |
| abdysn | ah, should ask there then :) thanks. | 13:10 |
| Tengu | abdysn: so you have a partial answer here: https://opendev.org/openstack/tripleo-common/src/branch/master/container-images/overcloud_containers.yaml.j2#L261 | 13:10 |
| Tengu | showing the possible overrides we're making | 13:11 |
| abdysn | Thanks :) | 13:11 |
| Tengu | abdysn: and here's the kolla thing: https://opendev.org/openstack/kolla/src/branch/master/docker/haproxy | 13:12 |
| *** marios|ruck is now known as marios|ruck|call | 13:12 | |
| *** ysandeep is now known as ysandeep|away | 13:13 | |
| *** mcornea has joined #tripleo | 13:14 | |
| abdysn | Tengu: great, appreciate your help :). we are trying to change the openssl version on the haproxy, but it seam that the haproxy is tightly coupled with the installed haproxy, so we want to try to build a new haproxy image that have what we want :). | 13:16 |
| *** ccamacho has joined #tripleo | 13:17 | |
| Tengu | abdysn: you might want to talk to bandini about that. | 13:19 |
| abdysn | great, will do :) | 13:20 |
| *** jpena|lunch is now known as jpena | 13:24 | |
| *** lbragstad has joined #tripleo | 13:26 | |
| *** amoralej|lunch is now known as amoralej | 13:28 | |
| *** pbandark has joined #tripleo | 13:30 | |
| *** paramite|clone has joined #tripleo | 13:33 | |
| *** ansmith has joined #tripleo | 13:34 | |
| *** paramite has quit IRC | 13:34 | |
| cloudnull | mornings | 13:35 |
| *** psachin has joined #tripleo | 13:37 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates stable/train: Remove BlockStorageCinderVolume service from certain DCN roles https://review.opendev.org/708878 | 13:47 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-ansible master: paunch: fail early if containers weren't deployed by Paunch before https://review.opendev.org/708880 | 13:52 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/paunch stable/train: Fallback to a rm -f action for podman https://review.opendev.org/708881 | 13:54 |
| *** Goneri has joined #tripleo | 13:55 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-ansible master: Upgrade molecule to 3.0a9 https://review.opendev.org/708026 | 14:01 |
| openstackgerrit | Sorin Sbarnea proposed openstack/ansible-role-collect-logs master: Upgrade to molecule 3.0 https://review.opendev.org/708020 | 14:02 |
| *** abdysn has quit IRC | 14:03 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-ansible master: Upgrade molecule to 3.0 https://review.opendev.org/708026 | 14:04 |
| *** marios|ruck|call is now known as marios|ruck | 14:06 | |
| *** ysandeep|away is now known as ysandeep | 14:09 | |
| openstackgerrit | Juan Badia Payno proposed openstack/tripleo-ansible master: BaR Rename the mysql_short_bootstrap_name https://review.opendev.org/708819 | 14:12 |
| *** marios|ruck is now known as marios|ruck|afk | 14:13 | |
| chem | matbu: around ? | 14:14 |
| openstackgerrit | Merged openstack/tripleo-validations master: Add basic molecule tests for uncovered roles https://review.opendev.org/704158 | 14:17 |
| EmilienM | cloudnull: hey | 14:17 |
| cloudnull | yo | 14:18 |
| EmilienM | cloudnull: have you already seen a race where | 14:18 |
| EmilienM | cloudnull: you create a systemd unit file with template module and right after you enable that service | 14:18 |
| EmilienM | and it fails to find the unit file | 14:18 |
| EmilienM | i can see it in a CI job | 14:18 |
| EmilienM | cloudnull: http://paste.openstack.org/show/789798/ | 14:19 |
| EmilienM | with more logs http://paste.openstack.org/show/789799/ | 14:21 |
| cloudnull | EmilienM nope , ive not seen thta | 14:21 |
| cloudnull | caused by async tasks maybe ? | 14:22 |
| EmilienM | no, I made the systemd tasks not using async for that reason | 14:22 |
| EmilienM | to avoid systemd races | 14:22 |
| *** ykarel|afk is now known as ykarel | 14:23 | |
| *** marios|ruck|afk is now known as marios|ruck | 14:25 | |
| cloudnull | i cant think of why there would be a race there then | 14:25 |
| cloudnull | maybe that service was just not deployed ? | 14:25 |
| *** leanderthal has joined #tripleo | 14:25 | |
| EmilienM | unfortunately, I can't check if the systemd file actually exists, because CI logs don't collect it | 14:26 |
| EmilienM | cloudnull: the service is being upgraded: | 14:26 |
| EmilienM | https://b7fcf12d6809cf4f87e3-e088871e0bec453fba7cc9495a9471da.ssl.cf1.rackcdn.com/708790/1/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/9dd54ef/logs/undercloud/home/zuul/undercloud_upgrade.log | 14:26 |
| openstackgerrit | Grzegorz Grasza proposed openstack/tripleo-ansible master: Make sure not to re-enroll already enrolled server https://review.opendev.org/704404 | 14:27 |
| openstackgerrit | Grzegorz Grasza proposed openstack/tripleo-ansible master: Added files to allow cleanup when stack delete is called https://review.opendev.org/706588 | 14:27 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-upgrade master: Modify overcloud upgrade run loop. https://review.opendev.org/707814 | 14:28 |
| cloudnull | EmilienM maybe the daemon hadn't been reloaded? | 14:30 |
| cloudnull | so it doesn't see the new service after the container-rm ? | 14:30 |
| openstackgerrit | Grzegorz Grasza proposed openstack/tripleo-ansible master: Add code to deregister nodes and services from freeipa https://review.opendev.org/700226 | 14:30 |
| openstackgerrit | Grzegorz Grasza proposed openstack/tripleo-ansible master: Make sure not to re-enroll already enrolled server https://review.opendev.org/704404 | 14:30 |
| openstackgerrit | Grzegorz Grasza proposed openstack/tripleo-ansible master: Added files to allow cleanup when stack delete is called https://review.opendev.org/706588 | 14:30 |
| *** ekultails has joined #tripleo | 14:32 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-ci master: collect-logs: collect tripleo related systemd unit files https://review.opendev.org/708889 | 14:32 |
| EmilienM | zbr: ^ is it the right place to add more files? | 14:32 |
| EmilienM | or should I patch ansible-role-collect-logs ? | 14:33 |
| EmilienM | sshnaidm: ^ | 14:33 |
| sshnaidm | EmilienM, yeah, it should be in tripleo-ci | 14:34 |
| EmilienM | thx | 14:34 |
| sshnaidm | EmilienM, no other changes needed | 14:34 |
| EmilienM | ack | 14:34 |
| *** jaosorior has joined #tripleo | 14:34 | |
| *** sshnaidm is now known as sshnaidm|afk | 14:34 | |
| zbr | yeah. | 14:36 |
| EmilienM | matbu: hey, what does the tripleo-ci-centos-7-scenario000-multinode-oooq-container-updates ? | 14:38 |
| zbr | EmilienM: sshnaidm|afk cloudnull ekultails : please read and comment on https://github.com/ansible-community/molecule/issues/2565 -- i want to make the mirror config more transparent. | 14:41 |
| *** marios|ruck is now known as marios|ruck|call | 14:42 | |
| EmilienM | zbr: awesome, would be a great addition indeed. | 14:42 |
| zbr | i really do not like the volume mount approach, maybe we can find a less intrusive way of doing it | 14:42 |
| zbr | most people do not know that molecule can also load a default molecule file, and is merging it with individual ones, that is another way to avoid copy/paste on each scenario. | 14:43 |
| zbr | apparently the volume mount is not so bad, seems to be ignored when file does not exist. still I will try to define it in a single place. | 14:45 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-operator-ansible master: Add update run role https://review.opendev.org/708744 | 14:45 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-docs master: Extend external Swift docs for Ceph RadosGW https://review.opendev.org/708892 | 14:48 |
| openstackgerrit | Grzegorz Grasza proposed openstack/tripleo-ansible master: Make sure not to re-enroll already enrolled server https://review.opendev.org/704404 | 14:48 |
| openstackgerrit | Grzegorz Grasza proposed openstack/tripleo-ansible master: Added files to allow cleanup when stack delete is called https://review.opendev.org/706588 | 14:48 |
| matbu | EmilienM: it's doing a noop update | 14:51 |
| EmilienM | matbu: ok so it's not updating containers right? | 14:51 |
| matbu | EmilienM: so it's just testing the framework | 14:51 |
| matbu | EmilienM: i think so, need to double check | 14:52 |
| matbu | EmilienM: for me the main interest is the validation of the python code mainly | 14:52 |
| EmilienM | ok | 14:52 |
| matbu | EmilienM: very usefull for the removal of the mistral actions for example | 14:52 |
| matbu | EmilienM: but i can check, but if so, i think just removed and redeployed the same container | 14:53 |
| matbu | EmilienM: because it's deploying master to master (or release to release) | 14:53 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/python-tripleoclient master: Use correct default key file and normalize the usage https://review.opendev.org/708781 | 14:53 |
| EmilienM | ok | 14:53 |
| EmilienM | matbu: is there a way to change an environment file or parameter before running the update? | 14:54 |
| matbu | EmilienM: in CI ? | 14:54 |
| *** TrevorV has joined #tripleo | 14:54 | |
| EmilienM | matbu: i'm trying to see how we can test the transition from paunch managed containers to ansible in a minor update job | 14:54 |
| matbu | EmilienM: like inject new env file ? | 14:54 |
| matbu | EmilienM: hm i think at the tripleo-upgrade level | 14:55 |
| EmilienM | ok, i'll take a look | 14:55 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-ci master: Remove centos-7 container for U https://review.opendev.org/708893 | 14:55 |
| matbu | EmilienM: feel free to query if need .. we are a bit in the dark of the scale lab this week , because we need to finish before losing the lab | 14:56 |
| bogdando | EmilienM: the similar situation was with upgrades from containers having always managed-by=paunch to custom values tripleo provides for new deployments | 14:56 |
| EmilienM | matbu: ack, thanks | 14:56 |
| bogdando | we had to add additional checks for paunch | 14:56 |
| bogdando | maybe the same approach for ansible may be taken | 14:57 |
| bogdando | to not consider those "orphants" as lost or unmanaged | 14:57 |
| ekultails | zbr: I left a comment on the GitHub issue. I like the idea and I feel like we could make something like that work with minimal effort. | 14:57 |
| *** abishop has joined #tripleo | 14:59 | |
| ade_lee | EmilienM, sorry -- I'm confused. according to https://review.opendev.org/#/c/701280/ we skip the scale_down tasks in the case that we have pre-provisioned nodes, right? | 15:01 |
| ade_lee | EmilienM, which is why my scale_down said "Ansible skipped." | 15:02 |
| EmilienM | bogdando: can you elaborate? i did some changes lately on how we treat orphans in tripleo_container_manage | 15:04 |
| *** jdandrea has joined #tripleo | 15:04 | |
| openstackgerrit | Merged openstack/tripleo-upgrade stable/stein: Fix ping loss detection https://review.opendev.org/708575 | 15:05 |
| *** jdandrea has quit IRC | 15:05 | |
| EmilienM | ade_lee: indeed | 15:06 |
| EmilienM | ade_lee: I'll look at it today if I can | 15:06 |
| *** jdandrea has joined #tripleo | 15:06 | |
| ade_lee | EmilienM, cool thanks | 15:06 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-ansible master: Add no_log to the podman login https://review.opendev.org/708895 | 15:07 |
| EmilienM | ade_lee: yeah now I remember. We talked about it with sbaker 6 weeks ago and some work need to happen to lookup the physical resources from the stack and get the IP/hostname in split stack case | 15:08 |
| bogdando | EmilienM: https://review.opendev.org/#/c/696589/ | 15:08 |
| bogdando | imagine the "managed-by: paunch" -> "managed-by: ansible" transition to follow that approach | 15:09 |
| bogdando | (as of Train containers will have managed-by tripleo-<RoleName> iirc) | 15:09 |
| bogdando | or they may have that after applied minor update for any release | 15:10 |
| openstackgerrit | Merged openstack/tripleo-operator-ansible master: Add update prepare role https://review.opendev.org/708742 | 15:10 |
| bogdando | anyway, ansible can take those legacy values into account EmilienM | 15:11 |
| *** social has quit IRC | 15:12 | |
| EmilienM | bogdando: in the role, I made it "easier" and static: https://github.com/openstack/tripleo-ansible/blob/57cc4d80758d99bb0f9e11f479ef5a413425cccf/tripleo_ansible/roles/tripleo_container_manage/tasks/podman/create.yml#L58 | 15:12 |
| EmilienM | i don't think we need to be able to change that | 15:12 |
| bogdando | right, I mean you can use managed_by values to identify paunch-managed containers | 15:13 |
| EmilienM | bogdando: and also check out https://github.com/openstack/tripleo-ansible/blob/74996fba58c2efa5dc8f03551200a245a71ff3f3/tripleo_ansible/ansible_plugins/filter/helpers.py#L131-L135 | 15:13 |
| bogdando | right... maybe I didn't get the problem then | 15:13 |
| EmilienM | bogdando: managed-by tripleo-<RoleName> will disappear | 15:14 |
| EmilienM | it'll be managed-by tripleo-ansible | 15:14 |
| bogdando | EmilienM: so what's the issue with "the transition from paunch managed containers to ansible in a minor update job"? | 15:16 |
| bogdando | aside of https://review.opendev.org/#/c/708880/ | 15:17 |
| EmilienM | bogdando: there is no issue so far. It's just that all containers previously managed by Paunch will be re-created | 15:17 |
| EmilienM | bogdando: the downtime should be minimal thanks to https://review.opendev.org/#/c/708247/ | 15:17 |
| bogdando | mhm, for upgrades, not updates? | 15:17 |
| EmilienM | https://review.opendev.org/#/c/708247/1/tripleo_ansible/roles/tripleo-container-manage/tasks/podman/create.yml | 15:17 |
| EmilienM | bogdando: updates as well, managed_by changed. | 15:18 |
| EmilienM | we could make it so it uses the same managed_by as before with paunch but to me it doesn't make sense to keep managed_by: "tripleo-{{ tripleo_role_name }}" | 15:19 |
| bogdando | I'm not sure changing paunch to ansible within minor updates is good idea | 15:19 |
| *** psachin has quit IRC | 15:22 | |
| openstackgerrit | Merged openstack/tripleo-upgrade master: Force error when "launch workload" task fails https://review.opendev.org/708661 | 15:25 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-docs master: Extend external Swift docs for Ceph RadosGW https://review.opendev.org/708892 | 15:26 |
| openstackgerrit | Juan Badia Payno proposed openstack/tripleo-ansible master: BaR Rename the mysql_short_bootstrap_name https://review.opendev.org/708819 | 15:33 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-upgrade stable/train: Configure undercloud_enable_paunch when set https://review.opendev.org/708790 | 15:40 |
| *** marios|ruck|call is now known as marios|ruck | 15:40 | |
| *** sshnaidm|afk is now known as sshnaidm | 15:43 | |
| *** jfrancoa has quit IRC | 15:44 | |
| *** artom has joined #tripleo | 15:47 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Generate /etc/hosts early on both under and overcloud https://review.opendev.org/705634 | 15:48 |
| *** jpich has quit IRC | 15:52 | |
| *** jpich has joined #tripleo | 15:52 | |
| *** ratailor has joined #tripleo | 15:56 | |
| *** bogdando has quit IRC | 15:57 | |
| sshnaidm | zbr, you comment here is incorrect: https://review.opendev.org/#/c/708808/ | 15:57 |
| openstackgerrit | Merged openstack/paunch stable/stein: Do not force remove containers https://review.opendev.org/704669 | 15:57 |
| sshnaidm | zbr, it's totally possible to run molecule jobs locally and anywhere | 15:57 |
| sshnaidm | zbr, you should try to run it locally before | 15:58 |
| openstackgerrit | Alex Schultz proposed openstack/ansible-role-container-registry master: Don't log login https://review.opendev.org/708902 | 15:59 |
| zbr | sshnaidm: yep, i discovered that the volume does not generate an error when file is missing. | 15:59 |
| sshnaidm | zbr, if you have better solution right now - please write it, if not - let's go with that approach | 15:59 |
| zbr | sshnaidm: i am working on one right now, involves https://github.com/ansible-community/molecule/pull/2566 and the linked bug. | 15:59 |
| sshnaidm | EmilienM, cloudnull mwhahaha please take a look: https://review.opendev.org/#/c/708808/ | 16:00 |
| zbr | i am confident that we can avoid having to add it to >50 files. | 16:00 |
| sshnaidm | zbr, when it will be ready, you're more than welcome to try it.. | 16:00 |
| EmilienM | rdo_mirror: http://mirror.gra1.ovh.openstack.org:8080/rdo | 16:00 |
| EmilienM | https://36d0834365e8b1dc2848-76fc92e5e7c4e5a091c792a95503ad1d.ssl.cf1.rackcdn.com/708808/4/check/tripleo-ansible-centos-7-molecule-tripleo_packages/4db18b1/reports.html | 16:00 |
| EmilienM | nice! | 16:01 |
| EmilienM | zbr: it sounds like a great interim solution until we get something cleaner, isn't? | 16:01 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-ansible master: Add no_log to the podman login https://review.opendev.org/708895 | 16:01 |
| *** leanderthal has quit IRC | 16:03 | |
| EmilienM | weshay|ruck, mwhahaha: https://review.opendev.org/#/c/708808/ will make our molecule more reliable and zbr proposed a plan to integrate a new interface in Molecule itself. https://github.com/ansible-community/molecule/issues/2565 | 16:03 |
| zbr | EmilienM: cloudnull: sshnaidm: are you aware of the existense of https://hub.docker.com/repository/docker/pycontribs/centos ? | 16:03 |
| EmilienM | negative | 16:04 |
| *** TrevorV has quit IRC | 16:04 | |
| zbr | on most molecule jobs, I use these because they are preloaded with python, avoiding the need to install it. | 16:04 |
| cloudnull | sshnaidm can you add that change here too https://github.com/openstack/tripleo-ansible/blob/master/_skeleton_role_/molecule/default/molecule.yml | 16:04 |
| sshnaidm | cloudnull, yeah, totally | 16:04 |
| cloudnull | otherwise your change looks good to me | 16:04 |
| *** rfolco is now known as rfolco|doctor | 16:04 | |
| *** ykarel is now known as ykarel|afk | 16:05 | |
| zbr | this speeds up provisioning considerably, but there is a catch: you need to be careful when to use them, or go for pure ones. | 16:05 |
| sshnaidm | cloudnull, is it fine to add it on top? | 16:05 |
| cloudnull | sure | 16:05 |
| zbr | lets say that these are the container version of the nodepool images :D | 16:05 |
| sshnaidm | cloudnull, then let's merge it, and I adding the followup.. | 16:06 |
| cloudnull | merging | 16:07 |
| EmilienM | ++ | 16:07 |
| EmilienM | we'll need that on train | 16:07 |
| *** jpena is now known as jpena|off | 16:07 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: Add CI proxy settings to role skeleton https://review.opendev.org/708903 | 16:08 |
| sshnaidm | cloudnull, ^^ | 16:08 |
| sshnaidm | zbr, take a look also https://review.opendev.org/#/c/708851/ | 16:12 |
| zbr | sshnaidm: tbh, I would be surprised to see podman driver working considering that I do not remember seeing it running on CI for more than month | 16:14 |
| sshnaidm | zbr, it worked for me today | 16:15 |
| zbr | that is good to know | 16:15 |
| zbr | we need to enable it back on CI | 16:15 |
| sshnaidm | zbr, but Ci version of ansible may be too old.. | 16:15 |
| sshnaidm | Ansible Version: 2.8.8 | 16:15 |
| zbr | my plan is to remove both docker and podman drivers from molecule core and replace them with generic containers driver, one that use first backend that it finds. | 16:16 |
| zbr | this will allow us to control which one is used by the jobs from outside, without touching each molecule.yml file. | 16:16 |
| sshnaidm | zbr, "first backend that it finds" open a lot of options to fail.. | 16:17 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-ansible master: DNM - wait for systemd files to be on the host before reloading service https://review.opendev.org/708904 | 16:17 |
| sshnaidm | zbr, there should be option to configure a specific backend | 16:18 |
| zbr | yep | 16:18 |
| sshnaidm | zbr, and also podman has little bit different options and behavior | 16:18 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-ansible stable/train: DNM - wait for systemd files to be on the host before reloading service https://review.opendev.org/708906 | 16:18 |
| sshnaidm | so instead of replacing, I'd add additional "generic" driver | 16:19 |
| zbr | i know, hiding the differences could prove an interesting challenge. | 16:19 |
| sshnaidm | I think it's gonna be also generic "container" module for podman and docker in ansible, so it will be possible to use it in molecule as is, instead of working on this in molecule itself | 16:19 |
| zbr | i am sure we will still need to allow to pass-over backend specific options | 16:20 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-upgrade stable/train: DNM - test upgrades to ansible managed containers with a sleep https://review.opendev.org/708907 | 16:20 |
| zbr | true, nobody say we have to drop these | 16:20 |
| sshnaidm | zbr, because the less molecule deals with backends, the better for all | 16:20 |
| zbr | we may have different oppinions here, if you like DIY, you probably need to use only delegated, or even better, do everyting in ansible, bash,... :D | 16:21 |
| zbr | the max value I see on molecule is that it hides the provisioning of test hosts, so I do not have to waste my time doing that work. | 16:22 |
| EmilienM | mwhahaha: I found a new one :D | 16:23 |
| mwhahaha | orly | 16:23 |
| EmilienM | https://05badc18f82a08bb81f1-14103d98dfeab14adaa7d6f3b3cde1a9.ssl.cf1.rackcdn.com/700738/37/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/1c93ad4/logs/undercloud/home/zuul/undercloud_upgrade.log | 16:23 |
| EmilienM | undercloud upgrade failed | 16:23 |
| EmilienM | everything is fine | 16:23 |
| EmilienM | "Deployment successful!" | 16:23 |
| mwhahaha | that's not new | 16:23 |
| EmilienM | https://review.opendev.org/#/c/702161/ | 16:24 |
| EmilienM | ? | 16:24 |
| mwhahaha | i think so | 16:24 |
| EmilienM | https://bugs.launchpad.net/tripleo/+bug/1859182 | 16:24 |
| openstack | Launchpad bug 1859182 in tripleo "tripleoclient report successful upgrade when ansible failed" [High,Fix released] - Assigned to Rabi Mishra (rabi) | 16:24 |
| mwhahaha | see not new | 16:25 |
| mwhahaha | DO TRY AND KEEP UP | 16:25 |
| EmilienM | well it was "fixed" | 16:25 |
| EmilienM | but i still see it | 16:25 |
| EmilienM | I'll check the client version | 16:26 |
| mwhahaha | yea there's probably an incorrect return code being returned or something | 16:26 |
| weshay|ruck | ha.. | 16:26 |
| EmilienM | None: AnsibleRunnerException: Ansible execution failed. playbook: deploy_steps_playbook.yaml, Run Status: failed, Return Code: 2 | 16:27 |
| mwhahaha | i see it | 16:28 |
| mwhahaha | do you want to know? | 16:28 |
| mwhahaha | or should i let you figure it out | 16:28 |
| EmilienM | :) | 16:28 |
| EmilienM | teach me plz | 16:29 |
| mwhahaha | NO | 16:29 |
| *** bandini has quit IRC | 16:29 | |
| EmilienM | a good news is that the upgrade isn't broken on other patches so we didn't miss a patch which break upgrades | 16:30 |
| *** jfrancoa has joined #tripleo | 16:30 | |
| EmilienM | mwhahaha: it's a standalone thing I think because in undercloud it catches the error | 16:30 |
| EmilienM | but I haven't digged much yet | 16:30 |
| * EmilienM thinks mwhahaha is writting the patch now | 16:31 | |
| mwhahaha | https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/tripleo_deploy.py#L1313-L1332 | 16:31 |
| mwhahaha | is the hint | 16:31 |
| *** ccamacho has quit IRC | 16:32 | |
| zbr | do we use the same "platforms:" block on all/most of molecule.yml files from tripleo-ansible? | 16:32 |
| mwhahaha | really the problem starts on 1333, but anyway i'll poke at it later | 16:32 |
| * mwhahaha is doing other things | 16:32 | |
| zbr | if so, you will be impressed how much we can remove from molecule.yml.... | 16:32 |
| *** jpich has quit IRC | 16:39 | |
| *** raildo has quit IRC | 16:39 | |
| *** jpich has joined #tripleo | 16:39 | |
| *** morazi has quit IRC | 16:40 | |
| EmilienM | please all don't run check-rdo until https://bugs.launchpad.net/tripleo/+bug/1863920 is closed | 16:41 |
| openstack | Launchpad bug 1863920 in tripleo "(intermittent?) cli-enable-ssh-admin.yaml fails during the overcloud deploy" [Critical,In progress] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 16:41 |
| EmilienM | OVB jobs fs001 at least will fail anyway until this bug is closed | 16:41 |
| weshay|ruck | yay | 16:42 |
| weshay|ruck | THIS MAN SPEAKS TRUTH | 16:42 |
| weshay|ruck | and french | 16:42 |
| EmilienM | the patch that will fix it is https://review.opendev.org/#/c/708781/ | 16:42 |
| *** weshay|ruck changes topic to "CI Status: GREEN'ish, OVB RED 1863920| community irc meeting Tues@1400 UTC - tripleo-ci-community meeting Tues@1330 UTC | https://docs.openstack.org/tripleo-docs/latest/" | 16:42 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Disable Paunch by default Standalone/Overcloud https://review.opendev.org/700738 | 16:43 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Disable Paunch by default Standalone/Overcloud https://review.opendev.org/700738 | 16:43 |
| *** slaweq has quit IRC | 16:43 | |
| EmilienM | weshay|ruck: thx | 16:43 |
| *** jdandrea has quit IRC | 16:44 | |
| *** lmiccini has quit IRC | 16:44 | |
| *** raildo has joined #tripleo | 16:45 | |
| *** dtantsur is now known as dtantsur|afk | 16:51 | |
| *** TrevorV has joined #tripleo | 16:51 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-docs master: WIP document the dlrn_hash_tag featureset_override testproject https://review.opendev.org/708914 | 16:52 |
| *** rpittau is now known as rpittau|afk | 16:53 | |
| zbr | revote on https://review.opendev.org/#/c/708026/ | 16:54 |
| *** jaosorior has quit IRC | 16:55 | |
| *** jdandrea has joined #tripleo | 16:55 | |
| *** jdandrea has quit IRC | 16:57 | |
| ramishra | EmilienM: https://bugs.launchpad.net/tripleo/+bug/1859182 | 16:57 |
| openstack | Launchpad bug 1859182 in tripleo "tripleoclient report successful upgrade when ansible failed" [High,In progress] - Assigned to Rabi Mishra (rabi) | 16:57 |
| *** jdandrea has joined #tripleo | 16:57 | |
| *** pbandark has quit IRC | 16:57 | |
| ramishra | EmilienM: https://bugs.launchpad.net/tripleo/+bug/1859182 was fixed but now again broken by https://review.opendev.org/#/c/704390/3/tripleoclient/utils.py | 16:58 |
| openstack | Launchpad bug 1859182 in tripleo "tripleoclient report successful upgrade when ansible failed" [High,In progress] - Assigned to Rabi Mishra (rabi) | 16:58 |
| *** jdandrea has quit IRC | 16:58 | |
| *** lucasagomes has quit IRC | 16:58 | |
| *** jdandrea_ has joined #tripleo | 16:58 | |
| *** tesseract has quit IRC | 17:00 | |
| openstackgerrit | Merged openstack/tripleo-ansible master: Don't fail symlinking puppet modules if directory exists https://review.opendev.org/708285 | 17:01 |
| openstackgerrit | Merged openstack/tripleo-ansible master: Use CI mirrors for molecule jobs https://review.opendev.org/708808 | 17:01 |
| *** jdandrea_ has quit IRC | 17:03 | |
| *** marios|ruck is now known as marios|out | 17:04 | |
| *** abishop has quit IRC | 17:06 | |
| *** ratailor has quit IRC | 17:10 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/ansible-role-collect-logs master: Assure pyyaml is installed before running sova test https://review.opendev.org/703781 | 17:14 |
| openstackgerrit | Sorin Sbarnea proposed openstack/ansible-role-collect-logs master: Upgrade to molecule 3.0 https://review.opendev.org/708020 | 17:15 |
| openstackgerrit | Rabi Mishra proposed openstack/python-tripleoclient master: Revert "Add raised exception when playbook fails" https://review.opendev.org/708926 | 17:16 |
| ramishra | cloudnull: ^^, I think that patch brings back https://bugs.launchpad.net/tripleo/+bug/1859182 | 17:17 |
| openstack | Launchpad bug 1859182 in tripleo "tripleoclient report successful upgrade when ansible failed" [High,In progress] - Assigned to Rabi Mishra (rabi) | 17:17 |
| *** chem has quit IRC | 17:21 | |
| *** marios|out has quit IRC | 17:25 | |
| *** jpena|off is now known as jpena | 17:27 | |
| *** chem has joined #tripleo | 17:28 | |
| *** suuuper has quit IRC | 17:28 | |
| zbr | EmilienM: cloudnull : revote https://review.opendev.org/#/c/708026/ | 17:29 |
| EmilienM | ramishra: nice catch | 17:30 |
| *** jfrancoa has quit IRC | 17:31 | |
| *** udesale has quit IRC | 17:33 | |
| cloudnull | zbr done | 17:34 |
| *** evrardjp has quit IRC | 17:34 | |
| *** evrardjp has joined #tripleo | 17:35 | |
| weshay|ruck | ALL https://lists.rdoproject.org/pipermail/dev/2020-February/009279.html | 17:36 |
| cloudnull | ramishra something else must be going on, if the playbook exits non-0 it raises an exception, without the fix being reverted the playbook will always return with the status code, even if that status code is >0 | 17:37 |
| *** dking_desktop has joined #tripleo | 17:38 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-ansible stable/train: Use CI mirrors for molecule jobs https://review.opendev.org/708930 | 17:38 |
| EmilienM | sshnaidm: ^ backport | 17:38 |
| cloudnull | ramishra EmilienM maybe the patch has merged but its not in the current package ? | 17:39 |
| ramishra | cloudnull: don't understand what you mean. it's not the playbook status code but the return from run_ansible_playbook() call which matters | 17:39 |
| *** jpich has quit IRC | 17:40 | |
| cloudnull | run_ansible_playbook returns the playbook status code | 17:40 |
| ramishra | it won't if you raise an exception | 17:40 |
| cloudnull | if it fails we raise an exception | 17:41 |
| cloudnull | which would exit without any return | 17:41 |
| cloudnull | and halt the run | 17:41 |
| ramishra | yes, but we check the rc in https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/tripleo_deploy.py#L1351 | 17:41 |
| *** shardy has quit IRC | 17:42 | |
| *** avivgt has quit IRC | 17:42 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: WIP: try podman driver https://review.opendev.org/708851 | 17:42 |
| ramishra | in the finally block.. If you want to raise an expection you've to refactor the finally block | 17:42 |
| cloudnull | https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/tripleo_deploy.py#L1320-L1328 | 17:43 |
| cloudnull | the rc object is set in that try block which traps Exception | 17:43 |
| ramishra | cloudnull: https://review.opendev.org/#/c/702161/ | 17:44 |
| ramishra | return and raise expection are mutually exclusive | 17:44 |
| ramishra | that's what that fix was about | 17:44 |
| cloudnull | i understand that, | 17:44 |
| ramishra | then how would you get rc when you raise an exception in run_ansible_playbook() | 17:45 |
| cloudnull | but you will have to refactor all of the playbook runs to support returning the rc always | 17:45 |
| cloudnull | you wouldn't you'd fix the except Exception | 17:45 |
| weshay|ruck | Tengu, what's the story here? | 17:45 |
| weshay|ruck | Feb 20 17:20:44 standalone.localdomain healthcheck_metrics_qdr[289453]: Error: exec failed: container_linux.go:345: starting container process caused "exec: \"/openstack/healthcheck\": permission denied": OCI runtime permission denied error | 17:45 |
| weshay|ruck | Feb 20 17:20:43 standalone.localdomain systemd[1]: tripleo_metrics_qdr_healthcheck.service: main process exited, code=exited, status=126/n/a | 17:45 |
| cloudnull | and trap a good exception instead of everything under the sun | 17:45 |
| weshay|ruck | https://043e10ad019e85b50614-986c99e3b3126abac54dc88cbca785e8.ssl.cf1.rackcdn.com/707361/2/gate/tripleo-ci-centos-7-scenario001-standalone/190b36f/logs/undercloud/var/log/extra/failed_services.txt | 17:45 |
| cloudnull | if https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/tripleo_deploy.py#L1329 is hit rc != 0 and you can set it accordingly | 17:46 |
| cloudnull | like we do here https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/tripleo_deploy.py#L1228 | 17:47 |
| ramishra | cloudnull: I mentioned if you want to refactor the finaly block you can, but catching execption and then put some return code there would not be correct | 17:47 |
| ramishra | the fact that you're expecting a return code when rasing an exception is inocrrect | 17:48 |
| cloudnull | the fact we're traping Exception is incorrect :D | 17:48 |
| ramishra | that's old code | 17:48 |
| cloudnull | old == still wrong | 17:48 |
| cloudnull | we can change it | 17:49 |
| cloudnull | https://github.com/openstack/python-tripleoclient/search?q=run_ansible_playbook&unscoped_q=run_ansible_playbook | 17:49 |
| ramishra | I did not say it's not wrong.. but we should not probably break working old code with new code right;) | 17:49 |
| cloudnull | but then we need to update all those to handle the rc info | 17:49 |
| cloudnull | so either way something is going to change | 17:50 |
| cloudnull | i.e. raised exception or implementing rc handling everytime the function is called | 17:51 |
| *** gfidente has quit IRC | 17:52 | |
| ramishra | I don't see any bug, if there was an issue your patch was fixing something | 17:52 |
| ramishra | Was https://review.opendev.org/#/c/704390/ fixing any bug? | 17:53 |
| *** avivgt has joined #tripleo | 17:54 | |
| cloudnull | i dont think there was a filed bug | 17:54 |
| ramishra | if not, I would prefer we revert it and then fix the way we want in a subsequent patch | 17:54 |
| cloudnull | it came up here in channel | 17:55 |
| *** morazi has joined #tripleo | 17:56 | |
| cloudnull | we're already defining the rc here https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/tripleo_deploy.py#L1228 i think it would be better to just redefine it https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/tripleo_deploy.py#L1329 which fixes that rc issue in the finally. | 17:57 |
| ramishra | we should definitely have bugs for issues | 17:57 |
| cloudnull | otherwise we have to add rc handling everywhere in the client | 17:57 |
| ramishra | I don't like that approach.. anyway time for me to hit bed | 17:58 |
| cloudnull | I don't like the other approach, so we're at an impasse :) | 17:58 |
| *** avivgt has quit IRC | 17:59 | |
| ramishra | cloudnull: I mean we can leave it for more people to provide opinion.. What I'm saying is lets revert the patch.. Have a proper bug on what issues we've at other places without rasing an exception and then fix it the way majority agrees on | 18:00 |
| ramishra | but I'll leave it to you, if you don't want to revert and fix it directly and keep that regression open | 18:01 |
| *** derekh has quit IRC | 18:02 | |
| cloudnull | your revert means regression elsewhere. the majority can merge that review if they thinks its the way we need to go. my opinion is expressed in the review | 18:03 |
| ramishra | I'm really shutting shop now.. goodnight | 18:03 |
| cloudnull | have a good one . | 18:03 |
| ramishra | cloudnull: yeah, but we don't have a bug for that;) anyway bye | 18:03 |
| *** holser has quit IRC | 18:09 | |
| *** chandankumar is now known as raukadah | 18:14 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/python-tripleoclient master: Add handler for exception raising https://review.opendev.org/708942 | 18:16 |
| cloudnull | ramishra EmilienM ^ should be a proper fix | 18:16 |
| openstackgerrit | Merged openstack/paunch master: Fallback to a rm -f action for podman https://review.opendev.org/708341 | 18:16 |
| EmilienM | cloudnull: thx, I'll look at it shortly | 18:20 |
| *** cfontain has joined #tripleo | 18:28 | |
| *** jpena is now known as jpena|off | 18:34 | |
| *** cfontain has quit IRC | 18:34 | |
| *** ykarel|afk is now known as ykarel|away | 18:34 | |
| *** amoralej is now known as amoralej|off | 18:36 | |
| zbr | cloudnull: i release 3.0.1 few minutes ago, https://github.com/ansible-community/molecule/releases -- feel free to use it to remove ~3000-3500 lines of molecule.yml from tripleo-ansible | 18:42 |
| cloudnull | ? | 18:43 |
| zbr | many of the files could become empty | 18:43 |
| zbr | read https://github.com/ansible-community/molecule/pull/2566 | 18:43 |
| cloudnull | so we can store all of the configs in .config/molecule/config.yml ? | 18:44 |
| zbr | molecule load the default values from the config file, so you don't have to write anything in individual scenarios, or just the stuff you want different than basic config. | 18:44 |
| zbr | yep | 18:44 |
| cloudnull | neat | 18:44 |
| zbr | to common part | 18:44 |
| zbr | platforms is inhereted too, if you do not alter it, is not "merged" because is a list. | 18:45 |
| zbr | the original feature is quite old, i just made it work with in-repo configs. | 18:46 |
| zbr | the user config was not of much use to me. | 18:46 |
| openstackgerrit | Emilien Macchi proposed openstack/paunch stable/train: Fallback to a rm -f action for podman https://review.opendev.org/708881 | 18:53 |
| *** saneax has quit IRC | 18:55 | |
| *** ansmith_ has joined #tripleo | 18:56 | |
| *** ansmith has quit IRC | 18:59 | |
| *** cfontain has joined #tripleo | 18:59 | |
| *** cfontain has quit IRC | 19:06 | |
| *** pbandark has joined #tripleo | 19:10 | |
| *** stevebaker has joined #tripleo | 19:15 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/ansible-role-container-registry master: Migrated to molecule 3.0 https://review.opendev.org/708953 | 19:15 |
| *** rlandy is now known as rlandy|mtg | 19:16 | |
| weshay|ruck | there I wf.. https://review.opendev.org/#/c/708930/ EmilienM :/ | 19:18 |
| weshay|ruck | ;P | 19:18 |
| zbr | cloudnull: i discovered that https://opendev.org/openstack/ansible-plugin-container-connection doesn't even have a .gitreview file | 19:20 |
| cloudnull | not been a lot of traction there. | 19:20 |
| cloudnull | sadly | 19:20 |
| zbr | gerrit may not be the right place to see traction, is not github | 19:21 |
| *** pbandark has quit IRC | 19:22 | |
| weshay|ruck | zbr, did you find a fix to the stupid queens issue btw? | 19:23 |
| EmilienM | weshay|ruck: lol | 19:23 |
| zbr | weshay|ruck: i send you the link | 19:23 |
| weshay|ruck | zbr, but I'm too stoopid.. | 19:24 |
| zbr | weshay|ruck: i am not sure where it needs to be done, but you need to https://github.com/ansible-community/molecule/blob/master/tools/test-setup.sh#L11 | 19:24 |
| zbr | "zipp<0.6.0;python_version=='2.7'" | 19:24 |
| weshay|ruck | zbr, /me wonders if ansible deal w/ that | 19:25 |
| weshay|ruck | k | 19:25 |
| zbr | or you risk having pip installing an incompatible newer version | 19:25 |
| weshay|ruck | so just make sure zipp is pinned under 0.6.0 | 19:25 |
| zbr | pick the exact condition! | 19:25 |
| zbr | this needs to be only for old pythons | 19:25 |
| zbr | if you pre-install it, you should be fine. | 19:26 |
| *** rlandy|mtg is now known as rlandy | 19:34 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: pip install zipp to fix contextlib2 error https://review.opendev.org/708791 | 19:35 |
| weshay|ruck | zbr, ^ | 19:35 |
| *** cfontain has joined #tripleo | 19:35 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-ci master: update centos8 build containers to current-tripleo build https://review.opendev.org/708956 | 19:38 |
| openstackgerrit | Dincer Celik proposed openstack/tripleo-common master: Cleanup template overrides for rabbitmq https://review.opendev.org/697888 | 19:47 |
| *** cfontain has quit IRC | 19:51 | |
| *** ansmith_ has quit IRC | 19:55 | |
| fultonj | tripleo-ci-centos-7-containerized-undercloud-upgrades has been failing often with Failed to start tripleo_memcached.service: Unit not found | 20:06 |
| fultonj | e.g. https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_6d5/708513/1/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/6d5a3b8/logs/undercloud/home/zuul/undercloud-ansible-V8ZRHj/ansible.log | 20:06 |
| fultonj | has anyone seen this ^ ? | 20:06 |
| fultonj | weshay|ruck: ^ ? | 20:08 |
| weshay|ruck | looking.. not familiar yet | 20:09 |
| fultonj | i can open a LP bug | 20:09 |
| fultonj | i've rechecked https://review.opendev.org/#/c/708513/ a few times | 20:10 |
| fultonj | didn't want to recheck again if it's a known issue | 20:10 |
| weshay|ruck | fultonj, http://dashboard-ci.tripleo.org/d/jobs/jobs-exploration?orgId=1&var-influxdb_filter=job_name%7C%3D%7Ctripleo-ci-centos-7-containerized-undercloud-upgrades | 20:11 |
| fultonj | so if i recheck there's a 76% chance it will pass | 20:12 |
| weshay|ruck | it's mostly passing in the gate | 20:12 |
| weshay|ruck | http://dashboard-ci.tripleo.org/d/jobs/jobs-exploration?orgId=1&var-influxdb_filter=job_name%7C%3D%7Ctripleo-ci-centos-7-containerized-undercloud-upgrades&var-influxdb_filter=pipeline%7C%3D%7Cgate | 20:13 |
| fultonj | oh yeah | 20:13 |
| fultonj | weshay|ruck: i guess i'll recheck then | 20:14 |
| fultonj | thanks mr ptl :) | 20:15 |
| * weshay|ruck wonders if the db connection is up and down | 20:16 | |
| fultonj | db connection? | 20:16 |
| fultonj | the failure in this instance was a missing unit | 20:16 |
| * weshay|ruck poking around | 20:17 | |
| weshay|ruck | fultonj, http://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_6d5/708513/1/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/6d5a3b8/logs/undercloud/var/log/extra/podman/podman_allinfo.log | 20:18 |
| fultonj | oh | 20:18 |
| fultonj | the memcached container is up | 20:19 |
| weshay|ruck | and down | 20:19 |
| fultonj | flapping | 20:19 |
| weshay|ruck | aye | 20:19 |
| *** jfrancoa has joined #tripleo | 20:20 | |
| fultonj | i have two independent patches stuck it | 20:21 |
| fultonj | 2 rechecks each today | 20:21 |
| weshay|ruck | fultonj, ya.. let's get a bug on it | 20:21 |
| fultonj | weshay|ruck: ok, i'll open a bug | 20:21 |
| fultonj | given the passing rate i'll recheck my two jobs unless you object | 20:22 |
| *** paramite|clone has quit IRC | 20:23 | |
| fultonj | weshay|ruck: https://bugs.launchpad.net/tripleo/+bug/1864105 | 20:27 |
| openstack | Launchpad bug 1864105 in tripleo "memcached container flapping on train upgrades job" [Undecided,Triaged] | 20:27 |
| weshay|ruck | fultonj, thanks | 20:30 |
| fultonj | sure | 20:30 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Fix dcn-hci resource mapping path https://review.opendev.org/708838 | 20:35 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/stein: Force facts cache refreshing after OS upgrade. https://review.opendev.org/708863 | 20:35 |
| openstackgerrit | Merged openstack/ansible-role-collect-logs master: Fix idempotence of infrared scenario https://review.opendev.org/708613 | 20:35 |
| openstackgerrit | Merged openstack/tripleo-ansible master: Upgrade molecule to 3.0 https://review.opendev.org/708026 | 20:37 |
| mwhahaha | cloudnull: did you figure out the ssh thing yet? | 20:41 |
| cloudnull | mwhahaha - https://review.opendev.org/#/c/708781 - i think so | 20:43 |
| cloudnull | i currently fighting the "how do i setup novaless environments" right now to verify | 20:43 |
| cloudnull | but the gate looks promissing | 20:44 |
| cloudnull | tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001 - passed | 20:44 |
| mwhahaha | yikes | 20:44 |
| mwhahaha | oh it's cause id_rsa_tripleo | 20:53 |
| *** artom has quit IRC | 20:54 | |
| openstackgerrit | Alex Schultz proposed openstack/ansible-role-container-registry master: Bump molecule version https://review.opendev.org/708964 | 20:57 |
| openstackgerrit | Alex Schultz proposed openstack/ansible-role-container-registry master: Don't log login https://review.opendev.org/708902 | 20:58 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-ansible master: Add strategy optimization https://review.opendev.org/704691 | 20:59 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-ansible master: Add additional strategy optimization https://review.opendev.org/704443 | 21:00 |
| *** TrevorV has quit IRC | 21:03 | |
| *** ansmith_ has joined #tripleo | 21:06 | |
| *** jfrancoa has quit IRC | 21:06 | |
| openstackgerrit | wes hayutin proposed openstack/python-tripleoclient master: turn off temp dir cleanup in container builds https://review.opendev.org/708967 | 21:13 |
| *** mcornea has quit IRC | 21:13 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: Add release file for master CentOS 8 https://review.opendev.org/708968 | 21:18 |
| EmilienM | fultonj: hey | 21:20 |
| EmilienM | [Bug 1864105] [NEW] memcached container flapping on train upgrades job | 21:20 |
| openstack | bug 1864105 in tripleo "memcached container flapping on train upgrades job" [Critical,Triaged] https://launchpad.net/bugs/1864105 | 21:20 |
| fultonj | EmilienM: hi | 21:20 |
| EmilienM | it's funny | 21:20 |
| fultonj | yeah | 21:20 |
| EmilienM | that's what I've been investigating all day | 21:21 |
| EmilienM | let's talk about it | 21:21 |
| EmilienM | so I added a task to make sure we have the file | 21:21 |
| EmilienM | check out http://paste.openstack.org/show/789811/ | 21:21 |
| EmilienM | normally, the module itself should do the reload before enabling the unit | 21:21 |
| EmilienM | but I suspect a race | 21:21 |
| EmilienM | cloudnull suggested something like http://paste.openstack.org/show/789812/ | 21:22 |
| EmilienM | to force the reload in a standalone task | 21:22 |
| EmilienM | I was going to try it next | 21:22 |
| mwhahaha | isn't there a bug around reload | 21:23 |
| fultonj | sec | 21:23 |
| EmilienM | I know nothing about a bug around reload | 21:23 |
| *** Goneri has quit IRC | 21:25 | |
| *** xek_ has quit IRC | 21:26 | |
| * fultonj reads | 21:26 | |
| fultonj | "Wait for /etc/systemd/system/tripleo_memcached.service to be created]" ack | 21:26 |
| EmilienM | yeah I tried this but it doesn't helop | 21:26 |
| fultonj | when container_sysd_name in containers_changed | 21:27 |
| fultonj | we need to get a log of what made the container die | 21:27 |
| EmilienM | it didn't die | 21:28 |
| EmilienM | it restarted with a new image | 21:28 |
| EmilienM | it's an upgrade job | 21:28 |
| EmilienM | it was deleted, then re-created | 21:28 |
| fultonj | so it's not flapping | 21:28 |
| EmilienM | no I think systemd doesn't know that the unit file is there yet | 21:28 |
| EmilienM | which is weird | 21:28 |
| *** raildo has quit IRC | 21:28 | |
| EmilienM | since it works fine in a regular deployment | 21:28 |
| fultonj | i assumed it was flapping ; i assumed restart=always | 21:29 |
| fultonj | so it tries to start hits what i thought would be the root cause, fails, restarts etc | 21:29 |
| * fultonj should look at the unit file though | 21:29 | |
| EmilienM | mwhahaha: https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/system/systemd.py#L388 | 21:30 |
| EmilienM | I haven't found any issue in the module | 21:30 |
| EmilienM | fultonj: unit file isn't collected until https://review.opendev.org/#/c/708889/ | 21:30 |
| * fultonj reading artificats of job | 21:31 | |
| *** jbadiapa has quit IRC | 21:31 | |
| fultonj | oh | 21:31 |
| EmilienM | fultonj: but I added depends-on on some patches and I can see the unit file exists | 21:31 |
| EmilienM | fultonj: take https://review.opendev.org/#/c/708907/ for example | 21:31 |
| EmilienM | you have the unit file and the same error | 21:31 |
| EmilienM | https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_574/708907/1/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/5749c07/ | 21:31 |
| mwhahaha | i vaguely recall a patch where the ansible bits didn't work. i found a 2.2 bug tho | 21:31 |
| mwhahaha | probably an old issue | 21:32 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-ansible master: tripleo_container_manage/systemd: force systemd daemon reload https://review.opendev.org/708904 | 21:32 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Disable Paunch by default Standalone/Overcloud https://review.opendev.org/700738 | 21:32 |
| EmilienM | let's see how that work with a manual relaod | 21:33 |
| openstackgerrit | Merged openstack/tripleo-common stable/stein: Make ()'s around step|int optional https://review.opendev.org/708392 | 21:35 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: DNM - test https://review.opendev.org/#/c/708942 https://review.opendev.org/708971 | 21:36 |
| EmilienM | cloudnull: ^ testing your patch | 21:36 |
| EmilienM | cloudnull: I put it on top of https://review.opendev.org/#/c/700738 on purpose | 21:36 |
| EmilienM | mhh wait | 21:36 |
| EmilienM | let me change that | 21:37 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: DNM - test https://review.opendev.org/#/c/708942 https://review.opendev.org/708971 | 21:37 |
| EmilienM | cloudnull: now ^ will fail if your tripleoclient patch works | 21:37 |
| *** ansmith_ has quit IRC | 21:38 | |
| EmilienM | cloudnull: since disabling Paunch seems to fail on the containerized undercloud job for now (systemd issue) | 21:38 |
| * cloudnull fingers crossed | 21:47 | |
| EmilienM | stevebaker: I can't stay longer today but we need to chat about re: https://review.opendev.org/#/c/701280/ | 21:51 |
| EmilienM | stevebaker: ade_lee was doing scale down with deployed servers and he apparently needs some tasks to run | 21:51 |
| fultonj | EmilienM: thanks for digging into that | 21:51 |
| EmilienM | fultonj: yeah no worries. I'll get this fixed before eow hopefully. | 21:52 |
| EmilienM | fultonj: I need this fixed so I can turn off paunch by default | 21:52 |
| EmilienM | and it won't break upgrades | 21:52 |
| stevebaker | EmilienM: ack, shall we talk your monday? | 21:52 |
| EmilienM | stevebaker: I'll read irc later but if you have ideas I'm open to hear about them | 21:52 |
| EmilienM | stevebaker: yeah, I'll put something in our calendar if you don't mind | 21:52 |
| stevebaker | EmilienM: sure thing | 21:53 |
| *** avivgt has joined #tripleo | 21:56 | |
| *** pbandark has joined #tripleo | 21:56 | |
| openstackgerrit | Merged openstack/tripleo-ci master: collect-logs: collect tripleo related systemd unit files https://review.opendev.org/708889 | 22:07 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Remove comment about tripleo_container_manage being experimental https://review.opendev.org/708521 | 22:07 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: WIP: try podman driver https://review.opendev.org/708851 | 22:11 |
| *** avivgt has quit IRC | 22:11 | |
| openstackgerrit | Merged openstack/tripleo-ansible master: Add no_log to the podman login https://review.opendev.org/708895 | 22:12 |
| openstackgerrit | Merged openstack/tripleo-ansible stable/train: Use CI mirrors for molecule jobs https://review.opendev.org/708930 | 22:12 |
| *** jmasud has quit IRC | 22:15 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-ansible master: module tripleo_baremetal_expand_roles https://review.opendev.org/706417 | 22:16 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-ansible master: module metalsmith_instances https://review.opendev.org/706418 | 22:16 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-ansible master: module tripleo_baremetal_check_existing https://review.opendev.org/707043 | 22:16 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-ansible master: module tripleo_baremetal_populate_environment https://review.opendev.org/707295 | 22:16 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-ansible master: Playbook for overcloud node provision https://review.opendev.org/707044 | 22:16 |
| openstackgerrit | Steve Baker proposed openstack/python-tripleoclient master: WIP switch to ansible for overcloud node provision https://review.opendev.org/708749 | 22:16 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: Add release file for master CentOS 8 https://review.opendev.org/708968 | 22:19 |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates stable/queens: NodeDataLookup utility should rely on python env https://review.opendev.org/708978 | 22:20 |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates stable/train: NodeDataLookup utility should rely on python env https://review.opendev.org/708979 | 22:20 |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates stable/stein: NodeDataLookup utility should rely on python env https://review.opendev.org/708980 | 22:21 |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates stable/rocky: NodeDataLookup utility should rely on python env https://review.opendev.org/708981 | 22:22 |
| openstackgerrit | Merged openstack/tripleo-ansible master: paunch: fail early if containers weren't deployed by Paunch before https://review.opendev.org/708880 | 22:24 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-ansible stable/train: Add no_log to the podman login https://review.opendev.org/708982 | 22:25 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates stable/stein: [Stein-only] add no_log to the podman login https://review.opendev.org/708984 | 22:28 |
| *** dpawlik has quit IRC | 22:33 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-ansible master: Add tmpwatch to the bootstrap https://review.opendev.org/708986 | 22:35 |
| *** iurygregory has quit IRC | 22:36 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates stable/stein: [Stein and older] Ensure tmpwatch installed https://review.opendev.org/708987 | 22:37 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart master: Sync fs035 job with fs001 settings https://review.opendev.org/708988 | 22:44 |
| *** jmasud has joined #tripleo | 22:44 | |
| *** pkopec has quit IRC | 22:47 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart master: Add release file for master CentOS 8 https://review.opendev.org/708968 | 22:49 |
| cloudnull | EmilienM mwhahaha with the "enable-swap" template, if the host has a swap partition, will that template use it, or does it always try and create a swap file ? | 22:53 |
| mwhahaha | enable-swap.yaml is a file | 22:54 |
| mwhahaha | there's a different one for partitions | 22:54 |
| mwhahaha | enable-swap-partition.yaml | 22:54 |
| cloudnull | i see | 22:54 |
| mwhahaha | (fittingly named) | 22:54 |
| cloudnull | does enable-swap-partition.yaml require config or does it seek an existing partition ? | 22:55 |
| mwhahaha | needs to be labled it looks like | 22:55 |
| * mwhahaha shrugs | 22:55 | |
| mwhahaha | i only ever use enable-swap.yaml | 22:55 |
| cloudnull | ++ | 22:55 |
| cloudnull | I'm tinkering with metalsmith and it has an option swap_size_mb | 22:56 |
| cloudnull | which creates a swap partition, but it doesn't enable it | 22:56 |
| cloudnull | puts nothing in fstab, and doesn't seem to run swapon $DEV for the partition | 22:57 |
| mwhahaha | right the image itself would have to know about the partitions | 22:57 |
| mwhahaha | overcloud-full by default doesn't | 22:57 |
| mwhahaha | there's a way to do custom partitions and images but that's out of my scope | 22:58 |
| *** tkajinam has joined #tripleo | 22:58 | |
| mwhahaha | looks like you'd just have to specify swap_partition_label: <a label> in parameter_defaults somewhere. though it assumes the name is consistent across all devices | 22:58 |
| mwhahaha | https://bugzilla.redhat.com/show_bug.cgi?id=1293408#c5 | 22:59 |
| openstack | bugzilla.redhat.com bug 1293408 in openstack-tripleo-heat-templates "Overcloud nodes are created without swap space" [High,Closed: errata] - Assigned to hbrock | 22:59 |
| * cloudnull will give that a spin | 23:00 | |
| openstackgerrit | Alex Schultz proposed openstack/ansible-role-container-registry master: Don't log login https://review.opendev.org/708902 | 23:02 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-ansible master: WIP: try podman driver https://review.opendev.org/708851 | 23:02 |
| *** sshnaidm is now known as sshnaidm|afk | 23:02 | |
| cloudnull | but in exciting news, i have a novaless environment deployed - https://pasted.tech/pastes/fd7ae3e111080ea0c65a76ce27b1e90542bc6359 | 23:02 |
| cloudnull | i dont think https://github.com/openstack/tripleo-heat-templates/blob/master/extraconfig/all_nodes/swap-partition.j2.yaml#L21 works | 23:06 |
| cloudnull | these swap partitions don't have a label, https://pasted.tech/pastes/8ae221164d290a0f97e3e65e90f94058b91c5752 | 23:08 |
| mwhahaha | probably | 23:08 |
| mwhahaha | it's generally untested afaik | 23:08 |
| mwhahaha | feel free to improve :D | 23:08 |
| * mwhahaha runs away for a bit | 23:09 | |
| cloudnull | :D | 23:09 |
| *** slaweq has joined #tripleo | 23:10 | |
| *** ahosam has quit IRC | 23:10 | |
| *** ahosam has joined #tripleo | 23:11 | |
| *** slaweq has quit IRC | 23:15 | |
| *** pbandark has quit IRC | 23:25 | |
| *** pbandark has joined #tripleo | 23:25 | |
| *** ahosam has quit IRC | 23:40 | |
| *** jmasud has quit IRC | 23:43 | |
| *** pbandark has quit IRC | 23:45 | |
| *** jmasud has joined #tripleo | 23:46 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-heat-templates master: Improve the ability to enable swap https://review.opendev.org/709015 | 23:48 |
| *** jmasud has quit IRC | 23:50 | |
| *** alexmcleod has quit IRC | 23:52 | |
| *** jmasud has joined #tripleo | 23:53 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!