| ssbarnea | mwhahaha: see https://review.openstack.org/#/c/615933/ - change is correct, we are not supposed to have py2 packages. | 00:00 |
|---|---|---|
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/560445 | 00:00 |
| mwhahaha | ssbarnea: yea but swift is special. is the client working with py3? | 00:00 |
| *** mcornea has quit IRC | 00:00 | |
| ssbarnea | mwhahaha i did not test but I see it has tox jobs running and passing on py3, so my guess is that yet. | 00:01 |
| ssbarnea | yes | 00:01 |
| mwhahaha | k | 00:01 |
| ssbarnea | considering that we take extra steps to assure we don't have a python2 installed, the assumption that we will have only py still stands. | 00:02 |
| mwhahaha | stevebaker: ehh i'm not sure i like the source as an option but i guess so | 00:04 |
| *** betherly has quit IRC | 00:04 | |
| openstackgerrit | Dan Sneddon proposed openstack/os-net-config master: Apply IP/netmask/route/MTU changes without bouncing interfaces. https://review.openstack.org/606396 | 00:05 |
| mwhahaha | stevebaker: i liked the build flag because it can also be used to selectively build some containers but not others. if you switch to the source you lose that logic. i'll convert it to source i guess. we can always switch back to the previous PS | 00:06 |
| ssbarnea | mwhahaha regarding number of reviews..., yep I made the same remark on #oooq channel, missing a mono-repo, at least for puppet stuff. | 00:06 |
| mwhahaha | ssbarnea: i assure you the mono-repo wasn't the greatest thing in the world :D | 00:07 |
| ssbarnea | mwhahaha, well maybe more a pure monorepo, more of lessrepos :D | 00:08 |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 00:10 |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged] | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 00:10 |
| *** toure is now known as toure|gone | 00:11 | |
| *** slaweq has quit IRC | 00:12 | |
| *** slaweq has joined #tripleo | 00:13 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Add image_source to containers https://review.openstack.org/616022 | 00:19 |
| mwhahaha | stevebaker: -^ | 00:19 |
| *** vinaykns has quit IRC | 00:20 | |
| dsneddon | Has anyone seen issues with undercloud installations on rocky and messed up chronyd.conf config? Somehow my chronyd.conf is getting "pool u'pool.ntp.org'" instead of "pool pool.ntp.org", so something is writing the config in the wrong way. | 00:26 |
| *** salmankhan has quit IRC | 00:26 | |
| dsneddon | ansible-role-chrony is probably responsible. | 00:26 |
| mwhahaha | dsneddon: yes | 00:26 |
| mwhahaha | dsneddon: it's actually tripleo-common | 00:26 |
| * mwhahaha digs up patch | 00:26 | |
| dsneddon | mwhahaha, Apparently the patch hasn't made it to RDO packages yet. | 00:27 |
| mwhahaha | dsneddon: https://review.openstack.org/#/c/614876/ | 00:27 |
| mwhahaha | dsneddon: correct not rdo but neither has the THT | 00:27 |
| mwhahaha | dsneddon: so did you git clone THT? | 00:28 |
| *** ssbarnea has quit IRC | 00:28 | |
| dsneddon | mwhahaha, I hadn't yet. I was still at the undercloud installation step. | 00:28 |
| *** phuongnh has joined #tripleo | 00:28 | |
| mwhahaha | oh containers | 00:28 |
| mwhahaha | thanks containers | 00:28 |
| mwhahaha | wait no | 00:28 |
| mwhahaha | how did you get it enabled but not the tripleo-common patch | 00:29 |
| dsneddon | mwhahaha, I installed via infrared with no options | 00:29 |
| mwhahaha | the THT enabling chrony landed after the tripleo-common patch | 00:29 |
| mwhahaha | yea i have no idea what infrared does | 00:29 |
| * mwhahaha wouldn't be shocked if it git pulled THT | 00:29 | |
| dsneddon | mwhahaha, It's a bunch of Ansible that automates installing packages from delorean, but it might be pulling THT from source by default. It's optional, so I can turn it off. | 00:30 |
| *** gfidente|afk has quit IRC | 00:31 | |
| mwhahaha | dsneddon: yea i know what infrared is, just not all the internals :D | 00:31 |
| mwhahaha | dsneddon: yea if that's enabled, turn that off. usually (upstream) we include the tripleo-* bits from the current repo so you don't hit this problem | 00:31 |
| mwhahaha | it'll fail until there is a tripleo promotion | 00:31 |
| dsneddon | mwhahaha, Thanks, that's probably enough to help me out of this hole. Worst case I'll do what I usually do, which is just use infrared to install the bare virt environment, then follow my normal installation procedure. | 00:32 |
| mwhahaha | dsneddon: np | 00:33 |
| dsneddon | mwhahaha, Looks like I ended up with a newer THT than tripleo-common, but only by a few hours? http://paste.openstack.org/show/734313/ | 00:35 |
| mwhahaha | odd. not sure how the dlrn queue works in RDO | 00:36 |
| *** irclogbot_3 has quit IRC | 00:38 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Add image_source to containers https://review.openstack.org/616022 | 00:42 |
| dsneddon | mwhahaha, Thanks for your help, it turned out I was pulling from the wrong trunk and was getting bleeding-edge packages instead of ones that passed integration testing. | 00:46 |
| mwhahaha | yay bleeding edge :D | 00:47 |
| mwhahaha | The tripleo house of pain, aka current | 00:47 |
| stevebaker | mwhahaha: images could always be set to image_source: kolla_disabled or somesuch | 00:50 |
| mwhahaha | stevebaker: yea but seems a bit awkward. Anyway i updated it | 00:51 |
| * mwhahaha wanders off | 00:51 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged] | 01:10 |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 01:10 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-heat-templates master: Create clouds.yaml for the undercloud https://review.openstack.org/615853 | 01:20 |
| *** tongl has quit IRC | 01:48 | |
| *** rlandy has quit IRC | 01:58 | |
| tonyb | stevebaker: How do I invoke modify-image as part of container prepare again? | 01:59 |
| * tonyb lost that from his git repo (stupid reflog expire :() | 01:59 | |
| stevebaker | tonyb: something like this https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/container_image_prepare.html#modifying-images-during-prepare | 02:00 |
| stevebaker | I didn't know reflogs expired | 02:00 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Fix NeutronSriovHostConfig path for ODL deployment with SRIOV https://review.openstack.org/612685 | 02:01 |
| openstackgerrit | Merged openstack/os-net-config stable/pike: Make onboot parameter for ifcfg configuration configurable https://review.openstack.org/612761 | 02:01 |
| tonyb | stevebaker: Thanks That's what I was looking for | 02:02 |
| tonyb | stevebaker: they don't unless you expire them .... I though I had all my HEADs point at branches but I lost a the stuff I did with conterized undercoud+ppc64le back in Feb :( | 02:03 |
| stevebaker | ah | 02:07 |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 02:10 |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged] | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 02:10 |
| openstackgerrit | Merged openstack/tripleo-common master: Implement tripleo-container-tag role https://review.openstack.org/615177 | 02:15 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Fix multinode image list https://review.openstack.org/604982 | 02:15 |
| *** huynq has joined #tripleo | 02:18 | |
| *** pvc has joined #tripleo | 02:38 | |
| pvc | can anyone review this | 02:38 |
| pvc | https://review.openstack.org/#/c/611519/ | 02:38 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-common master: WIP implement python based uploader https://review.openstack.org/616018 | 02:46 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-common master: WIP make python uploader the default https://review.openstack.org/616019 | 02:46 |
| *** mrsoul has quit IRC | 02:48 | |
| *** mschuppert has quit IRC | 02:49 | |
| *** thrash is now known as thrash|g0ne | 03:09 | |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 03:10 |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged] | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 03:10 |
| *** cdearborn_ has quit IRC | 03:37 | |
| openstackgerrit | Peter Cipriano proposed openstack/tripleo-docs master: Change the "Deploy repos via tripleo-repos" command based on the official tripleo documentation: https://docs.openstack.org/tripleo-docs/latest/install/ installation/installation.html https://review.openstack.org/611519 | 03:41 |
| *** trungnv has joined #tripleo | 03:42 | |
| *** radeks has joined #tripleo | 03:47 | |
| *** owalsh_ has joined #tripleo | 03:47 | |
| *** owalsh has quit IRC | 03:51 | |
| *** eck` is now known as eck`gone | 04:01 | |
| tonyb | My CentOS undercloud is failing to sync the time with chrony. I'm setting undercloud_ntp_server in my config file but that doesn't seem to be reflected in the chrony config. This worked about 3 weeks ago. | 04:05 |
| tonyb | What have I missed | 04:05 |
| tonyb | I'm guessing it's https://review.openstack.org/#/c/611969/ now to work out how to configure it | 04:08 |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 04:10 |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged] | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 04:10 |
| *** noama has joined #tripleo | 04:54 | |
| openstackgerrit | Merged openstack/tripleo-common master: Fix config-download timeout https://review.openstack.org/615653 | 04:58 |
| openstackgerrit | Merged openstack/python-tripleoclient stable/queens: Increase the deploy_plan timeout in tripleoclient https://review.openstack.org/615866 | 04:58 |
| *** phuongnh has quit IRC | 05:00 | |
| *** phuongnh has joined #tripleo | 05:00 | |
| *** agopi has joined #tripleo | 05:08 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 05:10 |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged] | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 05:10 |
| *** hamzy has joined #tripleo | 05:18 | |
| *** trungnv has quit IRC | 05:22 | |
| *** akrzos has quit IRC | 05:28 | |
| *** rascasoft has quit IRC | 05:28 | |
| *** lhinds has quit IRC | 05:28 | |
| *** rnoriega- has quit IRC | 05:28 | |
| *** rnoriega has joined #tripleo | 05:29 | |
| *** lhinds has joined #tripleo | 05:30 | |
| *** rascasoft has joined #tripleo | 05:31 | |
| *** aufi has joined #tripleo | 05:42 | |
| *** aufi has quit IRC | 05:46 | |
| tonyb | The docs job is broken until https://review.openstack.org/616074 merges | 05:57 |
| *** jrist has quit IRC | 05:58 | |
| *** d0ugal has quit IRC | 06:05 | |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged] | 06:10 |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 06:10 |
| *** jrist has joined #tripleo | 06:11 | |
| *** kforde has quit IRC | 06:18 | |
| *** haleyb has quit IRC | 06:20 | |
| *** d0ugal has joined #tripleo | 06:21 | |
| huynq | Hello there! Do you guys know how to resolve SSL CERTIFICATE FAILED error when importing overcloud nodes? | 06:21 |
| huynq | http://paste.openstack.org/show/734322/ | 06:21 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: Remove NeutronServicePlugins from octavia environment files https://review.openstack.org/607169 | 06:28 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: remove non-voting job from gate https://review.openstack.org/615944 | 06:28 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Reimplement HA docker tagging https://review.openstack.org/614827 | 06:28 |
| *** apetrich has quit IRC | 06:29 | |
| *** phuongnh has quit IRC | 06:32 | |
| *** phuongnh has joined #tripleo | 06:33 | |
| *** ade_lee has quit IRC | 06:35 | |
| *** radez has quit IRC | 06:44 | |
| *** apetrich has joined #tripleo | 06:44 | |
| *** radeks has quit IRC | 06:49 | |
| *** lyarwood has quit IRC | 06:53 | |
| *** andreaf has quit IRC | 06:53 | |
| *** andreaf has joined #tripleo | 06:55 | |
| *** agurenko has joined #tripleo | 07:04 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged] | 07:10 |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 07:10 |
| *** quiquell|off is now known as quiquell | 07:10 | |
| quiquell | Good morning | 07:11 |
| *** mschuppert has joined #tripleo | 07:20 | |
| openstackgerrit | Merged openstack/tripleo-upgrade master: Remove tripleo-ci-centos-7-containerized-undercloud-upgrades from gate https://review.openstack.org/615920 | 07:20 |
| jaosorior | good morning | 07:24 |
| *** paramite has joined #tripleo | 07:25 | |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart-extras master: Use correct package manager at DLRN https://review.openstack.org/615528 | 07:29 |
| *** ccamacho has joined #tripleo | 07:31 | |
| quiquell | jaosorior: o/ | 07:31 |
| *** ksambor has joined #tripleo | 07:31 | |
| quiquell | jaosorior: Do you have super powers to workflow https://review.openstack.org/#/c/615886 ? | 07:31 |
| quiquell | jaosorior: Fix heat-agents and we need it for f28 job | 07:32 |
| jaosorior | quiquell: I don't, that's under the heat team. | 07:32 |
| quiquell | jaosorior: Where were the place too see who is core of a project ? | 07:33 |
| *** apetrich has quit IRC | 07:33 | |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: DNM: To test standalone-28 https://review.openstack.org/615479 | 07:34 |
| *** pcaruana has joined #tripleo | 07:36 | |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Collect dnf logs and config https://review.openstack.org/615800 | 07:38 |
| *** paramite has quit IRC | 07:43 | |
| *** jfrancoa has joined #tripleo | 07:45 | |
| *** apetrich has joined #tripleo | 07:47 | |
| *** quiquell is now known as quiquell|brb | 07:49 | |
| *** cylopez has joined #tripleo | 07:54 | |
| *** cylopez has left #tripleo | 07:54 | |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates stable/queens: GATE CHECK for TripleO https://review.openstack.org/567224 | 08:00 |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/560445 | 08:00 |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO https://review.openstack.org/604298 | 08:00 |
| *** jtomasek has joined #tripleo | 08:01 | |
| *** quiquell|brb is now known as quiquell | 08:04 | |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged] | 08:10 |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 08:10 |
| *** nawar has joined #tripleo | 08:14 | |
| nawar | hi I'm trying to config network isolation but i'm keep getting this error "NotSupported: resources.VipPort: Update to properties network of VipPort (OS::TripleO::Network::Ports::ControlPlaneVipPort) is not supported" | 08:15 |
| *** aufi has joined #tripleo | 08:15 | |
| *** fuzzball81 has quit IRC | 08:22 | |
| *** diw has joined #tripleo | 08:27 | |
| *** jjoyce has joined #tripleo | 08:40 | |
| *** yprokule has joined #tripleo | 08:42 | |
| *** bogdando has joined #tripleo | 08:45 | |
| *** amoralej|off is now known as amoralej | 08:45 | |
| Tengu | hmm, does this error ring any bell? Error: Evaluation Error: Unknown function: 'normalize_ip_for_uri'. at /etc/puppet/modules/tripleo/manifests/profile/base/swift/proxy.pp:145:48 on node undercloud.TLD | 08:49 |
| *** gfidente has joined #tripleo | 08:50 | |
| *** jpena|off is now known as jpena | 08:50 | |
| *** radeks has joined #tripleo | 08:52 | |
| *** lyarwood has joined #tripleo | 08:55 | |
| *** jpich has joined #tripleo | 09:01 | |
| *** akrivoka has joined #tripleo | 09:03 | |
| openstackgerrit | Christian Schwede proposed openstack/tripleo-heat-templates master: Fix Swift device config update issues https://review.openstack.org/616116 | 09:08 |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 09:10 |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged] | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 09:10 |
| *** shardy has joined #tripleo | 09:19 | |
| *** ssbarnea has joined #tripleo | 09:19 | |
| *** owalsh_ is now known as owalsh | 09:23 | |
| *** jtomasek has quit IRC | 09:26 | |
| *** jtomasek has joined #tripleo | 09:32 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Support HA container tagging with podman https://review.openstack.org/614830 | 09:37 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Rework neutron/own agent wrapper tools for podman https://review.openstack.org/606975 | 09:37 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Set setype for log and persistant data directories https://review.openstack.org/611801 | 09:37 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Move "/run/libvirt" shared directory to a persistent location https://review.openstack.org/615956 | 09:37 |
| * Tengu hates git review | 09:38 | |
| Tengu | -.- | 09:38 |
| Tengu | seriously, this tool lacks a proper "dry-run"....... | 09:38 |
| Tengu | "ir might or might not maybe probably update the following changes, but no guarantee nor certitude". yay | 09:39 |
| Tengu | aaaanyway. | 09:40 |
| quiquell | What tool ? | 09:42 |
| Tengu | quiquell: git-review | 09:42 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-quickstart master: Move standalone job to podman instead of docker https://review.openstack.org/613975 | 09:43 |
| *** panda|off is now known as panda | 09:43 | |
| Tengu | when you work on a stack of patches, there is now way to really know what will be done with "git review" -.-. | 09:43 |
| Tengu | --dry-run will show you "new branch". yay. great help. | 09:44 |
| Tengu | anyway. | 09:44 |
| *** derekh has joined #tripleo | 09:44 | |
| Tengu | I apparently didn't break anything this time -.-' | 09:44 |
| quiquell | Tengu: maybe you canl help me, I have a lot of "Unknown function" at puppet at one review | 09:45 |
| quiquell | Tengu: It has Depends-On changes at some puppet projects, could be that we need a promotion to make it work ? | 09:45 |
| quiquell | http://logs.openstack.org/79/615479/18/check/tripleo-ci-centos-7-containers-multinode/4b2c753/logs/undercloud/home/zuul/install-undercloud.log.txt.gz | 09:45 |
| Tengu | quiquell: ah, I also have a Unknown function: 'normalize_ip_for_uri' - same for you? | 09:46 |
| quiquell | Yep | 09:47 |
| *** agurenko has quit IRC | 09:47 | |
| Tengu | guess we need a prom' then... | 09:47 |
| quiquell | Tengu: so ci is broken ? | 09:47 |
| quiquell | Let me check noop jobs | 09:48 |
| Tengu | https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/swift/proxy.pp#L145 | 09:48 |
| Tengu | o___O | 09:49 |
| quiquell | Tengu: It's pretty broken https://review.openstack.org/#/c/560445/ | 09:49 |
| Tengu | that function was added 2 (TWO) years ago. | 09:49 |
| quiquell | sshnaidm|afk: Are we broken ? https://review.openstack.org/#/c/560445/ | 09:49 |
| Tengu | quiquell: I think we're broken. | 09:51 |
| Tengu | yay. | 09:51 |
| Tengu | quiquell: ah, and I have this issue locally, against master. | 09:51 |
| quiquell | Tengu: so puppet-triploe is calling normalize_ip_for_uri and that's not present and puppet-nova ? | 09:52 |
| quiquell | Tengu: It's like this ? | 09:52 |
| Tengu | quiquell: no idea where that function should be defined in fact. | 09:52 |
| Tengu | I know there are multiple calls from within puppet-tripleo | 09:52 |
| * Tengu digging | 09:53 | |
| Tengu | quiquell: what's the "hound" uri already? I didn't bookmark it | 09:54 |
| quiquell | codesearch.openstack.org | 09:55 |
| Tengu | thanks. | 09:55 |
| shardy | jistr: Hey is the failure of tripleo-ci-centos-7-containerized-undercloud-upgrades expected? | 09:55 |
| shardy | jistr: https://review.openstack.org/#/c/611800/ fails and it seems unrelated, we're checking for a barbican systemd service that doesn't exist | 09:55 |
| shardy | wanted to check if that's a known issue or not | 09:55 |
| quiquell | shardy: jobs are broken | 09:56 |
| Tengu | http://git.openstack.org/cgit/openstack/fuel-plugin-tempest/tree/deployment_scripts/puppet/modules/openstacklib/lib/puppet/parser/functions/normalize_ip_for_uri.rb http://git.openstack.org/cgit/openstack/puppet-openstacklib/tree/lib/puppet/functions/normalize_ip_for_uri.rb | 09:56 |
| Tengu | here are current, existing definitions of this function. | 09:56 |
| shardy | quiquell: Ok, I'm trying to figure out if the upgrade with that patch is OK, since that's why it was previously reverted | 09:57 |
| Tengu | quiquell: care to create an LP with all the right tags? | 09:57 |
| Tengu | hmmm. apparently the function was moved from /lib/puppet/functions to /lib/puppet/parser/functions | 09:58 |
| Tengu | and that's not the only one. | 09:58 |
| Tengu | http://git.openstack.org/cgit/openstack/puppet-openstacklib/commit/?id=fd3156bc60ec537ffc53aea1d0e0035c9fe17638 | 09:58 |
| quiquell | Tengu: I see others | 09:58 |
| Tengu | quiquell: definitions, or calls ? | 09:58 |
| quiquell | failing calls | 09:59 |
| Tengu | ah, yeah, well, that's "normal" I'd say. | 09:59 |
| Tengu | apparently the functions were moved to another subdir | 09:59 |
| Tengu | *maybe* we have some issues with puppet version. | 09:59 |
| quiquell | Tengu: So we have to adapt the puppets that depend on that I suppose | 09:59 |
| *** kopecmartin|off is now known as kopecmartin | 09:59 | |
| Tengu | you will probably have the normalize_ip_for_uri, os_database_connection and os_transport_url, among others probably. | 10:00 |
| quiquell | Tengu: Let me open a LP so we track this sh.. | 10:00 |
| Tengu | according to http://git.openstack.org/cgit/openstack/puppet-openstacklib/commit/?id=fd3156bc60ec537ffc53aea1d0e0035c9fe17638 | 10:00 |
| Tengu | (note: NOT sure this is the root cause, but it smells like it) | 10:00 |
| Tengu | timeline seems to match as well. | 10:00 |
| jistr | shardy: i don't think it's expected. Looks like we're using Podman CLI when the container probably still runs on Docker. | 10:01 |
| jistr | http://logs.openstack.org/00/611800/6/check/tripleo-ci-centos-7-containerized-undercloud-upgrades/c174c2f/logs/undercloud/home/zuul/undercloud_upgrade.log.txt.gz#_2018-11-06_11_35_42 | 10:01 |
| jistr | likely due to https://review.openstack.org/#/c/615601/ | 10:01 |
| *** sshnaidm|afk is now known as sshnaidm|rover | 10:01 | |
| jistr | cc EmilienM ^ | 10:01 |
| jistr | shardy: but yes it looks unrelated to your patch | 10:03 |
| shardy | jistr: ack, any thoughts on how we unblock that revert? I'm not quite clear which job was failing when it was reverted tbh | 10:03 |
| * shardy losing motivation to rebase it every day... | 10:03 | |
| shardy | marios: ^^ also | 10:03 |
| sshnaidm|rover | quiquell, yep..we ar | 10:04 |
| shardy | there are other patches blocked by that patch as well ref https://review.openstack.org/#/q/status:open+project:openstack/tripleo-heat-templates+branch:master+topic:bug/1792613 | 10:04 |
| jistr | shardy: it was probably the same job failing, but due to a different reason. I'll try to come up with a fix for this. | 10:04 |
| quiquell | sshnaidm|rover: Openning a bug | 10:04 |
| shardy | and some more I've not yet written | 10:04 |
| shardy | jistr: Ok thanks, any help much appreciated! :) | 10:04 |
| quiquell | sshnaidm|rover, Tengu: https://bugs.launchpad.net/tripleo/+bug/1802083 | 10:05 |
| openstack | Launchpad bug 1802083 in tripleo "Undercloud install fails at Unknown function: 'normalize_ip_for_uri'" [Critical,Confirmed] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 10:05 |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802083 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged] | 10:10 |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 10:10 |
| openstack | Launchpad bug 1802083 in tripleo "Undercloud install fails at Unknown function: 'normalize_ip_for_uri'" [Critical,Confirmed] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 10:10 |
| *** phuongnh has quit IRC | 10:10 | |
| *** iurygregory is now known as iury|afk | 10:11 | |
| quiquell | Tengu, sshnaidm|rover: Use puppet4 functions-api | 10:15 |
| quiquell | I mean https://review.openstack.org/#/q/message:+Use+puppet4+functions-api | 10:15 |
| quiquell | Looks like some puppet projects are missing there | 10:15 |
| quiquell | They are not merged | 10:15 |
| Tengu | missing w+1 apparently. | 10:15 |
| Tengu | they should have been linked in order to avoid the current issue. | 10:16 |
| *** dsneddon has quit IRC | 10:16 | |
| quiquell | Yep... :-( | 10:16 |
| quiquell | We need standalone jobs there | 10:16 |
| quiquell | Humm I think we are running them now | 10:16 |
| Tengu | shardy: would be so kind and take care of the missing +w on the unmerged reviews here? https://review.openstack.org/#/q/message:+Use+puppet4+functions-api :) | 10:16 |
| openstackgerrit | Marios Andreou proposed openstack-infra/tripleo-ci master: Capture json console and write it to workspace for reproducer https://review.openstack.org/615191 | 10:16 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-quickstart-extras master: Introduce zuul.projects and executed pre-run playbooks for reproducer https://review.openstack.org/613572 | 10:16 |
| quiquell | Tengu: aragg but not at openstacklib project :-( | 10:17 |
| quiquell | Tengu: we have to run a full standalone stuff at openstacklib | 10:17 |
| marios | shardy: looking (is that revert revert still not merged?!¬) | 10:17 |
| Tengu | quiquell: that would be great indeed. | 10:17 |
| quiquell | Tengu: humm chkumar is at PTO | 10:17 |
| quiquell | Tengu: Let's see if we can replicate what he did | 10:17 |
| quiquell | sshnaidm|rover: Will try to add stanadalone to puppet-openstacklib | 10:18 |
| shardy | marios: thanks, yeah it's still not merged... | 10:18 |
| * shardy grumbles about CI | 10:18 | |
| sshnaidm|rover | quiquell, yeah, seems like we need that | 10:18 |
| Tengu | shardy: :D | 10:18 |
| sshnaidm|rover | posted revert: https://review.openstack.org/#/c/616140/ | 10:18 |
| shardy | Tengu: sure will check the reviews | 10:19 |
| sshnaidm|rover | EmilienM, mwhahaha ^^ | 10:19 |
| jistr | shardy: are you ok if i rebase your patch on the fix for the docker/podman issue? | 10:19 |
| Tengu | sshnaidm|rover: no, please no | 10:19 |
| Tengu | shardy: thanks! | 10:19 |
| Tengu | sshnaidm|rover: we need either all, either none of those changes :/. | 10:19 |
| sshnaidm|rover | Tengu, yeah, that's why I posted a revert) | 10:19 |
| sshnaidm|rover | none of them is merged and one is even -w | 10:20 |
| Tengu | sshnaidm|rover: understaffed for core I think. | 10:20 |
| shardy | Tengu: I don't have core on any puppet-repos except the puppet-tripleo repo | 10:20 |
| *** salmankhan has joined #tripleo | 10:20 | |
| Tengu | shardy: ;_; | 10:21 |
| *** suuuper has joined #tripleo | 10:21 | |
| Tengu | shardy: ah, good catch for the puppet-nova. so yeah. revert is fine. | 10:21 |
| Tengu | although we have to find core for that one anyway. | 10:21 |
| Tengu | sshnaidm|rover: -^^ sorry | 10:21 |
| Tengu | E_TABFAIL | 10:21 |
| Tengu | that was a nice team work, sshnaidm|rover quiquell :) | 10:23 |
| Tengu | now is time for food :D | 10:23 |
| quiquell | Tengu: In a few we have some tripleo coverage there | 10:23 |
| openstackgerrit | Christian Schwede proposed openstack/tripleo-heat-templates master: Check if source directory does exist before linking https://review.openstack.org/613912 | 10:24 |
| quiquell | sshnaidm|rover, Tengu: standalone coverage with full tempest on it https://review.openstack.org/616144 | 10:27 |
| quiquell | sshnaidm|rover: Maybe you want to do this the parent to check that it's passing with the revert | 10:28 |
| quiquell | I can do the opposite | 10:29 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates master: Handle difference between future and current container_cli https://review.openstack.org/616146 | 10:29 |
| jistr | shardy: i think this could do it ^ | 10:30 |
| * jistr also reported an alert bug | 10:30 | |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: DNM: To test standalone-28 https://review.openstack.org/615479 | 10:31 |
| shardy | jistr: thanks! :) | 10:32 |
| *** rfolco|ruck has joined #tripleo | 10:37 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-quickstart-extras master: Introduce zuul.projects and executed pre-run playbooks for reproducer https://review.openstack.org/613572 | 10:39 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-quickstart-extras master: Introduce zuul.projects and executed pre-run playbooks for reproducer https://review.openstack.org/613572 | 10:41 |
| openstackgerrit | Christian Schwede proposed openstack/tripleo-heat-templates master: Do not ignore Swift ring changes to trigger container restart https://review.openstack.org/616116 | 10:45 |
| *** dtantsur|afk is now known as dtantsur | 10:47 | |
| openstackgerrit | Christian Schwede proposed openstack/tripleo-heat-templates master: Do not ignore Swift ring changes to trigger container restart https://review.openstack.org/616116 | 10:47 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Run atop for monitoring deployment https://review.openstack.org/615357 | 10:51 |
| openstackgerrit | Natal Ngétal proposed openstack/tripleo-heat-templates master: [Templates] Use str_replace for hosts. https://review.openstack.org/616157 | 10:53 |
| *** agurenko has joined #tripleo | 10:57 | |
| *** ksambor has quit IRC | 11:04 | |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802083 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802085 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged] | 11:10 |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 11:10 |
| openstack | Launchpad bug 1802083 in tripleo "Undercloud install fails at Unknown function: 'normalize_ip_for_uri'" [Critical,Confirmed] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 11:10 |
| openstack | Launchpad bug 1802085 in tripleo "containerized UC upgrade job failing on container_cli discrepancy" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 11:10 |
| *** morazi has quit IRC | 11:16 | |
| sshnaidm|rover | rfolco|ruck, I updated https://review.rdoproject.org/etherpad/p/ruckrover-sprint21 with latest issues | 11:17 |
| sshnaidm|rover | rfolco|ruck, feel free to add there if I missed something | 11:17 |
| rfolco|ruck | sshnaidm|rover, is this the same issue as "panic: runtime error: invalid memory address or nil pointer dereference" ? | 11:17 |
| rfolco|ruck | sshnaidm|rover, the gate ... | 11:17 |
| sshnaidm|rover | rfolco|ruck, no, this is different | 11:17 |
| sshnaidm|rover | rfolco|ruck, do we have a bug about "panic: runtime error"? | 11:18 |
| rfolco|ruck | sshnaidm|rover, I did not find one | 11:18 |
| rfolco|ruck | opening | 11:18 |
| sshnaidm|rover | rfolco|ruck, check please if it happened before gates were broken | 11:18 |
| rfolco|ruck | ok | 11:18 |
| *** huynq has quit IRC | 11:19 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Move "/run/libvirt" shared directory to a persistent location https://review.openstack.org/615956 | 11:21 |
| openstackgerrit | Yurii Prokulevych proposed openstack/tripleo-upgrade stable/rocky: Add cli_opts_params.yaml to update prepare/converge. https://review.openstack.org/613960 | 11:30 |
| *** florianf has quit IRC | 11:30 | |
| *** ssbarnea has quit IRC | 11:34 | |
| openstackgerrit | Slawek Kaplonski proposed openstack/tripleo-heat-templates master: Remove external_network_bridge Neutron option https://review.openstack.org/604764 | 11:36 |
| *** thrash|g0ne is now known as thrash | 11:38 | |
| *** ksambor has joined #tripleo | 11:38 | |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: DNM: To test standalone-28 https://review.openstack.org/615479 | 11:41 |
| sshnaidm|rover | rfolco|ruck, better to discuss it here | 11:47 |
| *** florianf has joined #tripleo | 11:48 | |
| rfolco|ruck | EmilienM, can you please clarify on podman issue https://bugs.launchpad.net/tripleo/+bug/1800737... podman was disabled, then reverted. We still see timeouts and panic runtime errors... | 11:49 |
| openstack | Launchpad bug 1800737 in tripleo "relabel failed /var/lib/config-data: no such file or directory" [Critical,Fix released] - Assigned to Emilien Macchi (emilienm) | 11:49 |
| *** ssbarnea has joined #tripleo | 11:52 | |
| *** snecker has joined #tripleo | 11:52 | |
| *** snecker is now known as snecklifter | 11:53 | |
| *** snecklifter has quit IRC | 11:54 | |
| *** snecklifter has joined #tripleo | 11:54 | |
| *** jfrancoa has quit IRC | 11:58 | |
| openstackgerrit | Yurii Prokulevych proposed openstack/tripleo-upgrade stable/queens: Pass stack name during minor update. https://review.openstack.org/616173 | 11:59 |
| *** jfrancoa has joined #tripleo | 11:59 | |
| *** akrzos has joined #tripleo | 12:00 | |
| quiquell | sshnaidm|rover: Coverage working https://github.com/openstack-infra/openstack-zuul-jobs | 12:03 |
| quiquell | Tengu: ^ | 12:03 |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart master: Enable smoke tests and some tempest scenario on standalone https://review.openstack.org/615133 | 12:05 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: Remove bogus "EXPERIMENTAL" in neutron-sriov.yaml environment file https://review.openstack.org/584678 | 12:05 |
| sshnaidm|rover | quiquell, what do you mean? | 12:07 |
| sshnaidm|rover | rfolco|ruck, I'm merging https://review.rdoproject.org/r/#/c/17285/ please check on host that it works without errors | 12:07 |
| Tengu | quiquell: good for the next steps :) | 12:07 |
| quiquell | sshnaidm|rover: just that standalone is failing as expected | 12:07 |
| Tengu | sshnaidm|rover: I'm faster at recheck :D | 12:07 |
| rfolco|ruck | sshnaidm|rover, thanks | 12:07 |
| sshnaidm|rover | quiquell, ok :) | 12:07 |
| rfolco|ruck | sshnaidm|rover, I am filing a new bug on podman, as the relabel error is gone and panic remains | 12:08 |
| sshnaidm|rover | rfolco|ruck, great, thanks | 12:09 |
| jaosorior | Anybody seen this? | 12:09 |
| sshnaidm|rover | EmilienM, mwhahaha please merge is asap, it's blocking everything: https://review.openstack.org/#/c/616140/ | 12:09 |
| jaosorior | 2018-11-07 11:29:32 | "Error: Evaluation Error: Unknown function: 'os_transport_url'. at /etc/puppet/modules/tripleo/manifests/profile/base/keystone.pp:217:37 on node undercloud.localdomain", | 12:09 |
| Tengu | jaosorior: we're on it already | 12:10 |
| Tengu | jaosorior: you're slow today ;) | 12:10 |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 12:10 |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802083 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802085 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 12:10 |
| openstack | Launchpad bug 1802083 in tripleo "Undercloud install fails at Unknown function: 'normalize_ip_for_uri'" [Critical,Confirmed] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 12:10 |
| openstack | Launchpad bug 1802085 in tripleo "containerized UC upgrade job failing on container_cli discrepancy" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 12:10 |
| jaosorior | Tengu: yeah, still working on stuff for the Summit :/ | 12:10 |
| Tengu | jaosorior: same as the normalize_foo thingy ;) | 12:10 |
| Tengu | jaosorior: basically, a commit "hides" all the functions for puppet. | 12:10 |
| sshnaidm|rover | we need puppet cores in emea time | 12:11 |
| jaosorior | sshnaidm|rover: thanks for rechecking, although the hostname patch failed on the puppet function issue | 12:11 |
| jaosorior | so... the recheck will fail on that too I'm afraid :/ | 12:11 |
| sshnaidm|rover | jaosorior, yeah, I see | 12:11 |
| Tengu | jaosorior: you might depend on https://review.openstack.org/#/c/616140/ | 12:11 |
| jaosorior | I' | 12:11 |
| jaosorior | I'll add that | 12:11 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-quickstart-extras master: Set undercloud hostname from ansible variables https://review.openstack.org/615730 | 12:12 |
| sshnaidm|rover | who does have puppet repos superpowers here? | 12:12 |
| Tengu | there's a channel for puppet openstack things iirc | 12:12 |
| jaosorior | sshnaidm|rover: iury|afk could check it out | 12:13 |
| jaosorior | though I see he's afk :/ | 12:13 |
| Tengu | sshnaidm|rover: :#openstack-puppet | 12:13 |
| Tengu | I'm on it, asking for votes. | 12:13 |
| Tengu | sorry, #puppet-openstack sshnaidm|rover | 12:14 |
| Tengu | -.- they didn't put a redirect -.- | 12:14 |
| sshnaidm|rover | Tengu, yeah, we talked with Tobias there | 12:14 |
| Tengu | sshnaidm|rover: oh, ok. | 12:14 |
| Tengu | well. | 12:14 |
| Tengu | they'll get a new reminder to vote :) | 12:15 |
| sshnaidm|rover | he's ok, but seems like other cores are in western hemisphere | 12:15 |
| *** florianf has quit IRC | 12:16 | |
| Tengu | sshnaidm|rover: EmilienM has core on that one I think, and should be online shortly hopefully. | 12:16 |
| snecklifter | folks, I'm not sure that an answers-file supports composable roles and networks, correct? | 12:18 |
| Tengu | jaosorior: can you change the title and tell the CI is dead for now? | 12:18 |
| snecklifter | when I try and use roles and networks under environments, I get invalid YAML which is expected | 12:20 |
| openstackgerrit | Dmitry Tantsur proposed openstack/diskimage-builder master: Add an element to configure iBFT network interfaces https://review.openstack.org/391787 | 12:20 |
| jaosorior | #topic DO NOT MERGE PATCHES | CI Status: A sad apple (RED) | https://docs.openstack.org/tripleo-docs/latest/ | 12:21 |
| jaosorior | agh, confused the command with the one I use for meetings lol | 12:22 |
| *** jaosorior changes topic to "DO NOT MERGE PATCHES | CI Status: A sad apple (RED) | https://docs.openstack.org/tripleo-docs/latest/" | 12:22 | |
| Tengu | jaosorior: thanks :) | 12:22 |
| *** raildo has joined #tripleo | 12:24 | |
| Tengu | jaosorior: do you want me to send a thing on the openstack-dev ML for the CI status? or is it "too much"? | 12:25 |
| jaosorior | probably a good idea | 12:26 |
| Tengu | I'm on it then. | 12:26 |
| naturalblue | Hi Everyone. I was able to fnally get undercloud installed with Triple UI on the routable/MGMT NIC. The only issue I have is hiera gives back a nil instead of a password when I run it "sudo hiera admin_password". Can anyone suggest why it is not reading the password details | 12:26 |
| jaosorior | Tengu: sent | 12:29 |
| Tengu | jaosorior: ah, thanks | 12:29 |
| jaosorior | naturalblue: you might need to add the "-c /etc/puppet/hiera.yaml" parameter too | 12:29 |
| *** jfrancoa has quit IRC | 12:30 | |
| *** radez has joined #tripleo | 12:31 | |
| naturalblue | jaosorior: cool let me check | 12:35 |
| naturalblue | jaosorior: that also just replies back "nil" | 12:36 |
| EmilienM | Tengu: what's up | 12:36 |
| Tengu | EmilienM: CI/gate broken, but we should have to votes for the corrective patch https://review.openstack.org/#/c/616140/ | 12:37 |
| EmilienM | ok | 12:37 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-docs master: Fix args order for tripleo-repos command https://review.openstack.org/611519 | 12:40 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-docs master: Add simple diagram for contributor guide promotion stages https://review.openstack.org/616187 | 12:46 |
| *** jfrancoa has joined #tripleo | 12:47 | |
| *** sshnaidm|rover is now known as sshnaidm|afk | 12:50 | |
| openstackgerrit | Tobias Urdin proposed openstack/puppet-tripleo master: Add openstacklib to requirements in metadata.json https://review.openstack.org/616188 | 12:51 |
| *** jpena is now known as jpena|lunch | 12:53 | |
| *** rlandy has joined #tripleo | 12:54 | |
| naturalblue | jaosorior: when I "cat /etc/puppet/hieradata/service_configs.json" there is passwords for almost everything but i still cant see the tripleo ui admin password | 12:55 |
| *** morazi has joined #tripleo | 12:58 | |
| *** vpickard_ is now known as vpickard | 13:00 | |
| jaosorior | naturalblue: yeah, nto sure how to get that. | 13:03 |
| *** nawar has left #tripleo | 13:04 | |
| *** eck`gone is now known as eck` | 13:05 | |
| weshay | jistr++ | 13:06 |
| *** amoralej is now known as amoralej|lunch | 13:06 | |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802083 | 13:10 |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,In progress] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802085 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802096 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 13:10 |
| openstack | Launchpad bug 1802083 in tripleo "Undercloud install fails at Unknown function: 'normalize_ip_for_uri'" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 13:10 |
| openstack | Launchpad bug 1802085 in tripleo "containerized UC upgrade job failing on container_cli discrepancy" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 13:10 |
| openstack | Launchpad bug 1802096 in tripleo "(containers) panic: runtime error: invalid memory address or nil pointer dereference" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 13:10 |
| EmilienM | sshnaidm|afk: https://bugs.launchpad.net/tripleo/+bug/1802096 | 13:12 |
| openstack | Launchpad bug 1802096 in tripleo "(containers) panic: runtime error: invalid memory address or nil pointer dereference" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 13:12 |
| EmilienM | i'm taking it | 13:12 |
| EmilienM | Tengu: ^ we need to take a look at this one | 13:13 |
| EmilienM | Tengu: red herring | 13:14 |
| EmilienM | I'm commenting | 13:14 |
| EmilienM | rfolco|ruck: I closed https://bugs.launchpad.net/tripleo/+bug/1802096 - see in the comment why | 13:15 |
| openstack | Launchpad bug 1802096 in tripleo "(containers) panic: runtime error: invalid memory address or nil pointer dereference" [Critical,Invalid] - Assigned to Emilien Macchi (emilienm) | 13:15 |
| Tengu | EmilienM: ah, so now it's a thing? :( | 13:15 |
| EmilienM | Tengu: no | 13:16 |
| EmilienM | Tengu: it's not, see my comment | 13:16 |
| Tengu | yep, just saw it | 13:16 |
| *** dprince has joined #tripleo | 13:18 | |
| *** jpena|lunch is now known as jpena | 13:18 | |
| *** trown|outtypewww is now known as trown | 13:21 | |
| quiquell | EmilienM: Have a 'No module names paunch" at the f28 job, maybe you know why http://logs.openstack.org/79/615479/20/check/tripleo-ci-fedora-28-standalone/81f34f1/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2018-11-07_12_27_53 | 13:21 |
| quiquell | Could be that we are not installing paunch at fedora | 13:22 |
| *** ansmith has joined #tripleo | 13:30 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates master: Enable _member_ role for undercloud install. https://review.openstack.org/611677 | 13:30 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates master: Enable _member_ role for undercloud install. https://review.openstack.org/611677 | 13:31 |
| openstackgerrit | Adriano Petrich proposed openstack/python-tripleoclient master: Change the overcloud delete command to use undeploy_plan https://review.openstack.org/616199 | 13:32 |
| *** vinaykns has joined #tripleo | 13:32 | |
| *** agurenko has quit IRC | 13:36 | |
| *** diw has quit IRC | 13:37 | |
| *** lblanchard has joined #tripleo | 13:37 | |
| chem | weshay: hi, can we push https://review.openstack.org/#/c/604706/ and https://review.openstack.org/#/c/604736/ to the gate, I think there are good to go (unless some upgrade error krept in) | 13:38 |
| *** iury|afk is now known as iurygregory | 13:39 | |
| iurygregory | jaosorior, hey o/ whats up? | 13:39 |
| weshay | chem, totally.. give us a few to review | 13:40 |
| chem | weshay: ack, thanks. | 13:41 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Fix runtime Ansible warnings https://review.openstack.org/611844 | 13:43 |
| Tengu | quiquell: revert in Gate! finger crossed :D | 13:44 |
| jaosorior | iurygregory: hey, we needed some review for a puppet patch, but it's been handled. thanks though :D | 13:44 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-heat-templates master: Use correct python version for puppet-docker.py https://review.openstack.org/616203 | 13:44 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-heat-templates master: Use correct python version for puppet-docker.py https://review.openstack.org/616203 | 13:45 |
| quiquell | Tengu: yey !!! | 13:45 |
| quiquell | Tengu: let's also mege the coverage | 13:45 |
| Tengu | quiquell: :) | 13:45 |
| *** pcaruana has quit IRC | 13:45 | |
| iurygregory | jaosorior, oh sorry i was onboarding a new hire =( if you need anything else please let me know | 13:46 |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: DNM: To test standalone-28 https://review.openstack.org/615479 | 13:46 |
| *** agopi has quit IRC | 13:47 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-specs master: Bare metal provisioning without Nova and Glance https://review.openstack.org/604761 | 13:47 |
| larsks | When a stack update fails and I'm looking at 'openstack stack failures list --long', how do I tell on which host the failure occurred? | 13:48 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates master: Crane: Add deployment/crane-puppet.yaml https://review.openstack.org/609508 | 13:49 |
| *** naturalblue has quit IRC | 13:56 | |
| *** jfrancoa has quit IRC | 14:00 | |
| *** pcaruana has joined #tripleo | 14:00 | |
| *** jfrancoa has joined #tripleo | 14:00 | |
| *** mcornea has joined #tripleo | 14:03 | |
| *** florianf has joined #tripleo | 14:05 | |
| *** radeks_ has joined #tripleo | 14:06 | |
| *** bfournie has quit IRC | 14:06 | |
| weshay | ooh http://logs.openstack.org/40/591540/73/check/tripleo-ci-fedora-28-standalone/e5d78d3/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz | 14:07 |
| *** radeks has quit IRC | 14:07 | |
| *** cfontain has joined #tripleo | 14:08 | |
| *** toure|gone is now known as toure | 14:09 | |
| openstackgerrit | Merged openstack/tripleo-upgrade stable/rocky: Set containerized_undercloud_upgrade to true https://review.openstack.org/615511 | 14:09 |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802083 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802085 | 14:10 |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,In progress] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 14:10 |
| openstack | Launchpad bug 1802083 in tripleo "Undercloud install fails at Unknown function: 'normalize_ip_for_uri'" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 14:10 |
| openstack | Launchpad bug 1802085 in tripleo "containerized UC upgrade job failing on container_cli discrepancy" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 14:10 |
| *** cdearborn has joined #tripleo | 14:16 | |
| *** agopi has joined #tripleo | 14:17 | |
| *** rh-jelabarre has joined #tripleo | 14:21 | |
| *** tzumainn has joined #tripleo | 14:24 | |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates master: Handle LP openvswitch meta-package on upgrade https://review.openstack.org/605200 | 14:25 |
| *** amoralej|lunch is now known as amoralej | 14:25 | |
| openstackgerrit | Natal Ngétal proposed openstack/tripleo-heat-templates master: [Templates] Use str_replace for hosts. https://review.openstack.org/616157 | 14:31 |
| *** mjturek has joined #tripleo | 14:32 | |
| *** pcaruana has quit IRC | 14:33 | |
| *** pcaruana has joined #tripleo | 14:34 | |
| *** florianf has quit IRC | 14:38 | |
| *** f_ has joined #tripleo | 14:39 | |
| *** f_ is now known as florianf | 14:39 | |
| *** bfournie has joined #tripleo | 14:39 | |
| openstackgerrit | Natal Ngétal proposed openstack/tripleo-heat-templates master: [Templates] Use str_replace for hosts. https://review.openstack.org/616157 | 14:42 |
| *** jcoufal has joined #tripleo | 14:43 | |
| openstackgerrit | Dan Prince proposed openstack/python-tripleoclient master: WIP: test enable crane for the undercloud https://review.openstack.org/616214 | 14:48 |
| openstackgerrit | Thomas Herve proposed openstack/paunch master: Remove log message when container doesn't exist https://review.openstack.org/616216 | 14:50 |
| openstackgerrit | Quique Llorente proposed openstack/tripleo-heat-templates master: Use correct python version for puppet-docker.py https://review.openstack.org/616203 | 14:53 |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: DNM: To test standalone-28 https://review.openstack.org/615479 | 14:53 |
| *** ade_lee has joined #tripleo | 14:54 | |
| *** aufi has quit IRC | 14:54 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-heat-templates master: Move "/run/libvirt" shared directory to a persistent location https://review.openstack.org/615956 | 14:55 |
| *** jistr is now known as jistr|call | 15:00 | |
| Tengu | quiquell: Merged openstack/puppet-openstacklib master: Revert "Use puppet4 functions-api" https://review.openstack.org/616140 \o/ | 15:03 |
| Tengu | we can breath again. jaosorior you might want to send the "it's ok" email maybe? -^^ | 15:03 |
| Tengu | although it's more "more or less orange-ish OK". | 15:04 |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802083 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802085 | 15:10 |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,In progress] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 15:10 |
| openstack | Launchpad bug 1802083 in tripleo "Undercloud install fails at Unknown function: 'normalize_ip_for_uri'" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 15:10 |
| openstack | Launchpad bug 1802085 in tripleo "containerized UC upgrade job failing on container_cli discrepancy" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 15:10 |
| *** rpioso|afk is now known as rpioso | 15:13 | |
| quiquell | Tengu: let's merge the coverage first | 15:17 |
| quiquell | Tengu: before we send the email | 15:17 |
| quiquell | jaosorior: ^ | 15:17 |
| jaosorior | quiquell: lets do that | 15:17 |
| jaosorior | quiquell: what's the patch? | 15:17 |
| jaosorior | sshnaidm|afk, Tengu: should we close this https://bugs.launchpad.net/tripleo/+bug/1802083 ? | 15:17 |
| openstack | Launchpad bug 1802083 in tripleo "Undercloud install fails at Unknown function: 'normalize_ip_for_uri'" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 15:17 |
| quiquell | jaosorior, Tengu: https://review.openstack.org/#/c/616144 | 15:18 |
| Tengu | jaosorior: maybe wait for the coverage merge? quiquell ? | 15:18 |
| quiquell | It has to pass now | 15:18 |
| quiquell | did recheck | 15:18 |
| *** agurenko has joined #tripleo | 15:19 | |
| openstackgerrit | Christian Schwede proposed openstack/tripleo-heat-templates master: Restrict number of Swift workers to 1 on the undercloud https://review.openstack.org/616221 | 15:19 |
| *** gkadam has joined #tripleo | 15:20 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart master: standalone support for quickstart on libvirt https://review.openstack.org/591540 | 15:21 |
| chem | bandini: I've tried to reproduce the ubuntu error for 4.8.2 in there https://review.openstack.org/#/c/609998/ | 15:24 |
| chem | bandini: without any luck, it's working locally. Would you mind If I tried a debug run to check what is happening in the ci ? | 15:24 |
| chem | bandini: I would need to push a review | 15:24 |
| *** aufi has joined #tripleo | 15:25 | |
| *** ayoung has joined #tripleo | 15:27 | |
| ayoung | ade_lee, jaosorior Is it possible to rotate passwords on the undercloud without doing a redeploy? | 15:28 |
| jaosorior | ayoung: not at the moment. | 15:28 |
| ayoung | jaosorior, is it even possible to change them at all? | 15:29 |
| ayoung | Does the undercloud still use Keystone? It must, right? | 15:29 |
| jaosorior | ayoung: it does. | 15:29 |
| ayoung | so the question is, how would a customer with strict password rotation policy handle changing, say, the nova service users password | 15:29 |
| *** jistr|call is now known as jistr | 15:29 | |
| jaosorior | and it is possible to change the passwords. However, there have been bugs related to it... once we get the switch to standalone jobs going (and CI is more stable) there is a proposal to add a job that tests the password changes. | 15:30 |
| jaosorior | ayoung: they do need to redeploy unfortunately. | 15:30 |
| ade_lee | ayoung, see pm | 15:30 |
| ayoung | jaosorior, if I wanted to change the Ironic service users password, I could manually update the conf file, and the Keystone server using, say, an Ansible playbook. There is no Puppet to manage the undercloud, right? | 15:32 |
| jaosorior | ayoung: puppet still manages the undercloud's configuration | 15:32 |
| ayoung | jaosorior, so it gets pushed out from Heat? | 15:33 |
| jaosorior | you could do it via ansible, however, you would also need to udpate the passwords file which is used to persist the undercloud's passwords. | 15:33 |
| jaosorior | And, since we're not yet testing that scenario, I'm not sure if that works :/. Would need to try it out in a lab and see | 15:33 |
| ayoung | of course | 15:34 |
| *** Vorrtex has joined #tripleo | 15:34 | |
| ayoung | jaosorior, OK, I think I have enough to manage customer expectations accordingly. TYVM | 15:35 |
| *** dtantsur is now known as dtantsur|afk | 15:42 | |
| openstackgerrit | Carlos Goncalves proposed openstack/tripleo-common master: Move Octavia config opts to common config directory https://review.openstack.org/596373 | 15:43 |
| *** vpickard is now known as vpickard_ | 15:46 | |
| *** kopecmartin is now known as kopecmartin|off | 15:51 | |
| *** ksambor has quit IRC | 15:53 | |
| openstackgerrit | Merged openstack/tripleo-docs master: Fix args order for tripleo-repos command https://review.openstack.org/611519 | 15:54 |
| quiquell | mwhahaha: Can you merge this ? https://review.openstack.org/#/c/615886 | 15:55 |
| mwhahaha | quiquell: no | 15:55 |
| quiquell | therve: who else can ? | 15:55 |
| mwhahaha | it's not a tripleo project | 15:55 |
| quiquell | EmilienM: ^ ? | 15:55 |
| EmilienM | I can't either | 15:56 |
| therve | Let me do it | 15:56 |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates stable/queens: GATE CHECK for TripleO https://review.openstack.org/567224 | 16:00 |
| mwhahaha | EmilienM, dprince, sshnaidm|afk, weshay, panda: it would be nice to get https://review.openstack.org/#/q/topic:container-build-job+status:open so we can get the container build in check. | 16:01 |
| EmilienM | k | 16:02 |
| mandre | has anyone tried to provision an undercloud with quickstart recently? mine failed because of (many) puppet errors while generating the config files for the container | 16:04 |
| mwhahaha | mandre: there was a bug, the revert just landed | 16:04 |
| weshay | aye | 16:04 |
| mandre | mwhahaha: ah, thanks! | 16:05 |
| panda | mwhahaha: this is building all the containers ? | 16:05 |
| mwhahaha | panda: yes | 16:05 |
| panda | we had no check job, but we have the promotion running every 4 hours on this process. | 16:06 |
| mwhahaha | panda: yea and we've merged code that breaks this, it needs a check (especially since it only takes 45 mins) | 16:06 |
| mwhahaha | also we don't build containers if undercloud doesn't install | 16:07 |
| openstackgerrit | Thomas Herve proposed openstack/paunch master: Remove log message when container doesn't exist https://review.openstack.org/616216 | 16:07 |
| mwhahaha | soooo we actually didn't know the grafana thing was an issue (yet) | 16:07 |
| mandre | mwhahaha: do you have the link for the patch so I can apply it locally without re-deploying the undercloud? | 16:09 |
| mwhahaha | mandre: it was in openstacklib, it'd be easier to redeploy the undercloud | 16:10 |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 16:10 |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,In progress] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802083 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802085 | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 16:10 |
| openstack | Launchpad bug 1802083 in tripleo "Undercloud install fails at Unknown function: 'normalize_ip_for_uri'" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 16:10 |
| openstack | Launchpad bug 1802085 in tripleo "containerized UC upgrade job failing on container_cli discrepancy" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 16:10 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-quickstart-extras master: OVB - external network masquerading https://review.openstack.org/609830 | 16:10 |
| mwhahaha | mandre: https://review.openstack.org/#/c/616140/ | 16:10 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-quickstart-extras master: OVB - external network masquerading https://review.openstack.org/609830 | 16:10 |
| ayoung | jaosorior, one last question: is it possible to do an openstack undercloud deploy as a redeploy, and not bring down the services on top of it? If I did that with new passwords, would it "just" change the passwrods, at least in theory? | 16:11 |
| mandre | mwhahaha: ack, I'll redeploy then, thanks | 16:11 |
| jaosorior | ayoung: it would just change the passwords, but it would also restart the containers. | 16:11 |
| ayoung | jaosorior, so there would be a service disruption, but you would not have to redeploy the overcloud? | 16:11 |
| jaosorior | ayoung: if you're changing the undercloud's passwords, correct. there would be service disruption on the undercloud, but the overcloud would just keep working as normal | 16:12 |
| openstackgerrit | Christophe Fontaine proposed openstack/tripleo-heat-templates stable/rocky: Fix NeutronSriovHostConfig path for ODL deployment with SRIOV https://review.openstack.org/612998 | 16:12 |
| jaosorior | ayoung: if you need to change the overcloud passwords, that's another story, then you would have a service disruption there | 16:13 |
| *** agopi is now known as agopi|lunch | 16:14 | |
| ayoung | jaosorior, understood | 16:14 |
| jaosorior | quiquell: so... what was the coverage patch you were talking about? | 16:16 |
| *** lblanchard has quit IRC | 16:16 | |
| EmilienM | mwhahaha: I think we could run this job into kolla's check (maybe non voting to start), since we build kolla with delorean | 16:17 |
| mwhahaha | EmilienM: yes | 16:17 |
| mwhahaha | EmilienM: actualy that job just uses source | 16:17 |
| mwhahaha | no packages | 16:17 |
| EmilienM | mwhahaha: interesting it didn't run on https://review.openstack.org/#/c/615632/ | 16:18 |
| mwhahaha | EmilienM: it doesn't run by default, just on changed files | 16:18 |
| panda | mwhahaha: where are the delorean repo set before copying them in /var/www/html ? is what you're putting in TODO ? | 16:18 |
| *** quiquell is now known as quiquell|off | 16:19 | |
| EmilienM | mwhahaha: I'm sad to see Ansible roles in tripleo-ci repo | 16:19 |
| mwhahaha | panda: https://review.openstack.org/#/c/614392/31/playbooks/tripleo-buildcontainers/pre.yaml@63 | 16:19 |
| mwhahaha | EmilienM: in order to use them in zuulv3 they have to be there :/ | 16:19 |
| EmilienM | yea | 16:19 |
| mwhahaha | https://review.openstack.org/#/c/614392/31/playbooks/tripleo-buildcontainers/pre.yaml@4 | 16:19 |
| honza | jtomasek: florianf: what version of node/npm do we use again? | 16:20 |
| jtomasek | honza: I use node v8.9.4 and npm 5.6.0 | 16:21 |
| mwhahaha | EmilienM: so the containers job only runs when these files are changed: https://review.openstack.org/#/c/614392/31/zuul.d/build-containers.yaml@31 | 16:21 |
| mwhahaha | EmilienM: but it did tun on https://review.openstack.org/#/c/611598/5 | 16:22 |
| EmilienM | mwhahaha: and it failed :P | 16:22 |
| mwhahaha | EmilienM: grafana | 16:22 |
| EmilienM | yaeh | 16:22 |
| EmilienM | it's still in gate? | 16:22 |
| mwhahaha | it hadn't been rechecked since i fixed that | 16:22 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Enable Scale up/redeploy of a deployment https://review.openstack.org/607931 | 16:22 |
| mwhahaha | it failed, i rechecked | 16:22 |
| EmilienM | dang | 16:22 |
| honza | jtomasek: florianf: that sounds right, thanks --- i'll double check the ci setting --- perhaps we should upgrade to lts? | 16:23 |
| jtomasek | honza: sounds good, I noticed there is a new one | 16:24 |
| *** dtrainor has quit IRC | 16:26 | |
| *** jfrancoa has quit IRC | 16:32 | |
| *** aufi has quit IRC | 16:33 | |
| *** pcaruana has quit IRC | 16:38 | |
| *** snecklifter has quit IRC | 16:41 | |
| *** moguimar has quit IRC | 16:41 | |
| panda | mwhahaha: is there a design/discussion/track of what's the idea and what's going to happen here ? e.g. do you plan to add also tripleo-build-containers-fedora-28 ? | 16:43 |
| mwhahaha | panda: this is missing coverage from the original implementation of the build process. Yes int he future it will be a good idea to expand it to also do fedora28 but kolla doesn't support fedora so that is still in a conceptional phase | 16:44 |
| mwhahaha | panda: we have a build images job, but we never actually implemented a build containers CI job and we've broken it twice in like two weeks. It's a preventative action from two CIX | 16:45 |
| mwhahaha | panda: i'm not sure what the particular issue is with adding proper CI coverage to core functionality is | 16:45 |
| *** shardy has quit IRC | 16:46 | |
| panda | why does everybody assume I'm against an idea when I start asking questions ?? :) | 16:46 |
| mwhahaha | https://bugs.launchpad.net/tripleo/+bug/1801969 https://bugs.launchpad.net/tripleo/+bug/1801076 | 16:46 |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 16:46 |
| openstack | Launchpad bug 1801076 in tripleo "Crane container fails to build in promotion because "No package mod_xsendfile available"" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 16:47 |
| mwhahaha | panda: because the way you are asking is not very pro coverage | 16:47 |
| weshay | I voting pro coverage in the upcoming election | 16:47 |
| weshay | vote YES on +2 | 16:47 |
| panda | I'm actually going to -1 this, but the questions are giving me more context | 16:48 |
| dprince | mandre: I created an undercloud today. Hit an issue with the chrony ansible role missing | 16:56 |
| *** agurenko has quit IRC | 16:57 | |
| mwhahaha | dprince: you have an older python-tripleoclient (it was added as a requirement) vs THT | 16:59 |
| florianf | honza, jtomasek: I can't start the dev server after a fresh npm install using v8.9.4 | 17:02 |
| mwhahaha | oh sorry, older openstack-tripleo-heat-tempaltes package thats' where we added the requirement | 17:02 |
| *** irclogbot_3 has joined #tripleo | 17:02 | |
| *** gkadam has quit IRC | 17:04 | |
| *** eck` is now known as eck`gone | 17:04 | |
| florianf | honza, jtomasek: a build seems to work though | 17:07 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-quickstart-extras master: Add custom environment files for standalone https://review.openstack.org/614820 | 17:07 |
| honza | florianf: jtomasek same here | 17:07 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-quickstart master: Enable low memory for standalone https://review.openstack.org/614824 | 17:07 |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802083 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802085 | 17:10 |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,In progress] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 17:10 |
| openstack | Launchpad bug 1802083 in tripleo "Undercloud install fails at Unknown function: 'normalize_ip_for_uri'" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 17:10 |
| openstack | Launchpad bug 1802085 in tripleo "containerized UC upgrade job failing on container_cli discrepancy" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 17:10 |
| honza | arxcruz: kopecmartin|off would you mind voting on this? https://review.openstack.org/#/c/607323/4 | 17:10 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/puppet-pacemaker master: Skip some tests for Ubuntu 18.04 and puppet 4.8. https://review.openstack.org/616249 | 17:13 |
| panda | mwhahaha: not against coverage, this came out of the blue for me, and I'm trying to get a bit more context, starting from the initial need. I got so far: Promotions are not enough which goes to the reason why this is here and I agree to, and there will be more job like this, which goes to the overall idea. Anything else you can tell me at the moment to help me review this ? I usually ask this general | 17:13 |
| panda | question live and the specific question on the review itself, after I have the general informations. | 17:13 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/puppet-pacemaker master: DNM: testing ci. https://review.openstack.org/616250 | 17:14 |
| mwhahaha | panda: i've been working on this job for over a week and with 31 patch sets i would have hoped you would have noticed one of them. If you're not keeping up on the open reviews don't be surpised when things pop up. It's not been secret. | 17:14 |
| mwhahaha | panda: other than this is a long standing tech debt that we should have had 3 releases ago, no there isn't much else to say about this job | 17:15 |
| *** trown is now known as trown|lunch | 17:15 | |
| mwhahaha | panda: i don't want to use quickstart, or an undercloud because it's uncessary and makes the job too long. hence being native zuulv3 | 17:15 |
| mwhahaha | panda: it's not perfect but it's a first shot at having a job for this functionality | 17:16 |
| *** bogdando has quit IRC | 17:16 | |
| mwhahaha | panda: so unless you have something major that has to be addressed right now, i would like for constructive reviews so we can make progress on this | 17:16 |
| *** mjturek has quit IRC | 17:18 | |
| *** florianf is now known as florianf|afk | 17:18 | |
| *** tongl has joined #tripleo | 17:19 | |
| arxcruz | honza: checking | 17:27 |
| *** jpich has quit IRC | 17:28 | |
| chem | dciabrin: hey, so I spend some more time on https://review.openstack.org/#/c/609998/6 | 17:30 |
| *** yprokule has quit IRC | 17:30 | |
| chem | dciabrin: (bandini is certainly hicking in the mountain, cannot find him) | 17:30 |
| chem | dciabrin: that was a pain | 17:30 |
| chem | dciabrin: I end up with https://review.openstack.org/#/c/616249/ | 17:31 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common stable/rocky: Switch stable scenarios to voting https://review.openstack.org/616258 | 17:34 |
| *** rh-jelabarre has quit IRC | 17:35 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common stable/queens: Switch stable scenarios to voting https://review.openstack.org/616259 | 17:36 |
| *** eck`gone is now known as eck` | 17:37 | |
| honza | arxcruz: thanks | 17:38 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common stable/pike: Switch stable scenarios to voting https://review.openstack.org/616260 | 17:38 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common stable/rocky: Switch stable scenarios to voting https://review.openstack.org/616258 | 17:39 |
| weshay | jaosorior, two +2's | 17:39 |
| weshay | https://review.openstack.org/#/c/615730/ | 17:39 |
| weshay | I'm getting there :) | 17:39 |
| weshay | too many mtgs | 17:40 |
| mwhahaha | did we ever figure out why that started to break on rocky all of the sudden? | 17:40 |
| weshay | ah.. wrong change.. ha | 17:40 |
| weshay | nvrmind | 17:40 |
| weshay | mwhahaha, the hostname | 17:40 |
| weshay | ? | 17:40 |
| mwhahaha | yea that thing | 17:40 |
| weshay | rfolco|ruck, was looking into it | 17:40 |
| weshay | rfolco|ruck, any conclusions? | 17:40 |
| weshay | dang it | 17:41 |
| mwhahaha | i would have thought it would ahve also broken on master | 17:41 |
| * weshay looks for chem's patches | 17:41 | |
| weshay | sorry chem I'm getting there | 17:41 |
| weshay | got em | 17:41 |
| jaosorior | weshay: it had 2 +2s before I changed the commit message | 17:42 |
| weshay | jaosorior, ah ok.. sorry I thought it was tripleo-upgrade anyhooooo | 17:42 |
| rfolco|ruck | this weshay ? https://review.openstack.org/#/c/615730/ | 17:42 |
| weshay | https://review.openstack.org/#/c/604706/ https://review.openstack.org/#/c/604736/ | 17:43 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common stable/queens: Switch stable scenarios to voting https://review.openstack.org/616259 | 17:43 |
| weshay | rfolco|ruck, ya.. and the bug | 17:43 |
| *** derekh has quit IRC | 17:44 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common stable/pike: Switch stable scenarios to voting https://review.openstack.org/616260 | 17:44 |
| rfolco|ruck | weshay, no clues... let me try logstash to see when it started actually | 17:45 |
| mwhahaha | under cloud upgrades are broken on stable/queens | 17:46 |
| mwhahaha | don't suppose we have a bug for that | 17:46 |
| weshay | mr ruck? | 17:46 |
| *** noama has quit IRC | 17:46 | |
| *** jpena is now known as jpena|off | 17:46 | |
| mwhahaha | see https://review.openstack.org/#/c/615911/ | 17:46 |
| weshay | chem, you can do the honors.. https://review.openstack.org/#/c/604706 | 17:48 |
| weshay | chem++ | 17:48 |
| *** dtrainor has joined #tripleo | 17:48 | |
| rfolco|ruck | weshay, looking at undercloud upgrades stable/queens | 17:49 |
| mwhahaha | is the /etc/heat/policy.json thing | 17:50 |
| mwhahaha | http://logs.openstack.org/11/615911/2/check/tripleo-ci-centos-7-undercloud-upgrades/d69684b/logs/undercloud/home/zuul/undercloud_upgrade.log.txt.gz#_2018-11-07_15_56_05 | 17:50 |
| mwhahaha | i think i saw a bug for that | 17:50 |
| rfolco|ruck | I got https://bugs.launchpad.net/tripleo/+bug/1796639 which points to https://bugs.launchpad.net/tripleo/+bug/1798553 | 17:51 |
| openstack | Launchpad bug 1796639 in tripleo "[stable/queens] The tripleo-ci-centos-7-undercloud-upgrades is failing on queens for No such file or directory: '/etc/heat/policy.json" [High,Triaged] - Assigned to Marios Andreou (marios-b) | 17:51 |
| rfolco|ruck | mwhahaha, ^ | 17:51 |
| openstack | Launchpad bug 1798553 in tripleo queens "o->p->q undercloud upgrade fails with "Cannot use v2 authentication with domain scope"" [Medium,In progress] - Assigned to Sofer Athlan-Guyot (sofer-athlan-guyot) | 17:51 |
| * mwhahaha sighs | 17:53 | |
| *** jcoufal has quit IRC | 18:01 | |
| rfolco|ruck | mwhahaha, I don't see a cherry-pick to queens in any of the fixes there | 18:02 |
| mwhahaha | it seems to be a packaging issue or something | 18:03 |
| mwhahaha | i guess i'll take a look at it this after noon | 18:03 |
| *** salmankhan has quit IRC | 18:04 | |
| jtomasek | honza, florianf|afk: what's the error? | 18:08 |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801769 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802083 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802085 | 18:10 |
| openstack | Launchpad bug 1801769 in tripleo "FailedValidation: Hostname is not fully qualified, fails undercloud install" [Critical,In progress] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 18:10 |
| openstack | Launchpad bug 1802083 in tripleo "Undercloud install fails at Unknown function: 'normalize_ip_for_uri'" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 18:10 |
| openstack | Launchpad bug 1802085 in tripleo "containerized UC upgrade job failing on container_cli discrepancy" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 18:10 |
| *** panda is now known as panda|off | 18:10 | |
| honza | jtomasek: florianf|afk https://paste.fedoraproject.org/paste/hvfFz2CRb6iQ4BSfV4X9Xg | 18:10 |
| *** cylopez has joined #tripleo | 18:11 | |
| jtomasek | honza, florianf|afk: it says it already runs something on 0.0.0.0:3000 | 18:12 |
| jtomasek | that's why it's failing to star | 18:12 |
| jtomasek | t | 18:12 |
| honza | jtomasek: LOL | 18:12 |
| honza | jtomasek: yes, that's it for me | 18:13 |
| jtomasek | honza: :) | 18:13 |
| honza | jtomasek: i saw a big error and assumed it was the same thing that florianf|afk had | 18:13 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-quickstart-extras master: Create a reproducer-quickstart dir https://review.openstack.org/615635 | 18:14 |
| *** agopi|lunch is now known as agopi | 18:15 | |
| *** trown|lunch is now known as trown | 18:20 | |
| mwhahaha | weshay, jaosorior, EmilienM: so when we switched the scenarios to non-voting, it switched the branch version to non-voting as well. should we switch the stable branches to voting or remove them from the gate? see https://review.openstack.org/#/q/topic:bug/1802144+(status:open+OR+status:merged) | 18:21 |
| weshay | mwhahaha, since those run less often.. my thought would be to make them vote | 18:22 |
| EmilienM | mwhahaha: yeah I asked myself too | 18:22 |
| mwhahaha | thats what i thought as well | 18:22 |
| mwhahaha | and for master+ we'll switch to standalone | 18:22 |
| *** suuuper has quit IRC | 18:22 | |
| mwhahaha | which will be new jobs that shouldn't be on branches | 18:22 |
| mwhahaha | so i'll work on getting the other repos as well | 18:22 |
| *** suuuper has joined #tripleo | 18:22 | |
| EmilienM | to make them vote, I suspect we have to override the job definition in each project/branch | 18:22 |
| mwhahaha | EmilienM: yea that's what those patches are | 18:23 |
| mwhahaha | that's for tripleo-common | 18:23 |
| mwhahaha | i'll get the ones for tht/tripleoclient/etc later today | 18:23 |
| mwhahaha | err i need to fixup those patches | 18:28 |
| mwhahaha | 8/9 aren't voting in the gate | 18:28 |
| * mwhahaha addresses | 18:28 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common stable/rocky: Switch stable scenarios to voting https://review.openstack.org/616258 | 18:28 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common stable/queens: Switch stable scenarios to voting https://review.openstack.org/616259 | 18:29 |
| *** radeks_ has quit IRC | 18:29 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common stable/pike: Switch stable scenarios to voting https://review.openstack.org/616260 | 18:29 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates stable/rocky: Switch stable scenarios to voting https://review.openstack.org/616267 | 18:30 |
| *** suuuper has quit IRC | 18:30 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates stable/queens: Switch stable scenarios to voting https://review.openstack.org/616268 | 18:32 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates stable/pike: Switch stable scenarios to voting https://review.openstack.org/616269 | 18:33 |
| *** haleyb has joined #tripleo | 18:36 | |
| openstackgerrit | Alex Schultz proposed openstack/puppet-tripleo stable/rocky: Switch stable scenarios to voting https://review.openstack.org/616270 | 18:37 |
| *** jtomasek has quit IRC | 18:40 | |
| *** mjturek has joined #tripleo | 18:40 | |
| fultonj | marios: i have a WIP patch based on our conversation yesterday https://review.openstack.org/#/c/615988/ and I see my new scenario worked (though its just a copy for now). Do you have a few minutes to talk about the next step? | 18:41 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Set undercloud hostname from ansible variables https://review.openstack.org/615730 | 18:42 |
| jaosorior | has anybody seen an issue like this http://paste.openstack.org/show/734365/ ? | 18:44 |
| *** rh-jelabarre has joined #tripleo | 18:52 | |
| *** rh-jelabarre has quit IRC | 18:52 | |
| *** rh-jelabarre has joined #tripleo | 18:52 | |
| *** cfontain has quit IRC | 18:52 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-docs master: Add split control plane example to standlone document https://review.openstack.org/614636 | 18:56 |
| *** gfidente is now known as gfidente|afk | 18:58 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-docs master: Add split control plane example to standlone document https://review.openstack.org/614636 | 18:59 |
| *** amoralej is now known as amoralej|off | 19:00 | |
| *** cylopez has quit IRC | 19:06 | |
| *** dsneddon has joined #tripleo | 19:09 | |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802083 | 19:10 |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1802085 | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1802083 in tripleo "Undercloud install fails at Unknown function: 'normalize_ip_for_uri'" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 19:10 |
| openstack | Launchpad bug 1802085 in tripleo "containerized UC upgrade job failing on container_cli discrepancy" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 19:10 |
| openstackgerrit | Merged openstack/tripleo-common stable/queens: Add UEFI support for hardened images https://review.openstack.org/614453 | 19:11 |
| openstackgerrit | Merged openstack/tripleo-image-elements stable/queens: Add overcloud-secure-uefi element https://review.openstack.org/614455 | 19:11 |
| EmilienM | weshay: why did you approve https://review.openstack.org/#/c/604736/ ? | 19:12 |
| EmilienM | I thought the gate was hossed | 19:12 |
| EmilienM | or broken | 19:13 |
| EmilienM | same for chem with https://review.openstack.org/#/c/614729/ | 19:13 |
| EmilienM | is it fixed now? | 19:13 |
| weshay | dam.. ur right.. I was excited about the upgrade job???? | 19:13 |
| fultonj | if it's fixed can we reply to http://lists.openstack.org/pipermail/openstack-dev/2018-November/136360.html ? | 19:13 |
| fultonj | i still see the sited bugs on the alert though | 19:14 |
| hkominos_ | Hi all. Can i bother somebody a couple of minutes for some debugging ? | 19:14 |
| hkominos_ | fultonj: perhaps ? | 19:14 |
| fultonj | hkominos_: sure | 19:14 |
| EmilienM | https://review.openstack.org/#/c/616022/ is in gate | 19:14 |
| EmilienM | we should not approve stuff until it lands | 19:15 |
| hkominos_ | fultonj: Hi!. So after your tips yesterday I had some time today to look into my mistral failures. So the issue at at hand is an error like this (The WebSocket timed out before the Workflow completed) | 19:15 |
| hkominos_ | Looking however at the task lists and the workflow lists the tasks seem to complete just fine | 19:16 |
| *** morazi has quit IRC | 19:16 | |
| hkominos_ | fultonj: https://paste.fedoraproject.org/paste/9eZdNH9ueG0nHwJ3Um6gZQ Initially I thought it was the swift issue that I mentioned yesterday but it seems that the urls are generated just fine | 19:16 |
| hkominos_ | So for the ID that fails the tasks seem to have been completed fine https://paste.fedoraproject.org/paste/DQU9t1B51W3X0OEifLFVZg | 19:17 |
| hkominos_ | SO I am assuming it is just a timing issue. Probably because aarch64 is so damn slow ?! | 19:17 |
| *** rh-jelabarre has quit IRC | 19:18 | |
| fultonj | hkominos_: are you using a trust anchor? | 19:18 |
| hkominos_ | I have no idea what that is | 19:19 |
| fultonj | i've seen something like this before | 19:19 |
| *** radeks_ has joined #tripleo | 19:20 | |
| fultonj | hkominos_: https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/ssl.html#certificate-details | 19:20 |
| EmilienM | i've reduced the # of alerts to 1 | 19:20 |
| fultonj | hkominos_: to answer that question in depth (what a trust anchor is) | 19:20 |
| EmilienM | https://bugs.launchpad.net/tripleo/+bug/1802083 : fix is merged (revert) | 19:20 |
| openstack | Launchpad bug 1802083 in tripleo "Undercloud install fails at Unknown function: 'normalize_ip_for_uri'" [Critical,Triaged] - Assigned to Sagi (Sergey) Shnaidman (sshnaidm) | 19:20 |
| fultonj | hkominos_: in short, a yaml file containing certs to have an SSL conversation | 19:20 |
| EmilienM | https://bugs.launchpad.net/tripleo/+bug/1802085 : no need to alert, it's a non voting job | 19:21 |
| openstack | Launchpad bug 1802085 in tripleo "containerized UC upgrade job failing on container_cli discrepancy" [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 19:21 |
| EmilienM | so we still have one alert: https://bugs.launchpad.net/tripleo/+bug/1801969 (fix is in gate) | 19:21 |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 19:21 |
| EmilienM | so yeah don't recheck or approve anything yet | 19:21 |
| weshay | EmilienM, fwiw.. the tripleo gate is at 1hr34min | 19:22 |
| weshay | no failing jobs atm, will check the patches associated w/ the alerts | 19:22 |
| EmilienM | weshay: already done | 19:22 |
| weshay | of course | 19:22 |
| *** jcoufal has joined #tripleo | 19:22 | |
| weshay | moving at the speed of Evilien | 19:22 |
| EmilienM | that's what | 19:23 |
| weshay | sad apple | 19:23 |
| EmilienM | no nothing | 19:23 |
| fultonj | hkominos_: i have an idea for an easy workaround if this is what the root cause is | 19:23 |
| hkominos_ | fultonj: I will look into this. What makes you think that this is an ssl issue ? Is there anything in the logs ? cause I see that tasks are completed properly | 19:23 |
| hkominos_ | fultonj: | 19:23 |
| hkominos_ | fultonj: Go ahead. maybe it will work | 19:23 |
| fultonj | and then they fail trying to ack to the UC that they're done | 19:23 |
| fultonj | cause they fail with an ssl handshake error? | 19:24 |
| weshay | EmilienM, so looks clear to me now | 19:24 |
| *** cdearborn has quit IRC | 19:24 | |
| weshay | oh you did | 19:24 |
| weshay | lolz | 19:24 |
| fultonj | hkominos_: https://bugzilla.redhat.com/show_bug.cgi?id=1565861#c6 | 19:25 |
| openstack | bugzilla.redhat.com bug 1565861 in ceph-ansible "enable_ssh_admin task unable to signal undercloud due to SSL handshake error" [Low,Closed: notabug] - Assigned to johfulto | 19:25 |
| weshay | it's like as I think of things.. EmilienM does them | 19:25 |
| EmilienM | weshay: 616022 will land only if the 2 patches on top won't fail | 19:25 |
| weshay | does that mean I control EmilienM | 19:25 |
| EmilienM | you wish, uh | 19:25 |
| weshay | that would be some funny shit | 19:26 |
| fultonj | hkominos_: workaround idea: add "generate_service_certificate=false" to undercloud.conf re-run undercloud install to update setting, redeploy oc | 19:26 |
| fultonj | hkominos_: if that's the issue | 19:26 |
| hkominos_ | fultonj: Ok I will give it a go. Btw those logs in your bugs I think are from overcloud nodes!? | 19:26 |
| fultonj | yes | 19:26 |
| hkominos_ | My deploy tasks seems to time out very quickly | 19:26 |
| hkominos_ | The overcloud nodes are not up | 19:27 |
| fultonj | i understand | 19:27 |
| hkominos_ | a ok | 19:27 |
| *** dprince has quit IRC | 19:27 | |
| fultonj | hkominos_: i want to rule out communication between oc and uc failing due to cert issues | 19:27 |
| fultonj | hkominos_: if we don't solve this problem now, you'll just have to solve it later i'm afraid | 19:27 |
| hkominos_ | fultonj: I will give it a go. This will take a while | 19:27 |
| fultonj | the uc update? | 19:28 |
| hkominos_ | yes | 19:28 |
| hkominos_ | unless there is some easy command that I am missing | 19:28 |
| hkominos_ | usually I redeploy the UC with apex from the ground up | 19:28 |
| hkominos_ | so the VM gets destroyed etc.. | 19:28 |
| hkominos_ | Dont tell me that there is quicker way | 19:29 |
| *** rh-jelabarre has joined #tripleo | 19:29 | |
| fultonj | hkominos_: who not just update undercloud.conf and rerun undercloud install command | 19:29 |
| fultonj | s/who/why | 19:29 |
| hkominos_ | good point | 19:29 |
| hkominos_ | Probably cause I never knew htat there was such a command.. | 19:30 |
| hkominos_ | I deploy with tha opnfv-apex installer | 19:30 |
| hkominos_ | so I usually use the commands from that project | 19:30 |
| hkominos_ | fultonj: I will let you know how it goes soon. | 19:31 |
| fultonj | https://docs.openstack.org/tripleo-docs/latest/install/installation/installation.html#updating-undercloud-components | 19:31 |
| fultonj | hkominos_: ^ | 19:31 |
| fultonj | sec | 19:32 |
| fultonj | openstack undercloud install | 19:32 |
| fultonj | that's what i meant | 19:32 |
| fultonj | i deeplinked you to upgrade, not install (which idempotently applies updates) last I tried | 19:33 |
| fultonj | not sure if containerization will break it TBH | 19:33 |
| fultonj | hkominos_: another observation, you're failing on validations portion of the workflow, you can disable validations | 19:33 |
| fultonj | https://docs.openstack.org/tripleo-docs/latest/install/validations/validations.html | 19:34 |
| fultonj | hkominos_: my bad, i don't think it's validations | 19:37 |
| hkominos_ | fultonj: So lets just wait for the generate_service_certificate=false to take effect | 19:37 |
| hkominos_ | See if the certificates is the issue | 19:37 |
| fultonj | Task 'complete_swift' (fbfe2dfb-9e0a-4abd-943d-7740bddb6736) [RUNNING -> SUCCESS | 19:37 |
| fultonj | hkominos_: another idea, reboot undercloud before redeploying | 19:38 |
| fultonj | i had a strange timeout issue like this in the past too and rebooting was workaround | 19:39 |
| * fultonj looks for bug | 19:39 | |
| fultonj | hkominos_: https://review.openstack.org/#/c/615866/ | 19:39 |
| hkominos_ | fultonj: tried many many many times | 19:39 |
| fultonj | w/ reboots? | 19:39 |
| dciabrin | chem, oh that's awesome! | 19:39 |
| fultonj | you have ^ patches | 19:40 |
| fultonj | https://bugs.launchpad.net/tripleo/+bug/1789680 | 19:40 |
| openstack | Launchpad bug 1789680 in tripleo "mistral MessagingTimeout correlates with containerized undercloud uptime" [Critical,Fix released] - Assigned to Dougal Matthews (d0ugal) | 19:40 |
| hkominos_ | fultonj: yes. I should mention that selinux is disabled atm (same issue with selinux enabled though) | 19:40 |
| hkominos_ | Not that I think it matters but just saying. | 19:40 |
| fultonj | hkominos_: you could check you have all patches related to bug ^ | 19:41 |
| *** betherly has joined #tripleo | 19:42 | |
| hkominos_ | fultonj: another question. IF i wanted to apply the patch that you sent me for example, would I apply it on the host filesystem or inside the mistral container ? | 19:42 |
| hkominos_ | fultonj: I will check now. btw there is no ERROR or timeout or anything in my api.log | 19:43 |
| fultonj | hkominos_: i could be jumping to the wrong conclusion | 19:43 |
| hkominos_ | only a warning like this | 19:43 |
| hkominos_ | Unique keys not in sort_keys. The sorting order may be unstable | 19:43 |
| fultonj | https://review.openstack.org/#/q/topic:bug/1789680+(status:open+OR+status:merged) | 19:44 |
| fultonj | hkominos_: depends on the patch | 19:44 |
| fultonj | https://review.openstack.org/#/c/615866/1/tripleoclient/workflows/deployment.py | 19:46 |
| *** betherly has quit IRC | 19:47 | |
| *** radeks_ has quit IRC | 19:51 | |
| hkominos_ | fultonj: ok the certificate issue is ruled out I think. generate_service_certificate = false was already in my undercloud.conf | 19:52 |
| *** adobrodu has joined #tripleo | 19:52 | |
| fultonj | hkominos_: ok, that's good to have ruled out | 19:52 |
| fultonj | seems like a timeout with sending messages, right? | 19:53 |
| fultonj | potentially replated to bug/1789680 ? | 19:53 |
| fultonj | you probably have a new enough deployment to have most of the fixes except the python client | 19:54 |
| *** radeks_ has joined #tripleo | 19:54 | |
| openstackgerrit | Ronelle Landy proposed openstack-infra/tripleo-ci master: Make zuul inventory available for the reproducer https://review.openstack.org/616284 | 19:54 |
| hkominos_ | fultonj: That was my original assumtion. Looking at https://paste.fedoraproject.org/paste/DQU9t1B51W3X0OEifLFVZg, I see that the tasks are completed as before. However overall those tasks take 15 minutes. And some tasks take 4 mns to complete | 19:55 |
| hkominos_ | I dont know if there is any individual timeout | 19:55 |
| adobrodu | Hello! Does anyone tried to use containerized SriovAgent in Rocky? Any success? for me it seems to lack permission to write to udev rules and sysconfig/allocate_vfs. | 19:56 |
| hkominos_ | fultonj: I will try to applyu some patches manually about timeouts and retry. | 19:57 |
| fultonj | hkominos_: ok, i'm not sure what else to do | 19:57 |
| hkominos_ | fultonj: Lets not panic yet:P what about this ?https://review.openstack.org/#/c/609993/2/tripleoclient/workflows/deployment.py | 19:58 |
| hkominos_ | is this on the host ? | 19:58 |
| fultonj | the undercloud | 19:58 |
| hkominos_ | yes. I mean is this on the undercloud's filesystem or within a container ? | 19:59 |
| fultonj | undercloud filesystem | 20:00 |
| fultonj | /usr/lib/python2.7/site-packages/tripleoclient/workflows/deployment.py | 20:01 |
| hkominos_ | yeah I got it. | 20:05 |
| hkominos_ | I am still waiting for the UC to finish installation | 20:05 |
| hkominos_ | as I said aarch64 is not exaclty quick | 20:06 |
| hkominos_ | *exactly | 20:06 |
| *** morazi has joined #tripleo | 20:08 | |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1801969 | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1801969 in tripleo "container build failing on grafana when trying to build" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 20:10 |
| openstackgerrit | John Fulton proposed openstack/tripleo-quickstart master: WIP/DNM: Add featureset 57 for split controlplane https://review.openstack.org/615987 | 20:11 |
| *** radeks_ has quit IRC | 20:20 | |
| *** rfolco|ruck is now known as rfolco|off | 20:40 | |
| *** thrash has quit IRC | 20:47 | |
| *** rajinir has quit IRC | 20:47 | |
| *** thrash has joined #tripleo | 20:47 | |
| *** kopecmartin|off has quit IRC | 20:47 | |
| *** mnaser has quit IRC | 20:48 | |
| *** mwhahaha has quit IRC | 20:48 | |
| *** rajinir has joined #tripleo | 20:48 | |
| *** mwhahaha has joined #tripleo | 20:49 | |
| *** ChanServ sets mode: +v mwhahaha | 20:49 | |
| *** mnaser has joined #tripleo | 20:49 | |
| *** kopecmartin has joined #tripleo | 20:49 | |
| *** andreaf has quit IRC | 20:51 | |
| *** andreaf has joined #tripleo | 20:52 | |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Add standalone upgrade role and playbook. https://review.openstack.org/604736 | 20:57 |
| openstackgerrit | Merged openstack/tripleo-upgrade master: Set pipefail during overcloud upgrade tasks. https://review.openstack.org/614729 | 20:57 |
| openstackgerrit | Merged openstack/tripleo-common master: Add image_source to containers https://review.openstack.org/616022 | 20:57 |
| *** raildo has quit IRC | 21:02 | |
| *** rh-jelabarre has quit IRC | 21:10 | |
| *** gfidente|afk has quit IRC | 21:11 | |
| *** dsneddon has quit IRC | 21:15 | |
| EmilienM | ok now we can recheck | 21:18 |
| *** EmilienM changes topic to "CI Status: Green (for how long?) | https://docs.openstack.org/tripleo-docs/latest/" | 21:18 | |
| EmilienM | weshay: ^ | 21:18 |
| fultonj | thanks EmilienM | 21:20 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Allow the environment to enable Podman on the overcloud https://review.openstack.org/612522 | 21:21 |
| weshay | EmilienM, so tru.. for how long | 21:22 |
| *** rh-jelabarre has joined #tripleo | 21:28 | |
| *** jcoufal has quit IRC | 21:32 | |
| *** salmankhan has joined #tripleo | 21:34 | |
| *** dsneddon has joined #tripleo | 21:35 | |
| *** ansmith has quit IRC | 21:37 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add support for configuring ppc64le in ironic https://review.openstack.org/580829 | 21:42 |
| openstackgerrit | Merged openstack/tripleo-common master: Move some container deps out of base https://review.openstack.org/614371 | 22:00 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Add openstacklib to requirements in metadata.json https://review.openstack.org/616188 | 22:00 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-heat-templates master: Move UndercloudPostDeployment to python https://review.openstack.org/614540 | 22:02 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-heat-templates master: Create clouds.yaml for the undercloud https://review.openstack.org/615853 | 22:03 |
| *** trown is now known as trown|outtypewww | 22:06 | |
| *** bfournie has quit IRC | 22:08 | |
| openstackgerrit | Tony Breeds proposed openstack/tripleo-heat-templates stable/rocky: Add support for configuring ppc64le in ironic https://review.openstack.org/616362 | 22:18 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Introduce zuul.projects and executed pre-run playbooks for reproducer https://review.openstack.org/613572 | 22:31 |
| *** rfolco|off has quit IRC | 22:34 | |
| *** ssbarnea has quit IRC | 22:37 | |
| *** rh-jelabarre has quit IRC | 22:54 | |
| *** dmacpher has quit IRC | 22:55 | |
| *** saneax has joined #tripleo | 22:57 | |
| *** slaweq has quit IRC | 23:01 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Change atop log location https://review.openstack.org/616374 | 23:25 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Run atop for monitoring deployment https://review.openstack.org/615357 | 23:27 |
| *** mcornea has quit IRC | 23:28 | |
| *** rlandy is now known as rlandy|bbl | 23:29 | |
| *** Vorrtex has quit IRC | 23:34 | |
| *** dhellmann has quit IRC | 23:45 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!