openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Cleaning up tempest skip tests for pike https://review.openstack.org/587844 | 00:08 |
---|---|---|
openstackgerrit | Merged openstack/tripleo-heat-templates master: Expose dnsmasq_local_resolv option https://review.openstack.org/577936 | 00:08 |
openstackgerrit | Merged openstack/tripleo-heat-templates master: Disable recursion in Designate-managed BIND https://review.openstack.org/577937 | 00:08 |
openstackgerrit | Merged openstack/python-tripleoclient master: New function: run_ansible_playbook https://review.openstack.org/586538 | 00:08 |
*** hamzy has joined #tripleo | 00:09 | |
*** ooolpbot has joined #tripleo | 00:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 00:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 00:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 00:10 |
*** ooolpbot has quit IRC | 00:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 00:10 |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 00:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 00:10 |
*** tonyb has quit IRC | 00:16 | |
*** edmondsw has joined #tripleo | 00:37 | |
*** edmondsw has quit IRC | 00:42 | |
*** jcoufal has joined #tripleo | 00:49 | |
*** mrsoul has joined #tripleo | 01:06 | |
*** ooolpbot has joined #tripleo | 01:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 01:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 01:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 01:10 |
*** ooolpbot has quit IRC | 01:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 01:10 |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 01:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 01:10 |
*** sai_p has quit IRC | 01:19 | |
*** sai_p has joined #tripleo | 01:24 | |
*** gbarros has joined #tripleo | 01:47 | |
*** dtrainor has quit IRC | 01:54 | |
*** gbarros has quit IRC | 01:57 | |
*** ramishra has joined #tripleo | 02:00 | |
*** jcoufal has quit IRC | 02:05 | |
*** ramishra has quit IRC | 02:08 | |
*** ooolpbot has joined #tripleo | 02:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 02:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 02:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 02:10 |
*** ooolpbot has quit IRC | 02:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 02:10 |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 02:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 02:10 |
*** tonyb has joined #tripleo | 02:22 | |
*** edmondsw has joined #tripleo | 02:25 | |
*** edmondsw has quit IRC | 02:30 | |
*** psachin has joined #tripleo | 02:35 | |
openstackgerrit | Merged openstack/python-tripleoclient master: Drop incorrect reference to heat_launch https://review.openstack.org/587857 | 02:44 |
openstackgerrit | Steve Baker proposed openstack/ansible-role-tripleo-modify-image master: Add Buildah support https://review.openstack.org/588390 | 03:07 |
*** dxiri has joined #tripleo | 03:08 | |
*** ooolpbot has joined #tripleo | 03:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 03:10 |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 03:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 03:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 03:10 |
*** ooolpbot has quit IRC | 03:10 | |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 03:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 03:10 |
*** rlandy|bbl is now known as rlandy | 03:14 | |
*** skramaja has joined #tripleo | 03:22 | |
*** errantekarmico has quit IRC | 03:26 | |
*** udesale has joined #tripleo | 03:31 | |
*** itlinux_ has joined #tripleo | 03:31 | |
openstackgerrit | wangqi proposed openstack/puppet-tripleo master: Set connect_timeout in mysql https://review.openstack.org/588433 | 03:39 |
*** dxiri has quit IRC | 03:41 | |
openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart-extras master: Use local docker registry host for tempest container https://review.openstack.org/584368 | 03:47 |
*** ratailor has joined #tripleo | 03:59 | |
*** ramishra has joined #tripleo | 04:03 | |
*** ooolpbot has joined #tripleo | 04:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 04:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 04:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 04:10 |
*** ooolpbot has quit IRC | 04:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 04:10 |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 04:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 04:10 |
*** dxiri has joined #tripleo | 04:21 | |
openstackgerrit | Steve Baker proposed openstack/tripleo-docs master: Document new push_destination behaviour https://review.openstack.org/588439 | 04:25 |
*** rlandy has quit IRC | 04:28 | |
*** thrash is now known as thrash|g0ne | 04:30 | |
*** udesale has quit IRC | 04:35 | |
*** holser_ has joined #tripleo | 04:47 | |
*** ykarel has joined #tripleo | 04:52 | |
jaosorior | bnemec: bummer, when you were available I was gone already (the different timezone are a bit hard) | 04:54 |
*** udesale has joined #tripleo | 04:55 | |
jaosorior | bandini: around? | 05:06 |
bandini | jaosorior: not for too long ;) | 05:06 |
jaosorior | bandini: could you check these out ? https://review.openstack.org/#/c/588151/ https://review.openstack.org/#/c/588136/ | 05:07 |
*** ooolpbot has joined #tripleo | 05:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 05:10 |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 05:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 05:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 05:10 |
*** ooolpbot has quit IRC | 05:10 | |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 05:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 05:10 |
jaosorior | bandini: thanks | 05:12 |
bandini | np | 05:12 |
*** pmannidi has joined #tripleo | 05:13 | |
*** dxiri has quit IRC | 05:14 | |
Tengu | hello there :) | 05:15 |
*** shyambiradar has joined #tripleo | 05:17 | |
jaosorior | Tengu: good days ir | 05:23 |
jaosorior | * day sir | 05:23 |
shyambiradar | jaosorior: Hi | 05:27 |
shyambiradar | If I use nova-compute container as base image, will it have nova-compute service started? | 05:28 |
jaosorior | shyambiradar: it won't | 05:28 |
jaosorior | shyambiradar: the containers only run the services that you configure them to run | 05:28 |
jaosorior | did you need the compute service to be running on the same container? | 05:29 |
shyambiradar | no | 05:29 |
shyambiradar | I don't want nova-compute service | 05:29 |
jhebden | hey all, quick question hopefully - doing a deployment, and getting the following - | 05:29 |
jhebden | Exception creating plan: 404 Client Error: Not Found for url: http://192.168.24.1:8080/v1/AUTH_f3e94f0102fb49a6b343a24bf9b50934/overcloud/docker_config_scripts/nova_statedir_ownership.py | 05:30 |
jaosorior | shyambiradar: so then your alright :) | 05:30 |
jhebden | I've searched my director host for nova_statedir_ownership.py, it's nowhere to be found - this is on a fresh RHOSP13 | 05:30 |
shyambiradar | jaosorior: I am implementing a nova-compute plugin | 05:30 |
shyambiradar | so I want all libraries | 05:30 |
shyambiradar | packages and other artifacts | 05:30 |
jhebden | if anyone has any ideas, I'd appreciate it | 05:30 |
jaosorior | then yes, making your container have nova-compute as the base sounds like the way to go | 05:31 |
jaosorior | jhebden: can you pass more logs? | 05:31 |
shyambiradar | great | 05:31 |
jhebden | jaosorior: absolutely | 05:31 |
jhebden | jaosorior: http://paste.openstack.org/show/727204/ | 05:32 |
jhebden | jaosorior: I've tried with and without docker/docker-ha and I get the same result | 05:33 |
shyambiradar | jaosorior: Can you please give me nova compute container location in redhat registry? | 05:33 |
shyambiradar | If you know | 05:34 |
jaosorior | shyambiradar: not sure where that is unfortunately | 05:34 |
shyambiradar | ok | 05:34 |
shyambiradar | I will search | 05:34 |
shyambiradar | thanks | 05:34 |
jaosorior | jhebden: is this a new deployment or were you updating the plan? | 05:34 |
jhebden | jaosorior: this was a new deployment, I had unsuccessfully deployed the overcloud previously | 05:35 |
jaosorior | jhebden: you could try deleting the plan | 05:35 |
jhebden | jaosorior: will try that now | 05:35 |
jhebden | hmm, no plan is showing j | 05:36 |
jhebden | * jaosorior | 05:36 |
jhebden | with openstack overcloud plan list | 05:36 |
jhebden | it seems odd to me that I can't find that file on the director host | 05:37 |
openstackgerrit | Cédric Jeanneret proposed openstack/python-tripleoclient master: New preflight validation: disk space on undercloud https://review.openstack.org/586541 | 05:37 |
jhebden | it seems that it should be uploaded to swift from the host either during deploy or undercloud install? | 05:37 |
jaosorior | jhebden: the plan is updated on deploy | 05:38 |
Tengu | oopss... wrong commig -.-'. | 05:39 |
*** quiquell has joined #tripleo | 05:39 | |
openstackgerrit | Cédric Jeanneret proposed openstack/python-tripleoclient master: New preflight check for the undercloud: disk space https://review.openstack.org/586541 | 05:40 |
*** jaosorior has quit IRC | 05:40 | |
*** shyambiradar has quit IRC | 05:41 | |
*** jaosorior has joined #tripleo | 05:41 | |
*** janki has joined #tripleo | 05:44 | |
openstackgerrit | Cédric Jeanneret proposed openstack/python-tripleoclient master: New preflight check for the undercloud: disk space https://review.openstack.org/586541 | 05:46 |
*** holser_ has quit IRC | 05:48 | |
*** jrist has quit IRC | 05:49 | |
*** shyambiradar has joined #tripleo | 05:50 | |
jhebden | so it seems like my issue is that https://review.openstack.org/#/c/587066/ hasn't landed yet | 05:54 |
jhebden | however the heat templates refer to scripts which are included in that change | 05:54 |
jaosorior | jhebden: oh damn :/ what is referring to that change then? | 05:56 |
jaosorior | lets try to get that merged asap | 05:56 |
jhebden | jaosorior: it looks like the nova-* templates are expecting this script | 05:56 |
jhebden | for openstack-tripleo-heat-templates | 05:56 |
*** apetrich has joined #tripleo | 05:57 | |
jhebden | jaosorior: docker/services/nova-compute.yaml, docker/services/nova-compute-common.yaml and docker/services/nova-ironic.yaml all seem to refer to the docker_config_scripts | 05:59 |
jhebden | jaosorior: this is openstack-tripleo-heat-templates-8.0.4-6.el7ost.src.rpm | 06:00 |
jhebden | also, thank you for you help jaosorior :) | 06:00 |
jaosorior | jhebden: well... that's interesting, nova_statedir_ownership.py is available in the t-h-t repo | 06:01 |
jhebden | I noticed that, maybe is is just missing from the RPM? | 06:02 |
quiquell | ykarel: I have progress at openshift 3.10, but is failing now at installing origin-node-3.9.0 | 06:02 |
quiquell | ykarel: http://logs.openstack.org/52/588152/3/check/tripleo-ci-centos-7-scenario009-multinode-oooq/90c035c/logs/undercloud/var/lib/mistral/overcloud/openshift/playbook.log.txt.gz | 06:02 |
jaosorior | jhebden: What version are you trying to install? | 06:02 |
jhebden | jaosorior: openstack-tripleo-heat-templates-8.0.4-6.el7ost.src.rpm | 06:02 |
jhebden | on RHOSP13 | 06:02 |
jaosorior | jhebden: OK, maybe the package hasn't been updated | 06:02 |
jaosorior | here's the patch that landed for queens https://review.openstack.org/#/c/585762/ | 06:03 |
ykarel | quiquell, checking | 06:03 |
quiquell | ykarel: I think we have to set openshift_pkg_version 3.10 or the like | 06:04 |
quiquell | ykarel: TASK [openshift_version : debug] *********************************************** | 06:05 |
quiquell | Thursday 02 August 2018 13:26:58 +0000 (0:00:00.114) 0:01:32.929 ******* | 06:05 |
quiquell | ok: [centos-7-rax-dfw-0001112025] => { | 06:05 |
quiquell | "msg": "openshift_pkg_version was not defined. Falling back to -3.9.0" | 06:05 |
quiquell | } | 06:05 |
ykarel | quiquell, hmm possibly | 06:06 |
quiquell | ykarel: Going to try | 06:06 |
ykarel | quiquell, should be 3.10.18 | 06:07 |
jhebden | jaosorior: helpful if I raise a bug for this? | 06:08 |
*** ooolpbot has joined #tripleo | 06:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 06:10 |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 06:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 06:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 06:10 |
*** ooolpbot has quit IRC | 06:10 | |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 06:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 06:10 |
quiquell | ykarel: Where is the place where we download openshift-ansible from git instead of package system ? | 06:14 |
quiquell | ykarel: You have a patch to revert it | 06:14 |
ykarel | quiquell, revert patch https://review.openstack.org/#/c/588198/ | 06:16 |
ykarel | quiquell, but looks like until and unless openshift repo is enabled, package install would fail | 06:16 |
quiquell | ykarel: 3.10.18 is the openshift-ansible version and 3.10.0 is the openshift version | 06:17 |
quiquell | ykarel: I am correct ? | 06:17 |
quiquell | ykarel: Learning in the process | 06:17 |
ykarel | quiquell, /me also no idea, looking into repo | 06:18 |
*** yprokule has joined #tripleo | 06:18 | |
*** jfrancoa has joined #tripleo | 06:18 | |
quiquell | ykarel: If we find more problems after upgrading to 3.10, then we can revert, to unlock it | 06:18 |
openstackgerrit | Quique Llorente proposed openstack/tripleo-heat-templates master: Replace openshift_node_labels with openshift_node_group_name https://review.openstack.org/588152 | 06:19 |
quiquell | ykarel: ^ | 06:21 |
ykarel | quiquell, so as i checked repo for openshift-3.10 is not avaiable:- http://mirror.centos.org/centos/7/extras/x86_64/Packages/, so may be we need to enable openshift ansible 3.9 repo, use openshift-ansible 3.10 and use openshift_pkg_version=3.9.0 | 06:22 |
quiquell | ykarel: https://github.com/openshift/openshift-ansible/issues/8399 | 06:24 |
quiquell | ykarel: It's from May and they don't have the 3.10 RPMs yet | 06:24 |
ykarel | quiquell, hmm | 06:24 |
quiquell | ykarel: Maybe we have to include this https://github.com/openshift/openshift-ansible/blob/master/images/installer/origin-extra-root/etc/yum.repos.d/centos-openshift-origin.repo | 06:25 |
quiquell | ykarel: Also in the github they say that RH has deliver 3.10 | 06:25 |
ykarel | quiquell, but that repo do not have 3.10 | 06:25 |
ykarel | so we have to stick to 3.9 | 06:25 |
ykarel | until it's available | 06:25 |
quiquell | ykarel: Yep just checked it... this is the one we have to install to get 3.9 and go back to 3.9 and 3.10 for ansible as you said | 06:26 |
shyambiradar | Hi | 06:26 |
shyambiradar | I used kolla nova compute container image as base image for my container | 06:26 |
shyambiradar | Once I launch my container I am getting following error | 06:27 |
shyambiradar | ()[42436@rhosp13-shyam-dev /]$ id nova | 06:27 |
shyambiradar | uid=162(nova) gid=162(nova) groups=162(nova),99(nobody),993(libvirt),107(qemu) | 06:27 |
shyambiradar | id: cannot find name for user ID 42436 | 06:27 |
shyambiradar | ---------------------------------- | 06:27 |
shyambiradar | Any thoughts? | 06:27 |
jaosorior | jhebden: well, I would wait for the next updates; but if you want you can do so, so it gets some following | 06:27 |
quiquell | ykarel: Lat's try, uncomment flaper87 to get centos-release-openshift-origin39 | 06:35 |
quiquell | last | 06:35 |
ykarel | quiquell, ack | 06:35 |
openstackgerrit | Quique Llorente proposed openstack/tripleo-heat-templates master: Replace openshift_node_labels with openshift_node_group_name https://review.openstack.org/588152 | 06:38 |
openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: Activate centos-release-openshift-origin39-1-1.el7.centos https://review.openstack.org/588448 | 06:39 |
quiquell | ykarel: 2~^ | 06:39 |
quiquell | ykarel: Maybe the Depends-On direction is the opposite ? | 06:40 |
quiquell | I don't see scenario009 running | 06:40 |
ykarel | quiquell, yup please reverse | 06:40 |
quiquell | ykarel: ack | 06:40 |
ykarel | quiquell, can run in experimental though | 06:41 |
ykarel | in tq i see scenario009 is in experimental pipeline | 06:41 |
quiquell | ykarel: But it's in check in other ones... | 06:42 |
quiquell | ykarel: Maybe that part is missing | 06:42 |
ykarel | possibly | 06:42 |
quiquell | ykarel: Going to add the Depends-On at TQ and put scenario009 at check | 06:42 |
ykarel | quiquell, ack | 06:42 |
quiquell | ykarel: If there are changes at featureset033 | 06:42 |
openstackgerrit | Quique Llorente proposed openstack/tripleo-heat-templates master: Replace openshift_node_labels with openshift_node_group_name https://review.openstack.org/588152 | 06:44 |
quiquell | ykarel: Nah, let's kee tq as it is, trown was already moving this thing to check | 06:45 |
quiquell | ykarel: At the end we are just testing if upgrade to openshift-ansible 3.10 is possible, is not let's unblock with your patch | 06:45 |
*** ccamacho has joined #tripleo | 06:45 | |
quiquell | ykarel: I just want to know if we were breaking it with our new CI workflow | 06:45 |
ykarel | quiquell, ack | 06:46 |
openstackgerrit | Quique Llorente proposed openstack/tripleo-quickstart master: Activate centos-release-openshift-origin39-1-1.el7.centos https://review.openstack.org/588449 | 06:47 |
openstackgerrit | Quique Llorente proposed openstack/tripleo-heat-templates master: Replace openshift_node_labels with openshift_node_group_name https://review.openstack.org/588152 | 06:47 |
openstackgerrit | Quique Llorente proposed openstack/tripleo-heat-templates master: Replace openshift_node_labels with openshift_node_group_name https://review.openstack.org/588152 | 06:48 |
*** rcernin has quit IRC | 06:54 | |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Replace TAGS with ansible var https://review.openstack.org/584508 | 06:54 |
jaosorior | bandini: could you also check these out https://review.openstack.org/#/c/588245/ https://review.openstack.org/#/c/588246/ ? | 06:54 |
*** shyambiradar has quit IRC | 06:54 | |
openstackgerrit | Carlos Camacho proposed openstack/tripleo-upgrade stable/pike: Add the overcloud stack name when calling upgrade-non-controller.sh https://review.openstack.org/588450 | 06:56 |
*** shyambiradar has joined #tripleo | 06:58 | |
ykarel | shyambiradar, looks like u are mounting /etc in ur container | 06:59 |
shyambiradar | yes | 06:59 |
ykarel | if yes try mounting only the required /etc subfolders | 06:59 |
shyambiradar | I am mounting | 06:59 |
shyambiradar | I want /etc/nova | 07:00 |
shyambiradar | ok | 07:00 |
shyambiradar | thanks | 07:00 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Move featureset BASH variables to ansible https://review.openstack.org/587340 | 07:00 |
openstackgerrit | Carlos Camacho proposed openstack/tripleo-quickstart-extras master: Use the TripleO Ansible to run upgrades from tripleo-upgrade https://review.openstack.org/588451 | 07:01 |
shyambiradar | ykarel: working now | 07:02 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Move featureset BASH variables to ansible https://review.openstack.org/587340 | 07:02 |
shyambiradar | Thanks a lot ykarel | 07:02 |
ykarel | shyambiradar, cool | 07:03 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Move featureset BASH variables to ansible https://review.openstack.org/587340 | 07:04 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Cleanup variables and toci_jobtype leftovers https://review.openstack.org/584465 | 07:06 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Replace TAGS with ansible var https://review.openstack.org/584508 | 07:06 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Move featureset BASH variables to ansible https://review.openstack.org/587340 | 07:06 |
openstackgerrit | Carlos Camacho proposed openstack/tripleo-quickstart-extras master: Use the TripleO Ansible role from tripleo-upgrade to run upgrades https://review.openstack.org/588451 | 07:06 |
*** amoralej|off is now known as amoralej | 07:09 | |
*** sai_p has quit IRC | 07:09 | |
*** ooolpbot has joined #tripleo | 07:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 07:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 07:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 07:10 |
*** ooolpbot has quit IRC | 07:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 07:10 |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 07:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 07:10 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Move nodes_file and nodes_args to common.yaml https://review.openstack.org/585714 | 07:10 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Move nodes_file and nodes_args to common.yaml https://review.openstack.org/585714 | 07:11 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Refactor DRY_RUN, STATS_* and COLLECT_CONF vars https://review.openstack.org/587344 | 07:16 |
*** pmannidi has quit IRC | 07:16 | |
openstackgerrit | Carlos Camacho proposed openstack/tripleo-quickstart-extras master: Use the TripleO Ansible role from tripleo-upgrade to run upgrades https://review.openstack.org/588451 | 07:18 |
openstackgerrit | Carlos Camacho proposed openstack/tripleo-quickstart-extras master: Use the TripleO Ansible role from tripleo-upgrade to run upgrades https://review.openstack.org/588451 | 07:22 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Replace toci ENV_VARS by ansible env_vars https://review.openstack.org/587360 | 07:22 |
*** tosky has joined #tripleo | 07:26 | |
quiquell | jfrancoa, ccamacho: we can +1w this https://review.openstack.org/#/c/577146/ | 07:26 |
jfrancoa | quiquell: my first +1w, what a wonderful moment :-D | 07:27 |
quiquell | jfrancoa: Hehe congrats :-) | 07:28 |
quiquell | jfrancoa: Sure it breaks everything, I am the driver of your kaos | 07:28 |
openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart-extras master: Cleaning up tempest skip tests for ocata https://review.openstack.org/588458 | 07:28 |
jfrancoa | quiquell: thanks, as they were saying in Counter Strike: "The bomb is planted" | 07:28 |
*** jtomasek_ has quit IRC | 07:29 | |
quiquell | jfrancoa: Same change in TQ was working find, this will work too | 07:30 |
jfrancoa | quiquell: yes, I saw that the other one was already merged and so far nothing got broken | 07:30 |
*** jtomasek has joined #tripleo | 07:31 | |
Tengu | jfrancoa: hey congrats :). | 07:33 |
jfrancoa | Tengu: thanks man | 07:34 |
* Tengu opens some bottle of fresh beverage :) | 07:34 | |
jfrancoa | Tengu: it's friday + my birthday, so let's open some rum instead :-D | 07:34 |
quiquell | jfrancoa: Double congrats !!! | 07:34 |
Tengu | jfrancoa: ah, sorry, have only Whiskies in that kind of beverage :) | 07:34 |
*** shyambiradar has quit IRC | 07:44 | |
*** mschuppert has joined #tripleo | 07:49 | |
*** jpich has joined #tripleo | 07:53 | |
openstackgerrit | RedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/560445 | 08:00 |
openstackgerrit | Cédric Jeanneret proposed openstack/python-tripleoclient master: Add some logging for the preflight checks https://review.openstack.org/586285 | 08:00 |
*** florianf has joined #tripleo | 08:01 | |
*** dtantsur|afk is now known as dtantsur | 08:01 | |
*** derekh has joined #tripleo | 08:03 | |
*** holser_ has joined #tripleo | 08:08 | |
*** ooolpbot has joined #tripleo | 08:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 08:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 08:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 08:10 |
*** ooolpbot has quit IRC | 08:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 08:10 |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 08:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 08:10 |
*** lebauce has quit IRC | 08:11 | |
*** lebauce has joined #tripleo | 08:12 | |
*** saneax has joined #tripleo | 08:12 | |
*** jpena|off is now known as jpena | 08:20 | |
*** leseb_ has quit IRC | 08:25 | |
quiquell | ykarel: This could be related to the openshift change ? | 08:26 |
quiquell | ykarel: http://logs.openstack.org/52/588152/8/check/tripleo-ci-centos-7-scenario009-multinode-oooq/eca355d/logs/undercloud/home/zuul/overcloud_prep_containers.log.txt.gz | 08:26 |
Tengu | florianf: heya! care to give a (hopefully) last review on https://review.openstack.org/#/c/583475/ ? I added a work regarding the default "running engine" for the validation that should meet the results of our discussions on that topic :). | 08:28 |
openstackgerrit | Tom Barron proposed openstack/tripleo-heat-templates stable/queens: Fix manila update, upgrade, and FFU tasks https://review.openstack.org/587982 | 08:29 |
Tengu | florianf: if you're OK with that, you might even give the final w+1 (or "call a friend" for review if you feel something still needs some wording/work) :) | 08:29 |
ykarel | quiquell, don't think so, should be unrelated | 08:29 |
openstackgerrit | Quique Llorente proposed openstack/tripleo-heat-templates master: Replace openshift_node_labels with openshift_node_group_name https://review.openstack.org/588152 | 08:30 |
*** shyambiradar has joined #tripleo | 08:38 | |
*** leanderthal has joined #tripleo | 08:40 | |
florianf | Tengu: Yup, I'll have a look! | 08:41 |
Tengu | florianf: thank you :) | 08:41 |
Tengu | hey guys, PTAL? https://review.openstack.org/586541 that would be really nice of you :). | 08:45 |
*** salmankhan has joined #tripleo | 08:48 | |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add a script to compare playbook executions https://review.openstack.org/588475 | 08:50 |
quiquell | ssbarnea: ^ | 08:50 |
*** leanderthal has quit IRC | 08:51 | |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add a script to compare playbook executions https://review.openstack.org/588475 | 08:55 |
*** ykarel has quit IRC | 08:57 | |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Cleanup variables and toci_jobtype leftovers https://review.openstack.org/584465 | 08:59 |
ssbarnea | testing the script now | 09:00 |
*** jaosorior has quit IRC | 09:03 | |
quiquell | ssbarnea: ack | 09:03 |
quiquell | ssbarnea: leftovers is ok , but tags review is not | 09:03 |
quiquell | ssbarnea: Adding some diff mechanism to it | 09:04 |
*** leanderthal has joined #tripleo | 09:05 | |
openstackgerrit | Yurii Prokulevych proposed openstack/tripleo-upgrade stable/pike: Don't invoke multiple migration requests for single vm. https://review.openstack.org/588479 | 09:05 |
*** ooolpbot has joined #tripleo | 09:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 09:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 09:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 09:10 |
*** ooolpbot has quit IRC | 09:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 09:10 |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 09:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 09:10 |
*** gfidente has joined #tripleo | 09:13 | |
shyambiradar | Hi, I want to mount /etc/nova from host to container | 09:16 |
shyambiradar | But I am not able to access it after mount | 09:16 |
shyambiradar | because of permissions | 09:16 |
shyambiradar | How can I handle this scenario? | 09:17 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add a script to compare playbook executions https://review.openstack.org/588475 | 09:17 |
openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: Add a script to compare playbook executions https://review.openstack.org/588475 | 09:18 |
shyambiradar | Any thought? | 09:18 |
ssbarnea | quiquell: i made a small improvement on the diff script output to make it easier to read. | 09:19 |
quiquell | ssbarnea: Cool, upload the change | 09:20 |
ssbarnea | alredy did | 09:20 |
ssbarnea | i was ready to add colors but decided to avoid over engineering from its first version | 09:21 |
quiquell | ssbarnea: Let's find the issues with tags patch | 09:22 |
quiquell | ssbarnea: Going to remove the list from the tripleo card | 09:22 |
quiquell | ssbarnea: Removed the string diff ? | 09:25 |
ssbarnea | not on putpose, my change was on previous changeset, going to refresh it | 09:27 |
quiquell | ssbarnea: ok | 09:27 |
quiquell | ssbarnea: humm didn't ask you to merge or something ? | 09:28 |
quiquell | That's weird | 09:28 |
quiquell | ssbarnea: I mean git review | 09:28 |
ssbarnea | indeed, i was wondering the same thing,... nothing | 09:28 |
quiquell | ssbarnea: ok np, just re-merge your stuff so we also get the differ thing | 09:28 |
quiquell | ssbarnea: It's broken with your changes, it's comparing only the keys not the conent | 09:30 |
quiquell | content | 09:30 |
quiquell | ssbarnea: going to revert them | 09:30 |
ssbarnea | ahh... | 09:30 |
ssbarnea | anyway, your pprint use probably fixed the problem, | 09:30 |
ssbarnea | i think i know what happend, there is only one minute between my change and yours, we probably found a bug in gerrit or git-review. | 09:32 |
quiquell | ssbarnea: Ahh yep have to be, ok no problem let's just use the differ | 09:32 |
quiquell | ssbarnea: Going to update with it, | 09:32 |
*** zoli is now known as zoli|lunch | 09:32 | |
quiquell | ssbarnea: thanks | 09:33 |
quiquell | updated | 09:33 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add a script to compare playbook executions https://review.openstack.org/588475 | 09:33 |
*** shyambiradar has quit IRC | 09:35 | |
quiquell | ssbarnea: Do you want a bj session to explain it to you ? | 09:35 |
*** shyambiradar has joined #tripleo | 09:36 | |
ssbarnea | first let me look at a diff, give me two numbers to check if you can | 09:36 |
openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: Te-broker: save all nodes console to html log dir https://review.openstack.org/588488 | 09:37 |
openstackgerrit | Carlos Camacho proposed openstack/tripleo-common stable/queens: Include the stack name when generating the tripleo-ansible-inventory https://review.openstack.org/588489 | 09:38 |
shyambiradar | Hi | 09:38 |
shyambiradar | openstack nova compute container is creating 'nova' user with a different it than 162 | 09:39 |
shyambiradar | Any idea? | 09:39 |
shyambiradar | I was expecting e created with 162 | 09:40 |
openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: Workflow for provisioning bare metals without Nova https://review.openstack.org/576856 | 09:40 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add a script to compare playbook executions https://review.openstack.org/588475 | 09:45 |
*** ykarel has joined #tripleo | 09:45 | |
*** shyambiradar has quit IRC | 09:46 | |
ykarel | quiquell, just in case u have not seen openshift 3.10 in testing repo: https://lists.centos.org/pipermail/centos-devel/2018-August/016857.html | 09:47 |
*** ykarel is now known as ykarel|travel | 09:48 | |
quiquell | ykarel|travel: Suppose we don't want to use testing repo | 09:49 |
openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: Workflow for provisioning bare metals without Nova https://review.openstack.org/576856 | 09:49 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add a script to compare playbook executions https://review.openstack.org/588475 | 09:50 |
*** ykarel|travel has quit IRC | 09:52 | |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add a script to compare playbook executions https://review.openstack.org/588475 | 09:52 |
openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: Switch away from using token_endpoint keystoneauth plugin https://review.openstack.org/585765 | 09:56 |
openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: [WIP] Collect required information for deploying without Nova https://review.openstack.org/586279 | 09:56 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Replace TAGS with ansible var https://review.openstack.org/584508 | 09:59 |
*** agopi has quit IRC | 10:08 | |
*** ooolpbot has joined #tripleo | 10:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 10:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 10:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 10:10 |
*** ooolpbot has quit IRC | 10:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 10:10 |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 10:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 10:10 |
panda|rover | I need some nova expert | 10:10 |
panda|rover | !summon nova-expert | 10:10 |
openstack | panda|rover: Error: "summon" is not a valid command. | 10:10 |
openstackgerrit | Carlos Camacho proposed openstack/tripleo-common stable/pike: Include the stack name when generating the tripleo-ansible-inventory https://review.openstack.org/588506 | 10:34 |
*** zoli|lunch is now known as zoli | 10:50 | |
openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: Add a script to compare playbook executions https://review.openstack.org/588475 | 10:52 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Move nodes_file and nodes_args to common.yaml https://review.openstack.org/585714 | 10:52 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Replace toci ENV_VARS by ansible env_vars https://review.openstack.org/587360 | 10:52 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Refactor DRY_RUN, STATS_* and COLLECT_CONF vars https://review.openstack.org/587344 | 10:53 |
*** shyambiradar has joined #tripleo | 10:55 | |
openstackgerrit | Jose Luis Franco proposed openstack/python-tripleoclient master: Handle negative/wrong answer in question when upgrading undercloud. https://review.openstack.org/586249 | 11:00 |
openstackgerrit | Carlos Camacho proposed openstack/tripleo-quickstart-extras master: Use the TripleO Ansible role from tripleo-upgrade to run upgrades https://review.openstack.org/588451 | 11:00 |
*** dtrainor has joined #tripleo | 11:02 | |
*** jpena is now known as jpena|lunch | 11:03 | |
*** ooolpbot has joined #tripleo | 11:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 11:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 11:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 11:10 |
*** ooolpbot has quit IRC | 11:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 11:10 |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 11:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 11:10 |
openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart-extras master: Collect logs: get the node console logs https://review.openstack.org/588516 | 11:10 |
openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: Workflow for provisioning bare metals without Nova https://review.openstack.org/576856 | 11:11 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Replace toci ENV_VARS by ansible env_vars https://review.openstack.org/587360 | 11:13 |
*** shyambiradar has quit IRC | 11:15 | |
*** shyambiradar has joined #tripleo | 11:15 | |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Move nodes_file and nodes_args to common.yaml https://review.openstack.org/585714 | 11:15 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Move featureset BASH variables to ansible https://review.openstack.org/587340 | 11:19 |
*** rh-jelabarre has joined #tripleo | 11:26 | |
Tengu | heya! on the RDO CI, I have an issue apparently unrelated to my patch: http://logs.rdoproject.org/85/586285/4/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/b4f057b/job-output.txt.gz - specifically: Could not find a version that satisfies the requirement container-check (from versions: )\nNo matching distribution found for container-check | 11:27 |
Tengu | is it a known issue? | 11:27 |
openstackgerrit | Merged openstack/python-tripleoclient master: Remove usage of resource registry for undercloud TLS https://review.openstack.org/588151 | 11:28 |
openstackgerrit | Merged openstack/tripleo-upgrade stable/pike: Add the overcloud stack name when calling upgrade-non-controller.sh https://review.openstack.org/588450 | 11:28 |
openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Fix centos opstools repo deactivation https://review.openstack.org/577146 | 11:29 |
*** lebauce has quit IRC | 11:30 | |
therve | EmilienM: Do we have specific containers for the undercloud or are they the same as the overcloud btw? | 11:30 |
openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: [DNM][TEST] dummy change for OVB tests iteration https://review.openstack.org/588517 | 11:31 |
*** shyambiradar has quit IRC | 11:34 | |
*** shyambiradar has joined #tripleo | 11:35 | |
openstackgerrit | Derek Higgins proposed openstack/tripleo-heat-templates master: [WIP] Add scenario 012 - overlcoud baremetal+ansible-ml2 https://review.openstack.org/579603 | 11:38 |
openstackgerrit | Derek Higgins proposed openstack/tripleo-quickstart master: [WIP] Add featureset 054 - overcloud baremetal+ansible-ml2 https://review.openstack.org/579601 | 11:40 |
openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci master: [WIP] Testing a ansible-ml2 job https://review.openstack.org/582294 | 11:42 |
*** ansmith has quit IRC | 11:45 | |
openstackgerrit | Carlos Camacho proposed openstack/tripleo-quickstart-extras master: Use the TripleO Ansible role from tripleo-upgrade to run upgrades https://review.openstack.org/588451 | 11:45 |
EmilienM | therve: same | 11:47 |
therve | EmilienM: OK. So it's not possible here | 11:47 |
therve | EmilienM: What are our options? | 11:47 |
*** thrash|g0ne is now known as thrash | 11:50 | |
*** abishop has joined #tripleo | 11:52 | |
*** karthiks has joined #tripleo | 11:53 | |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: [WIP] Run emit_releases_file.py for all featuresets https://review.openstack.org/585649 | 11:57 |
*** jpena|lunch is now known as jpena | 11:57 | |
*** zul has joined #tripleo | 12:01 | |
chandankumar | marios: Hey | 12:01 |
chandankumar | mandre: please have a look at this patch https://review.openstack.org/#/c/584368/12 | 12:02 |
openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: Add a script to compare playbook executions https://review.openstack.org/588475 | 12:02 |
chandankumar | marios: ^^ | 12:02 |
*** ratailor has quit IRC | 12:08 | |
*** shyambiradar has quit IRC | 12:10 | |
*** ooolpbot has joined #tripleo | 12:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 12:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 12:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 12:10 |
*** ooolpbot has quit IRC | 12:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 12:10 |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 12:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 12:10 |
mschuppert | hi, do we have any update on the ipmi issue reported in https://bugs.launchpad.net/tripleo/+bug/1784822 ? | 12:11 |
EmilienM | therve: so it's needed only in the heat containers on the undercloud right? | 12:16 |
therve | EmilienM: yes | 12:16 |
EmilienM | therve: is that terrible to install these plugins on the overcloud too? | 12:16 |
therve | EmilienM: Yes | 12:16 |
*** panda|rover is now known as panda|rover|off | 12:17 | |
openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart-extras master: Enable support for running refstack tests in TQE https://review.openstack.org/570719 | 12:17 |
therve | It changes basic behavior for server for example | 12:17 |
EmilienM | mhh | 12:17 |
EmilienM | in that case I see no other option than creating specific heat containers, that we'll use for the undercloud | 12:17 |
EmilienM | but;.. beurk | 12:17 |
openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: Add a script to compare playbook executions https://review.openstack.org/588475 | 12:18 |
therve | EmilienM: We can do a mount otherwise | 12:19 |
*** rbrady-afk is now known as rbrady | 12:19 | |
therve | Ie share /usr/lib/heat with the heat-engine container | 12:20 |
EmilienM | yeah we could do that indeed | 12:20 |
EmilienM | therve: with a conditional | 12:20 |
EmilienM | mount if HeatUndercloud is true | 12:20 |
EmilienM | false by default and true in environment/undercloud.yaml | 12:20 |
EmilienM | for ex. | 12:20 |
therve | Yeah | 12:21 |
therve | EmilienM: So just add a volume conditionally? | 12:23 |
EmilienM | therve: yeah | 12:25 |
openstackgerrit | Marios Andreou proposed openstack/tripleo-quickstart master: DNM: testing https://review.openstack.org/588527 | 12:25 |
EmilienM | I think it's easier and it should work | 12:25 |
therve | OK I think I see how to do that | 12:25 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add logic/tests for n->n+1 upgrades https://review.openstack.org/572420 | 12:25 |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add fs050 job to test from-release-upgrades https://review.openstack.org/574665 | 12:25 |
openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart-extras master: Use local docker registry host for tempest container https://review.openstack.org/584368 | 12:25 |
openstackgerrit | Thomas Herve proposed openstack/tripleo-heat-templates master: Mount /usr/lib/heat on undercloud Heat https://review.openstack.org/588529 | 12:27 |
EmilienM | therve: even better :P | 12:28 |
EmilienM | ++ I think it's good | 12:29 |
therve | It's a bit cheating but easy at least :) | 12:30 |
*** toure|gone is now known as toure | 12:35 | |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add a script to compare playbook executions https://review.openstack.org/588475 | 12:39 |
*** rlandy has joined #tripleo | 12:41 | |
*** skramaja has quit IRC | 12:48 | |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add a script to compare playbook executions https://review.openstack.org/588475 | 12:51 |
openstackgerrit | Marios Andreou proposed openstack-infra/tripleo-ci master: Remove tripleo.sh --bootstrap-subnodes add ceph loop device https://review.openstack.org/583195 | 12:52 |
*** rfolco|off is now known as rfolco|ruck | 12:52 | |
*** eck`gone is now known as eck` | 12:54 | |
*** saneax has quit IRC | 12:55 | |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Add a script to compare playbook executions https://review.openstack.org/588475 | 12:58 |
Tengu | yprokule: heya! could you give a try to jaosorior patch regarding your upgrade issue with the "enable-tls.yaml" ? | 12:59 |
*** edmondsw_ has joined #tripleo | 13:00 | |
yprokule | Tengu: this one worked for me https://review.openstack.org/#/c/588151/ | 13:01 |
Tengu | yprokule: great! sooo, not sure we can close the issue already, or wait for a package to be produced with the fix? | 13:01 |
yprokule | Tengu: yes, we need that in package | 13:02 |
Tengu | yprokule: need to import it downstream right? will try to see to that. | 13:04 |
yprokule | Tengu: jschlueter might help with that | 13:04 |
Tengu | yprokule: I think I have all what I need :). I'll call for help if needed, but it should be ok. | 13:06 |
*** leanderthal has quit IRC | 13:06 | |
openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: Replace TAGS with ansible var https://review.openstack.org/584508 | 13:07 |
*** amoralej is now known as amoralej|lunch | 13:07 | |
openstackgerrit | Alex Schultz proposed openstack/puppet-pacemaker master: Initial support for adding cluster nodes to an existing cluster https://review.openstack.org/585150 | 13:09 |
*** ooolpbot has joined #tripleo | 13:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 13:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 13:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 13:10 |
*** ooolpbot has quit IRC | 13:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 13:10 |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 13:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 13:10 |
*** agopi has joined #tripleo | 13:10 | |
*** agopi_ has joined #tripleo | 13:11 | |
*** gbarros has joined #tripleo | 13:12 | |
*** ansmith has joined #tripleo | 13:12 | |
*** quiquell is now known as quiquell|off | 13:13 | |
*** tzumainn has joined #tripleo | 13:13 | |
*** agopi has quit IRC | 13:14 | |
*** agopi_ is now known as agopi | 13:14 | |
*** quiquell|off has quit IRC | 13:17 | |
openstackgerrit | Raoul Scarazzini proposed openstack/tripleo-quickstart-extras master: Integrate Validate-HA work into OVB jobs https://review.openstack.org/537669 | 13:19 |
jschlueter | yprokule, Tengu: what's up? | 13:20 |
Tengu | jschlueter: nothing, thanks! was about to check backporting an upstream patch downstream, but after some discussions it has to wait - plus, I shouldn't be the one doing that anyway :). | 13:20 |
Tengu | jschlueter: but I have the doc for that kind of task, so I shouldn't get any issue when time comes for me to actually *do* a downstream import :]. | 13:21 |
mwhahaha | s/import/cherry-pick | 13:21 |
Tengu | well, yes. sorry | 13:21 |
openstackgerrit | Damien Ciabrini proposed openstack/tripleo-heat-templates master: Revert shebang change for InstanceHA startup script https://review.openstack.org/588552 | 13:21 |
*** ramishra has quit IRC | 13:22 | |
* jschlueter nods | 13:22 | |
mwhahaha | weshay: i just saw a patch with full ovb pass from 2am, is it fixed? or was the patch just lucky | 13:22 |
weshay | mwhahaha, lucky | 13:23 |
mwhahaha | booo | 13:23 |
Tengu | :] | 13:23 |
EmilienM | has anyone testing the https://github.com/openstack/ansible-role-tripleo-modify-image repo outside of CI? | 13:23 |
mwhahaha | weshay: did we poke the rdo cloud admins to see if they did any upgrades recently? | 13:23 |
Tengu | guess it was my patch | 13:23 |
EmilienM | it doesn't work for me, https://github.com/openstack/ansible-role-tripleo-modify-image | 13:24 |
EmilienM | https://bugs.launchpad.net/tripleo/+bug/1785243 | 13:24 |
openstack | Launchpad bug 1785243 in tripleo "modify-image role: variable error when getting "user" from container config" [Medium,Triaged] - Assigned to Emilien Macchi (emilienm) | 13:24 |
weshay | mwhahaha, working multiple fronts now atm | 13:24 |
mwhahaha | k | 13:24 |
*** jcoufal has joined #tripleo | 13:24 | |
Tengu | mwhahaha: you refer to that kind of job? http://logs.rdoproject.org/85/586285/4/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/b4f057b/job-output.txt.gz - apparently the issue is related to some pip package. | 13:25 |
*** agopi_ has joined #tripleo | 13:25 | |
mwhahaha | Tengu: no there's an issue with pxe boot | 13:25 |
Tengu | ah, so that's another one :] | 13:26 |
mwhahaha | though that's an interesting failure | 13:26 |
mwhahaha | looks like scenario001 in queens might be broken, seeing some repeated TIMED_OUTs | 13:27 |
Tengu | mwhahaha: a recheck is running on the RDO, maybe it was just temporary. | 13:27 |
*** agopi has quit IRC | 13:28 | |
*** agopi_ is now known as agopi | 13:31 | |
*** psachin has quit IRC | 13:33 | |
openstackgerrit | Merged openstack/tripleo-heat-templates master: Update and upgrade tasks for services deployed via external deploy tasks https://review.openstack.org/579170 | 13:35 |
openstackgerrit | Merged openstack/python-tripleoclient stable/pike: [Pike only] use --plan instead of --stack for tripleo-ansible-inventory https://review.openstack.org/586184 | 13:35 |
*** ansmith has quit IRC | 13:40 | |
*** gbarros has quit IRC | 13:42 | |
*** ansmith has joined #tripleo | 13:44 | |
weshay | mwhahaha, we've moved from 70% fail rate to 43% :/ | 13:47 |
weshay | otp w/ ironic folks atm | 13:47 |
mwhahaha | progress? :D | 13:47 |
weshay | pinche boots | 13:47 |
*** jrist has joined #tripleo | 13:48 | |
jrist | weird | 13:49 |
*** bnemec is now known as beekneemech | 13:51 | |
*** gbarros has joined #tripleo | 13:53 | |
openstackgerrit | Jon Schlueter proposed openstack/tripleo-common stable/queens: kolla overrides: Remove yum cache https://review.openstack.org/588579 | 14:00 |
*** saneax has joined #tripleo | 14:04 | |
*** edmondsw_ is now known as edmondsw | 14:08 | |
*** ooolpbot has joined #tripleo | 14:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 14:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 14:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 14:10 |
*** ooolpbot has quit IRC | 14:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 14:10 |
*** amoralej|lunch is now known as amoralej | 14:10 | |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 14:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 14:10 |
openstackgerrit | Jose Luis Franco proposed openstack/python-tripleoclient master: Handle negative/wrong answer in question when upgrading undercloud. https://review.openstack.org/586249 | 14:13 |
gfidente | mwhahaha maybe you have a link to that patch which removed the osp/ceph packages from the overcloud baremetal image? | 14:15 |
gfidente | I am struggling to find it, I remember you were on review | 14:15 |
mwhahaha | yea i wrote it | 14:16 |
mwhahaha | sec | 14:16 |
mwhahaha | we merged it long ago | 14:16 |
mwhahaha | gfidente: https://review.openstack.org/#/c/563608/ | 14:16 |
mwhahaha | gfidente: specifically the work was done via https://review.openstack.org/#/q/status:merged+project:openstack/tripleo-puppet-elements+branch:master+topic:bp/cleanup-overcloud-images | 14:17 |
gfidente | "i wrote it" | 14:17 |
gfidente | :D | 14:17 |
gfidente | mwhahaha++ | 14:17 |
*** gbarros has quit IRC | 14:27 | |
*** artom has joined #tripleo | 14:28 | |
*** janki has quit IRC | 14:28 | |
*** gbarros has joined #tripleo | 14:28 | |
openstackgerrit | Toure Dunnon proposed openstack/tripleo-common master: Improve output when registering nodes. https://review.openstack.org/569811 | 14:32 |
openstackgerrit | Merged openstack/tripleo-heat-templates master: fix manila scheduler service typo https://review.openstack.org/588407 | 14:35 |
*** jcoufal_ has joined #tripleo | 14:36 | |
*** jcoufal has quit IRC | 14:37 | |
*** dxiri has joined #tripleo | 14:38 | |
openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: fix tox python3 overrides https://review.openstack.org/588587 | 14:43 |
openstackgerrit | Toure Dunnon proposed openstack/tripleo-common master: Clean up node registration output. https://review.openstack.org/569811 | 14:44 |
*** janki has joined #tripleo | 14:51 | |
weshay | mwhahaha, rlandy bfournie they are going to reboot a few nodes in rdo.. my stack failed.. holding reattempts for the reboots | 14:51 |
openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: fix pep8 check and made it generic https://review.openstack.org/588588 | 14:52 |
openstackgerrit | Merged openstack/puppet-pacemaker master: Initial support for adding cluster nodes to an existing cluster https://review.openstack.org/585150 | 14:53 |
openstackgerrit | Sorin Sbarnea proposed openstack-infra/tripleo-ci master: fix pep8 check and made it generic https://review.openstack.org/588588 | 14:54 |
*** rpioso|afk is now known as rpioso | 14:55 | |
*** dxiri has quit IRC | 14:58 | |
*** abays has joined #tripleo | 15:05 | |
openstackgerrit | Merged openstack/tripleo-common master: Add python-novajoin to nova-api packages https://review.openstack.org/588136 | 15:08 |
*** toure has quit IRC | 15:09 | |
*** ooolpbot has joined #tripleo | 15:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 15:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 15:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 15:10 |
*** ooolpbot has quit IRC | 15:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 15:10 |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 15:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 15:10 |
*** toure has joined #tripleo | 15:14 | |
*** jcoufal has joined #tripleo | 15:15 | |
*** jcoufal_ has quit IRC | 15:16 | |
openstackgerrit | Carlos Goncalves proposed openstack/diskimage-builder master: Add netcat to redhat-common map-packages https://review.openstack.org/588601 | 15:20 |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Enable collectd to connect to metrics QDR https://review.openstack.org/576057 | 15:21 |
abays | gfidente: Would you have a moment to help with an OSP-12 ceph-ansible deployment issue? | 15:21 |
openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Enable collectd to connect to metrics QDR https://review.openstack.org/576057 | 15:23 |
gfidente | abays let's try but mileage may vary :D | 15:23 |
abays | gfidente: Cool, thank you! So I'm working with someone and they're trying to deploy a cloud with HCI compute nodes | 15:25 |
abays | We keep hitting this error at step 2 of the deployment... | 15:25 |
abays | http://paste.openstack.org/show/727239/ | 15:25 |
gfidente | abays so is this pike, queens or master? | 15:26 |
abays | Pike | 15:26 |
abays | He has ceph-ansible installed and we can see the playbooks, etc, in /usr/share/ceph-ansible | 15:28 |
*** janki has quit IRC | 15:29 | |
Tengu | have a great weekend ppl :). | 15:29 |
abays | He had 3.0.27 installed and we upgraded to 3.0.39 hoping that would help, but it didn't | 15:29 |
*** zoli is now known as zoli|gone | 15:38 | |
*** zoli|gone is now known as zoli | 15:38 | |
openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient master: Remove instack-undercloud and --use-heat https://review.openstack.org/588606 | 15:40 |
openstackgerrit | Marios Andreou proposed openstack/tripleo-quickstart master: Install openstack-heat-agents for Pike subnodes https://review.openstack.org/588527 | 15:43 |
*** itlinux_ has quit IRC | 15:43 | |
gfidente | abays 3.0.27 should be fine | 15:46 |
abays | gfidente: We just noticed that his /etc/ansible/ansible.cfg exists but is completely empty | 15:46 |
gfidente | abays I don't think this is the cause of the failure | 15:46 |
gfidente | do you see heat failing at workflow_step2 ? | 15:46 |
abays | Yes | 15:47 |
gfidente | abays there should be in /var/log/mistral a log file for the ceph-ansible run | 15:47 |
abays | In executor.log ? | 15:47 |
gfidente | no the file is called ceph-ansible-workflow.log | 15:47 |
*** yprokule has quit IRC | 15:47 | |
gfidente | if you don't have it, then mistral didn't get to launching ceph-ansible even | 15:47 |
rlandy | bfournie: weshay: I managed to reproduce the IPMI failure once - although I probably hit the controller reboot https://paste.fedoraproject.org/paste/1YLCgAWLcG4dvW~oDLfOkg | 15:48 |
abays | gfidente: Gotcha -- it's not there, so ceph-ansible didn't run | 15:48 |
*** itlinux has joined #tripleo | 15:49 | |
gfidente | abays it's probably failing at this stage https://github.com/openstack/tripleo-common/blob/stable/pike/workbooks/ceph-ansible.yaml#L68-L87 | 15:49 |
rlandy | bfournie: weshay: I ran again - and now introspection failed on the second and third nodes | 15:49 |
rlandy | Introspection of node b9718a5f-5c3c-438a-b502-49decf93f84d timed out. | 15:49 |
rlandy | Introspection of node cf2e79cd-0366-44f7-9340-edd7793cb611 timed out. | 15:49 |
rlandy | Retrying 2 nodes that failed introspection. Attempt 1 of 3 | 15:49 |
gfidente | abays can you see if you spot anything about that task in the mistral logs? | 15:50 |
rlandy | weshay: bfournie: bmc log from timed out introspection: https://paste.fedoraproject.org/paste/JVbwqTNM9mkBj5YcGyQ30Q | 15:50 |
abays | gfidente: Taking a look now | 15:51 |
bfournie | derekh: ^ | 15:53 |
*** abays has quit IRC | 15:54 | |
openstackgerrit | Merged openstack/tripleo-heat-templates stable/pike: Clean up env files for Cinder's Netapp backend https://review.openstack.org/583650 | 15:55 |
openstackgerrit | Alex Schultz proposed openstack/tripleo-quickstart-extras master: Add pauch to the includepkgs https://review.openstack.org/579223 | 15:55 |
openstackgerrit | Emilien Macchi proposed openstack/ansible-role-tripleo-modify-image master: Add Buildah support https://review.openstack.org/588390 | 15:55 |
*** abays has joined #tripleo | 15:57 | |
*** ccamacho has quit IRC | 15:57 | |
abays | gfidente: There's really nothing in the logs concerning "collect_nodes_uuid" other than some debug about validating the expression | 15:57 |
bfournie | rlandy: thanks, can you update bug with that stack trace? | 15:57 |
bfournie | rlandy: are there no libvirt logs? | 15:58 |
rlandy | I'll look | 15:58 |
bfournie | rlandy: thanks, also anything interesting in nova api log? | 15:58 |
rlandy | bfournie: I used the same setup as before - you can access it if you wish | 15:59 |
bfournie | rlandy: k, will do | 15:59 |
rlandy | I'll gather what I can | 15:59 |
openstackgerrit | RedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/560445 | 16:00 |
openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates stable/pike: GATE CHECK for TripleO https://review.openstack.org/564285 | 16:00 |
gfidente | abays no ansible output? | 16:00 |
*** jfrancoa has quit IRC | 16:00 | |
abays | gfidente: Just one line about validating the expression | 16:01 |
abays | Meaning, I guess, that it's validating the task syntax? | 16:01 |
abays | That was a debug with no error | 16:01 |
gfidente | abays judging from you stacktrace | 16:03 |
gfidente | amend what I said above I think it's failing in an earlier step even | 16:03 |
derekh | rlandy: weshay bfournie [84315.724241] openstackbmc[2567]: ClientException: Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible. | 16:03 |
gfidente | https://github.com/openstack/tripleo-common/blob/stable/pike/workbooks/access.yaml#L145 | 16:03 |
derekh | rlandy: weshay bfournie this is a api error talking to the cloud (rdo cloud?) | 16:04 |
-openstackstatus- NOTICE: The infra team is renaming projects in Gerrit. There will be a short ~10 minute Gerrit downtime in a few minutes as a result. | 16:04 | |
derekh | looks like a problem with the stability of the API on rdo-cloud | 16:04 |
gfidente | https://github.com/openstack/tripleo-common/blob/stable/pike/workbooks/access.yaml#L140 | 16:04 |
rlandy | derekh: yeah - I think hit the rdocloud reboot on that run | 16:05 |
gfidente | abays so yeah that is using the default ansible.cfg | 16:05 |
abays | gfidente: Okay | 16:06 |
abays | So a blank file would be bad news? | 16:06 |
derekh | rlandy: ok, can we try again for logs without a reboot as the reboot will cause all kinds of problems | 16:06 |
gfidente | https://github.com/openstack/tripleo-common/blob/stable/pike/tripleo_common/actions/ansible.py#L31-L44 | 16:06 |
*** rlandy is now known as rlandy|mtg | 16:07 | |
gfidente | abays yeah looks like it expects defaults section to exist | 16:07 |
gfidente | https://github.com/openstack/tripleo-common/blob/stable/pike/tripleo_common/actions/ansible.py#L39 | 16:07 |
* abays nods at gfidente | 16:07 | |
gfidente | not sure why it's empty there but | 16:08 |
gfidente | yes I'd try [defaults] | 16:08 |
abays | gfidente: Thank you. We will get valid cfg content in there and try the deployment again | 16:09 |
gfidente | abays cool also keep 0.27 | 16:09 |
*** ccamacho has joined #tripleo | 16:09 | |
abays | gfidente: Okay | 16:09 |
*** ooolpbot has joined #tripleo | 16:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 16:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 16:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 16:10 |
*** ooolpbot has quit IRC | 16:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 16:10 |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 16:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 16:10 |
*** holser_ has quit IRC | 16:10 | |
*** melwitt is now known as jgwentworth | 16:14 | |
*** holser_ has joined #tripleo | 16:15 | |
*** gbarros has quit IRC | 16:16 | |
*** gbarros has joined #tripleo | 16:18 | |
*** udesale has quit IRC | 16:20 | |
weshay | derekh, do you have the log? | 16:20 |
derekh | weshay: which log? rlandy|mtg posted one a while back from a bmc https://paste.fedoraproject.org/paste/JVbwqTNM9mkBj5YcGyQ30Q | 16:21 |
weshay | derekh, rlandy|mtg bfournie and I have been chatting about it. I'm in an out of 1-1's atm.. probably a little out of step w/ rlandy|mtg | 16:21 |
derekh | weshay: rlandy|mtg thinks that might have been during a reboot | 16:22 |
rlandy|mtg | weshay: derekh: correct - I caught the reboot - I mentioned that above | 16:22 |
weshay | ah k | 16:22 |
weshay | funny how ha doesn't work sometimes | 16:22 |
* weshay runs | 16:22 | |
rlandy|mtg | I can get diff results each time I run :) | 16:23 |
weshay | lolz | 16:23 |
derekh | weshay: First place I would start is to find a ipmi error and see if that corresponds back to a error on the BMC | 16:23 |
weshay | k | 16:23 |
weshay | rlandy|mtg, I think panda|rover|off was adding auth to the bmc node so we could collect logs from it | 16:23 |
weshay | as derekh makes a good point | 16:24 |
derekh | weshay: if the rdo-cloud api isn't stable then the reality is there isn't much we can do except wait until it is | 16:24 |
rlandy|mtg | weshay: sorry - just info transferring to myoung atm - re: zuulv3 | 16:24 |
rlandy|mtg | will get back to this | 16:24 |
*** gbarros has quit IRC | 16:25 | |
*** boazel has joined #tripleo | 16:26 | |
*** gbarros has joined #tripleo | 16:28 | |
*** psachin has joined #tripleo | 16:30 | |
boazel | wii boazel | 16:30 |
*** jpich has quit IRC | 16:32 | |
*** holser_ has quit IRC | 16:32 | |
weshay | mwhahaha, to review what deployed-server should or should not do for pike | 16:33 |
weshay | https://bugs.launchpad.net/tripleo/+bug/1785067 | 16:33 |
openstack | Launchpad bug 1785067 in tripleo "without bootstrap-subnodes pike deployment fails for missing os-collect-config" [High,In progress] - Assigned to Marios Andreou (marios-b) | 16:33 |
weshay | https://review.openstack.org/#/c/588527/2 | 16:33 |
weshay | does tripleo-bootstrap need to be ported back to pike? https://github.com/openstack/tripleo-heat-templates/blob/stable/pike/common/deploy-steps.j2 | 16:35 |
weshay | or can we put this yum install into ci | 16:35 |
mwhahaha | weshay: what did we document for that? | 16:35 |
weshay | ya. that was my next step | 16:35 |
mwhahaha | if we documented that an end user had to manually do that then sure put it in oooq | 16:35 |
mwhahaha | if we didn't then it's a bug to be fixed elsewhere | 16:35 |
weshay | mwhahaha, was deployed server supported in pike? | 16:35 |
mwhahaha | that's a question for slage tho as i don't remember what we did for pike (he's on pto) | 16:35 |
weshay | ya | 16:36 |
mwhahaha | i believe that was the first one where we did support it | 16:36 |
mwhahaha | you could maybe check the downstream docs | 16:36 |
weshay | I am now | 16:36 |
openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates master: Skip Ceph packages when running tripleo-packages https://review.openstack.org/588618 | 16:36 |
mwhahaha | since we don't version upstream docs it's hard to tell | 16:36 |
weshay | marios, fyi ^ | 16:36 |
mwhahaha | we should really version the upstream docs | 16:36 |
openstackgerrit | Giulio Fidente proposed openstack/tripleo-common master: Remove ceph-osd and update other Ceph packages if deployment succeeded https://review.openstack.org/588619 | 16:36 |
*** boazel has quit IRC | 16:39 | |
openstackgerrit | Raoul Scarazzini proposed openstack/tripleo-quickstart-extras master: Integrate Validate-HA work into OVB jobs https://review.openstack.org/537669 | 16:40 |
weshay | mwhahaha, I think it's supported w/ https://github.com/openstack/tripleo-heat-templates/blob/stable/pike/environments/deployed-server-bootstrap-environment-centos.yaml | 16:40 |
weshay | https://github.com/openstack/tripleo-heat-templates/blob/stable/pike/deployed-server/deployed-server-bootstrap-centos.yaml | 16:41 |
mwhahaha | weshay: so that would be the fix for pike is to enable that | 16:41 |
weshay | mwhahaha, ya.. | 16:41 |
* weshay looking for deployed-server-bootstrap-centos.sh | 16:41 | |
*** jpena is now known as jpena|off | 16:44 | |
weshay | mwhahaha, marios ok.. https://review.openstack.org/#/c/588527/2 | 16:46 |
weshay | see my comments | 16:47 |
*** openstackgerrit has quit IRC | 16:49 | |
*** gfidente has quit IRC | 16:50 | |
*** agopi_ has joined #tripleo | 16:51 | |
*** agonzalez has joined #tripleo | 16:52 | |
agonzalez | Hello, with Swift is possible enable Encryption with Barbican, is it possible to do the same with Ceph rgw? | 16:52 |
*** agopi has quit IRC | 16:54 | |
*** agopi_ is now known as agopi | 16:54 | |
derekh | weshay: rlandy|mtg I gotta go soon but will try a reproduce later, is it happening on a specific job? | 16:55 |
weshay | derekh, featureset001/35 ovb | 16:56 |
rlandy|mtg | derekh: it's sporadic on various jobs - but I will send you an email with explanations when I am out or meeting | 16:56 |
derekh | rlandy|mtg: weshay thanks, will mail if I find anything out | 16:57 |
derekh | also added some more thoughts to the bug | 16:57 |
*** derekh is now known as derekh_afk | 17:00 | |
*** boazel_ has joined #tripleo | 17:06 | |
*** boazel_ has joined #tripleo | 17:07 | |
*** boazel_ has quit IRC | 17:07 | |
*** boazel_ has joined #tripleo | 17:08 | |
*** boazel_ has quit IRC | 17:08 | |
*** boazel_ has joined #tripleo | 17:09 | |
*** boazel_ has quit IRC | 17:09 | |
*** ooolpbot has joined #tripleo | 17:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 17:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 17:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 17:10 |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 17:10 |
*** ooolpbot has quit IRC | 17:10 | |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 17:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 17:10 |
*** boazel_ has joined #tripleo | 17:13 | |
*** florianf has quit IRC | 17:14 | |
*** boazel_ has joined #tripleo | 17:14 | |
*** boazel_ has left #tripleo | 17:15 | |
*** openstackgerrit has joined #tripleo | 17:16 | |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: add required deployed server config for pike https://review.openstack.org/588623 | 17:16 |
openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: Remove tripleo.sh --bootstrap-subnodes add ceph loop device https://review.openstack.org/583195 | 17:17 |
openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: Remove tripleo.sh --bootstrap-subnodes add ceph loop device https://review.openstack.org/583195 | 17:17 |
*** dtantsur is now known as dtantsur|afk | 17:20 | |
*** boazel has joined #tripleo | 17:20 | |
*** dmellado has quit IRC | 17:22 | |
*** stevebaker has quit IRC | 17:23 | |
*** gouthamr has quit IRC | 17:23 | |
*** boazel has quit IRC | 17:25 | |
*** gouthamr has joined #tripleo | 17:26 | |
*** boazel has joined #tripleo | 17:28 | |
*** psachin has quit IRC | 17:29 | |
-openstackstatus- NOTICE: Project renames and review.openstack.org downtime are complete without any major issue. | 17:30 | |
*** gouthamr has quit IRC | 17:33 | |
*** salmankhan has quit IRC | 17:46 | |
*** hgibson has joined #tripleo | 17:56 | |
*** rlandy|mtg is now known as rlandy|brb | 18:02 | |
*** amoralej is now known as amoralej|off | 18:03 | |
*** ccamacho has quit IRC | 18:04 | |
*** ccamacho has joined #tripleo | 18:05 | |
weshay | mwhahaha, derekh_afk the quarterback is toast | 18:05 |
derekh_afk | weshay: ya, my reproduce script failed to get far and now can't do nova list ;-( | 18:08 |
openstackgerrit | Merged openstack/instack-undercloud master: Set connect_timeout in mysql https://review.openstack.org/586275 | 18:08 |
* derekh_afk will try again later | 18:08 | |
*** ccamacho has quit IRC | 18:09 | |
*** ooolpbot has joined #tripleo | 18:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 18:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 18:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 18:10 |
*** ooolpbot has quit IRC | 18:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 18:10 |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 18:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 18:10 |
*** openstackgerrit has quit IRC | 18:19 | |
*** derekh_afk has quit IRC | 18:21 | |
*** gbarros has quit IRC | 18:23 | |
*** gbarros has joined #tripleo | 18:26 | |
*** gbarros has quit IRC | 18:31 | |
*** rlandy|brb is now known as rlandy | 18:34 | |
*** openstackgerrit has joined #tripleo | 18:35 | |
openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo stable/queens: Release 8.3.5 (Queens) https://review.openstack.org/588636 | 18:35 |
openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo stable/pike: Release 7.4.15 (Pike) https://review.openstack.org/588637 | 18:36 |
openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo stable/ocata: Release 6.5.15 (Ocata) https://review.openstack.org/588638 | 18:37 |
openstackgerrit | Emilien Macchi proposed openstack/ansible-role-tripleo-modify-image master: Add Buildah support https://review.openstack.org/588390 | 18:40 |
*** gbarros has joined #tripleo | 18:41 | |
openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates master: Change NodeDataLookup type from string to json https://review.openstack.org/588640 | 18:48 |
*** gbarros has quit IRC | 18:51 | |
*** jtomasek has quit IRC | 18:51 | |
*** gbarros has joined #tripleo | 18:52 | |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-common master: ansible: replace yum module by package module when possible https://review.openstack.org/584480 | 18:52 |
*** gouthamr has joined #tripleo | 19:04 | |
openstackgerrit | Merged openstack/tripleo-common stable/queens: Include the stack name when generating the tripleo-ansible-inventory https://review.openstack.org/588489 | 19:07 |
*** ooolpbot has joined #tripleo | 19:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 19:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 19:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 19:10 |
*** ooolpbot has quit IRC | 19:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 19:10 |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 19:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 19:10 |
*** gbarros has quit IRC | 19:11 | |
*** derekh has joined #tripleo | 19:13 | |
*** derekh has quit IRC | 19:15 | |
*** boazel has quit IRC | 19:18 | |
*** salmankhan has joined #tripleo | 19:19 | |
*** colonwq has quit IRC | 19:21 | |
*** salmankhan has quit IRC | 19:24 | |
*** jcoufal has quit IRC | 19:26 | |
*** jcoufal has joined #tripleo | 19:29 | |
openstackgerrit | Rafael Folco proposed openstack/tripleo-quickstart-extras master: WIP: Fix libvirt_type kvm for overcloud deploy https://review.openstack.org/587572 | 19:35 |
openstackgerrit | Rafael Folco proposed openstack-infra/tripleo-ci master: Remove dup config on updates fs037 job https://review.openstack.org/588650 | 19:49 |
*** mjturek has joined #tripleo | 19:52 | |
mwhahaha | beekneemech: you around? | 19:54 |
beekneemech | mwhahaha: For a little while longer. | 19:54 |
mwhahaha | beekneemech: wanted to see if you wanted to throw things at ovb, or maybe wait until monday | 19:54 |
beekneemech | mwhahaha: I'm actually on PTO next week, so chances are I'm not going to have time to dig into anything significant until I get back. | 19:56 |
mwhahaha | oh all rgith | 19:56 |
openstackgerrit | Merged openstack/puppet-tripleo stable/ocata: Release 6.5.15 (Ocata) https://review.openstack.org/588638 | 19:56 |
mwhahaha | i didn't get a chance to launch an undercloud yet, let me see if i can get that done quickly | 19:56 |
*** dmellado has joined #tripleo | 19:56 | |
mwhahaha | otherwise i'll poke at it next week | 19:56 |
openstackgerrit | Merged openstack/puppet-tripleo stable/pike: Release 7.4.15 (Pike) https://review.openstack.org/588637 | 19:59 |
*** gouthamr has quit IRC | 20:00 | |
openstackgerrit | Merged openstack/puppet-tripleo stable/queens: Release 8.3.5 (Queens) https://review.openstack.org/588636 | 20:01 |
*** toure is now known as toure|gone | 20:01 | |
*** rbrady has quit IRC | 20:06 | |
*** artom has quit IRC | 20:07 | |
*** ooolpbot has joined #tripleo | 20:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 20:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 20:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 20:10 |
*** ooolpbot has quit IRC | 20:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 20:10 |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 20:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 20:10 |
*** stevebaker has joined #tripleo | 20:14 | |
openstackgerrit | Rafael Folco proposed openstack/tripleo-quickstart-extras master: WIP: Fix libvirt_type kvm for overcloud deploy https://review.openstack.org/587572 | 20:19 |
mwhahaha | beekneemech: ok you still around to troubleshoot some ipxe or are you about to leave | 20:22 |
beekneemech | mwhahaha: I have a little more time. | 20:23 |
*** raildo has quit IRC | 20:25 | |
openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates stable/queens: DO-NOT-MERGE: testing fix for related bug https://review.openstack.org/588654 | 20:27 |
*** jcoufal has quit IRC | 20:29 | |
*** ansmith has quit IRC | 20:34 | |
*** gbarros has joined #tripleo | 20:35 | |
*** abays has quit IRC | 20:36 | |
*** agopi has quit IRC | 20:43 | |
*** abishop has quit IRC | 20:45 | |
openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: Workflow for provisioning bare metals without Nova https://review.openstack.org/576856 | 20:45 |
openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: Switch away from using token_endpoint keystoneauth plugin https://review.openstack.org/585765 | 20:47 |
openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-common master: [WIP] Collect required information for deploying without Nova https://review.openstack.org/586279 | 20:47 |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: WIP - Podman support for docker-puppet https://review.openstack.org/588655 | 20:53 |
*** beekneemech is now known as bnemec-pto | 20:55 | |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: WIP - Podman support for docker-puppet https://review.openstack.org/588655 | 20:56 |
*** rfolco|ruck is now known as rfolco|off | 21:01 | |
*** gbarros has quit IRC | 21:02 | |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-common master: Add 'podman' rpm to base_centos containers https://review.openstack.org/588659 | 21:05 |
openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: DNM - test containerized undercloud with podman https://review.openstack.org/588660 | 21:06 |
*** ooolpbot has joined #tripleo | 21:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 21:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 21:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 21:10 |
*** ooolpbot has quit IRC | 21:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 21:10 |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 21:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 21:10 |
*** eck` is now known as eck`gone | 21:26 | |
weshay | bfournie, rlandy ok.. ovb job running introspection now | 21:31 |
weshay | I have the console up on all nodes | 21:31 |
*** gbarros has joined #tripleo | 21:33 | |
rlandy | ok | 21:33 |
*** apetrich has quit IRC | 21:34 | |
weshay | rlandy, it's hanging on node import | 21:35 |
rlandy | weshay: that's great | 21:35 |
rlandy | what's on your bmc log? | 21:35 |
weshay | hrm.. do I have access? | 21:35 |
weshay | rlandy, | 21:35 |
rlandy | weshay, yse | 21:37 |
rlandy | yes | 21:37 |
rlandy | horizon | 21:37 |
rlandy | click on the instance | 21:37 |
weshay | ya. I'm there | 21:37 |
rlandy | increase the log length | 21:37 |
rlandy | I have one node timing out on introspection | 21:38 |
*** pcaruana has quit IRC | 21:38 | |
rlandy | but it's trying again ... Retrying 1 nodes that failed introspection. Attempt 1 of | 21:38 |
rlandy | weshay: see any errors? | 21:38 |
rlandy | unreachable anything? | 21:38 |
weshay | no not yet | 21:39 |
weshay | just a boot log | 21:39 |
weshay | on the bmc | 21:39 |
rlandy | [104870.686499] openstackbmc[2567]: Reporting power state "False" for instance 6eea2153-83e9-41a6-92b4-a5c956f3a9e3 | 21:39 |
rlandy | ^^ nothing like | 21:39 |
weshay | nope | 21:39 |
rlandy | really? | 21:40 |
rlandy | one the console log - weird | 21:40 |
weshay | I can kill the job and rerun the import | 21:40 |
rlandy | weshay: what is the last command output you see on the undercloud? | 21:41 |
weshay | openstack overcloud node import instackenv.json | 21:42 |
rlandy | if it's making an ipmp call, we should see it on the bmc | 21:42 |
weshay | trying again | 21:42 |
rlandy | ok | 21:42 |
*** gbarros has quit IRC | 21:43 | |
*** gouthamr has joined #tripleo | 21:47 | |
weshay | hrm.. not sure rlandy | 21:48 |
weshay | let's chat here | 21:48 |
weshay | noticing the ovb bmc code fail to load | 21:48 |
weshay | http://paste.openstack.org/show/727312/ | 21:48 |
weshay | https://phx2.cloud.rdoproject.org/dashboard/project/instances/52a9b368-7d36-495f-ae8e-8c9fdd981439/console | 21:48 |
weshay | mwhahaha, fyi | 21:49 |
mwhahaha | the parse error: Invalid numeric literal at line 1, column 15 lines are not problems | 21:50 |
rlandy | not sure how that is sporadic | 21:50 |
mwhahaha | the ip netmask thing is tho | 21:50 |
mwhahaha | weshay: so yea if the neutron output in the bmc code isn't properly setting up the ips that would explain the introspection problems | 21:53 |
mwhahaha | let me go see where that comes from | 21:53 |
mwhahaha | weshay: the prefix len comes from... prefix_len=$(neutron subnet-show -f value -c cidr $private_subnet | awk -F / '{print $2}') | 21:55 |
* mwhahaha would like to point out that we shouldn't still be using neutron cli calls but that's for another day | 21:56 | |
*** rh-jelabarre has quit IRC | 21:56 | |
weshay | neutron net-show -f value -c subnets private-43901 | 21:56 |
weshay | Unable to establish connection to https://phx2.cloud.rdoproject.org:13000/v2.0/tokens: HTTPSConnectionPool(host='phx2.cloud.rdoproject.org', port=13000): Max retries exceeded with url: /v2.0/tokens (Caused by NewConnectionError('<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x1eace10>: Failed to establish a new connection: [Errno -2] Name or service not known',)) | 21:56 |
weshay | [ 121.954681] cloud-init[2067]: + private_subnet= | 21:56 |
mwhahaha | dns eh | 21:56 |
weshay | probably | 21:57 |
mwhahaha | weshay: what's the dns server on the bmc | 21:57 |
mwhahaha | if you tell me 8.8.8.8 i'm going to hurt you | 21:57 |
weshay | lolz | 21:57 |
* weshay checks | 21:58 | |
weshay | too many console windows up | 21:58 |
weshay | | dns_nameservers | 38.145.33.91 | 22:00 |
weshay | tenant provided | 22:00 |
mwhahaha | is that working correctly? | 22:00 |
weshay | http://paste.openstack.org/show/727314/ | 22:00 |
weshay | mwhahaha, sec.. I don't have access to the bmc node | 22:01 |
mwhahaha | weshay: I would would say try switching the dns servers to 1.1.1.1 and see if it goes away | 22:01 |
*** dsneddon has quit IRC | 22:02 | |
weshay | [zuul@undercloud ~]$ cat /etc/resolv.conf | 22:02 |
weshay | ; generated by /usr/sbin/dhclient-script | 22:02 |
weshay | search rdocloud | 22:02 |
weshay | nameserver 38.145.33.91 | 22:02 |
weshay | nameserver 38.145.32.66 | 22:02 |
weshay | [zuul@undercloud ~]$ ping phx2.cloud.rdoproject.org | 22:03 |
weshay | PING phx2.cloud.rdoproject.org (38.145.32.1) 56(84) bytes of data. | 22:03 |
weshay | 64 bytes from phx2.cloud.rdoproject.org (38.145.32.1): icmp_seq=1 ttl=63 time=2.68 ms | 22:03 |
weshay | mwhahaha, if it was dns it was a hiccup | 22:03 |
*** dsneddon has joined #tripleo | 22:03 | |
weshay | why the f. is it always dns | 22:03 |
mwhahaha | weshay: because reasons | 22:03 |
weshay | mwhahaha, bah.. you like the anti budda | 22:03 |
weshay | you are like | 22:03 |
weshay | I always come to you asking questions and you never have any answers ;P | 22:04 |
mwhahaha | oh i have answers | 22:04 |
mwhahaha | you just don't like them | 22:04 |
weshay | lolz | 22:04 |
weshay | rlandy, /me wonders if any jobs have started pass w/ fs001 | 22:04 |
weshay | since the reboots | 22:04 |
rlandy | checking | 22:04 |
* weshay needs more data.. and.. f.. access | 22:04 | |
rlandy | not sure if they would get there yet | 22:04 |
rlandy | reboot only finished | 22:05 |
rlandy | ~1hr | 22:05 |
weshay | rlandy, so I don't think we're downloading the ssh key for the bmc node in ci.. | 22:05 |
weshay | that is something I think panda|rover|off was fixing as well | 22:05 |
rlandy | no | 22:05 |
*** dsneddon has quit IRC | 22:05 | |
*** dsneddon has joined #tripleo | 22:05 | |
mwhahaha | weshay: yea it's the default key | 22:05 |
mwhahaha | so unless you inject the key into the image, i don't think we have a way of doing that | 22:06 |
mwhahaha | weshay: so can we switch the dns servers just for giggles? | 22:06 |
weshay | mwhahaha, I could in my tenant | 22:06 |
rlandy | 2018-08-03 15:27 || 99 min || /home/zuul/overcloud-prep-images.sh fail. | | 22:06 |
weshay | but I atm I can't get on the instance | 22:06 |
mwhahaha | we'd need to do it in the ci tneant | 22:06 |
rlandy | ^^ but they may be before | 22:06 |
*** hgibson has quit IRC | 22:06 | |
weshay | rlandy, k | 22:07 |
weshay | mwhahaha, meh.. this is my tenant | 22:07 |
weshay | guess I can do that | 22:07 |
mwhahaha | weshay: at this point i'm trying to rule out any flakiness in the rdo as much as possible | 22:08 |
weshay | mwhahaha, so you think the dhcp client will update the dns in a reasonable time or I should retry the whole thing w/ 1..1.1.1 | 22:08 |
mwhahaha | weshay: so not using a set of servers hosted in the cloud would be helpful | 22:08 |
weshay | mwhahaha, it's not a hosted dns service | 22:08 |
weshay | it's dnsmasque for the private network | 22:08 |
weshay | the tenant dns service | 22:09 |
weshay | faker | 22:09 |
weshay | PHAKER | 22:09 |
mwhahaha | no that would be 192.x addresses if that was true | 22:09 |
mwhahaha | those are set on the subnet | 22:09 |
openstackgerrit | Jill Rouleau proposed openstack/tripleo-heat-templates master: WIP: DeployedServer support for cidr via get_attr https://review.openstack.org/588122 | 22:09 |
mwhahaha | or are we using external addresses as the defaults | 22:09 |
*** ooolpbot has joined #tripleo | 22:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 22:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 22:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 22:10 |
*** ooolpbot has quit IRC | 22:10 | |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 22:10 |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 22:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 22:10 |
weshay | mwhahaha, rlandy of course he's right | 22:11 |
weshay | dang you | 22:11 |
mwhahaha | DON'T QUESTION ME SON | 22:11 |
weshay | mwhahaha, ok.. trying w/ 1.1.1.1 | 22:11 |
weshay | lolz | 22:11 |
weshay | I should crack a beer | 22:11 |
mwhahaha | 1.1.1.1 and 1.0.0.1 | 22:11 |
weshay | and just retry the introspection | 22:11 |
weshay | ? | 22:11 |
weshay | or re-doit | 22:11 |
mwhahaha | no you'd need to rebuild the env | 22:11 |
mwhahaha | or reboot your bmc | 22:12 |
mwhahaha | or manually re-run the cloud-init stuff | 22:12 |
mwhahaha | bash /var/lib/cloud/instances/*/scripts/part-001 | 22:12 |
weshay | hrm.. still getting dep errors on the service | 22:15 |
weshay | SPAM | 22:15 |
weshay | SPAM | 22:15 |
weshay | 3 | 22:15 |
weshay | 2 | 22:15 |
weshay | 1 | 22:15 |
weshay | See 'systemctl status config-bmc-ips.service' for details. | 22:15 |
weshay | [[1;33mDEPEND[0m] Dependency failed for openstack-bmc baremetal-43901_0 Service. | 22:15 |
weshay | [[1;33mDEPEND[0m] Dependency failed for openstack-bmc baremetal-43901_1 Service. | 22:15 |
weshay | [[1;33mDEPEND[0m] Dependency failed for openstack-bmc baremetal-43901_2 Service. | 22:15 |
weshay | [[1;33mDEPEND[0m] Dependency failed for openstack-bmc baremetal-43901_3 Service.[ 7.163603] os-net-config[761]: obj = objects.object_from_json(iface_json) | 22:15 |
weshay | [ 7.164878] os-net-config[761]: File "/usr/lib/python2.7/site-packages/os_net_config/objects.py", line 38, in object_from_json | 22:15 |
weshay | [ 7.166767] os-net-config[761]: return Interface.from_json(json) | 22:15 |
weshay | [ 7.167866] os-net-config[761]: File "/usr/lib/python2.7/site-packages/os_net_config/objects.py", line 305, in from_json | 22:15 |
weshay | [ 7.169671] os-net-config[761]: opts = _BaseOpts.base_opts_from_json(json) | 22:15 |
weshay | [ 7.170901] os-net-config[761]: File "/usr/lib/python2.7/site-packages/os_net_config/objects.py", line 256, in base_opts_from_json | 22:15 |
* mwhahaha shrugs | 22:16 | |
mwhahaha | is that from the reboot or did you rerun the cloud-init stuff | 22:16 |
weshay | reboot | 22:17 |
weshay | trying to import anyway atm | 22:17 |
mwhahaha | yea you probably need to rerun the cloud init bits | 22:17 |
mwhahaha | check the ips are properly defined | 22:17 |
weshay | oh ya.. it turns it off after the first time | 22:17 |
weshay | 1dot1dot1dot1.cloudflare-dns.com | 22:20 |
weshay | nice name | 22:20 |
mwhahaha | speaking of broken things | 22:21 |
mwhahaha | scenario001 in queens looks to be broken | 22:21 |
weshay | mwhahaha, /me checks the image | 22:22 |
mwhahaha | nothing but timeouts | 22:22 |
mwhahaha | it's hanging on step3 | 22:23 |
mwhahaha | lovely | 22:23 |
weshay | probably dns | 22:23 |
mwhahaha | who backported something that broke everything | 22:23 |
* mwhahaha flips tables | 22:23 | |
weshay | lolz | 22:23 |
mwhahaha | i bet it's some stupid ceph thing anyway | 22:24 |
weshay | mwhahaha, what log are you looking at? | 22:24 |
weshay | I see something else going on in the latest check jobs | 22:24 |
mwhahaha | http://logs.openstack.org/53/587353/1/gate/tripleo-ci-centos-7-scenario001-multinode-oooq-container/9f50b78/ | 22:25 |
mwhahaha | i'm currently looking at that one | 22:25 |
mwhahaha | but none of the stable/queens have passed recently on http://zuul.openstack.org/builds.html?job_name=tripleo-ci-centos-7-scenario001-multinode-oooq-container | 22:25 |
weshay | oh ya I was looking at the wrong thing | 22:25 |
* weshay looks again | 22:25 | |
weshay | yes.. see that | 22:27 |
rlandy | there are a bunch of jobs at the undercloud install stage - so ci should report on introspection soon | 22:28 |
mwhahaha | rlandy: won't do any good, cause dns change won't take effect until the bmc nodes were booted | 22:28 |
mwhahaha | so likely won't show up until the next batch of ci jobs start | 22:29 |
weshay | mwhahaha, rlandy I'm trying the dns change first locally | 22:29 |
mwhahaha | pffft | 22:29 |
mwhahaha | where's your sense of adventure | 22:29 |
weshay | tru | 22:29 |
mwhahaha | do it in prod on a friday afternoon | 22:29 |
weshay | lolz | 22:29 |
weshay | to quote 12 monkeys | 22:30 |
weshay | f.. the bozo's | 22:30 |
weshay | bozos | 22:30 |
weshay | lolz | 22:30 |
mwhahaha | i mean it's not like it's actually working right now anyway | 22:30 |
mwhahaha | as long as we don't change another thing till it's been vetted, should be fine :D | 22:30 |
*** boazel has joined #tripleo | 22:32 | |
openstackgerrit | Carlos Goncalves proposed openstack/diskimage-builder master: Install ca-certificate with redhat-common https://review.openstack.org/588676 | 22:32 |
weshay | ; generated by /usr/sbin/dhclient-script | 22:33 |
weshay | search rdocloud | 22:33 |
weshay | nameserver 1.1.1.1 | 22:33 |
mwhahaha | ok so back to scenario001, it's actually failing in step5 it seems | 22:34 |
mwhahaha | or is just running really long | 22:34 |
mwhahaha | i have no idea what would hang step5 | 22:34 |
mwhahaha | that's a new one | 22:34 |
* mwhahaha blames gnocchi or something | 22:34 | |
weshay | k.. crap | 22:34 |
* weshay will bug it | 22:34 | |
mwhahaha | yea so if it's scenario001 and step5 it's likely ceilometer | 22:35 |
* mwhahaha blames pradk but he never shows up anymore | 22:35 | |
mwhahaha | weshay: yea gnocchi_db_sync was running for about an hour | 22:39 |
mwhahaha | weshay: so gnocchi is hanging in queens | 22:39 |
* mwhahaha wish he was kidding when he first said gnocchi | 22:39 | |
mwhahaha | http://logs.openstack.org/53/587353/1/gate/tripleo-ci-centos-7-scenario001-multinode-oooq-container/9f50b78/logs/subnode-2/var/log/extra/docker/docker_allinfo.log.txt.gz | 22:40 |
mwhahaha | http://logs.openstack.org/53/587353/1/gate/tripleo-ci-centos-7-scenario001-multinode-oooq-container/9f50b78/logs/subnode-2/var/log/extra/docker/containers/gnocchi_db_sync/stdout.log.txt.gz | 22:40 |
mwhahaha | ceph vs gnocchi | 22:41 |
* mwhahaha throws things at fultonj cause he's the only one here | 22:41 | |
mwhahaha | TimedOut: [errno 110] error opening pool 'metrics' | 22:41 |
*** rlandy has quit IRC | 22:42 | |
*** derekh has joined #tripleo | 22:43 | |
bfournie | mwhahaha, weshay: looking at dependency issue in scrollback - wonder if is this due to https://bugzilla.redhat.com/show_bug.cgi?id=1578440 | 22:44 |
openstack | bugzilla.redhat.com bug 1578440 in os-net-config "Drop OVS dependency from os-net-config to fix future Entitlement issues" [High,Closed: errata] - Assigned to bfournie | 22:44 |
derekh | weshay: I've been trying to reproduce the problem, set up a env and have rerun introspection a bunch of times, no failures yet | 22:44 |
weshay | derekh, seems like the bmc is not starting due to dep issues | 22:44 |
mwhahaha | bfournie: no weshay's error was during the bmc setup which doesn't use that bits. we don't use ovs to configure the bmc | 22:45 |
bfournie | mwhahaha: k | 22:45 |
fultonj | does the metrics pool exist as per 'ceph df' ; and do we catpure that output in CI | 22:45 |
mwhahaha | fultonj: i don't think we capture that, through it would be a good item to add to the hostinfo | 22:46 |
fultonj | keyring looks ok | 22:47 |
fultonj | http://logs.openstack.org/53/587353/1/gate/tripleo-ci-centos-7-scenario001-multinode-oooq-container/9f50b78/logs/subnode-2/etc/ceph/ | 22:47 |
weshay | https://phx2.cloud.rdoproject.org/dashboard/project/instances/031a8a5b-1eb6-4476-89b9-2375000af709/console | 22:48 |
derekh | weshay: If its some dependency its weird that its intermittent | 22:48 |
weshay | DAM YOU mwhahaha | 22:48 |
* weshay putting in a change | 22:48 | |
mwhahaha | weshay: well since it is a sporadic failure thing, we'll only really know at scale :D | 22:49 |
weshay | tru | 22:49 |
mwhahaha | fultonj: i see calls to info about the metrics pool in the deploy logs http://logs.openstack.org/53/587353/1/gate/tripleo-ci-centos-7-scenario001-multinode-oooq-container/9f50b78/logs/subnode-2/var/log/journal.txt.gz#_Aug_02_17_52_32 | 22:49 |
* mwhahaha give sup for the day | 22:50 | |
mwhahaha | see you folks on monday | 22:50 |
fultonj | if the metrics pool was not there we would not have gotten past step4 | 22:51 |
fultonj | a bad metrics pool will prevent gnochi from standing up at that step | 22:51 |
*** agopi has joined #tripleo | 22:51 | |
mwhahaha | fultonj: so that's step5 now i thought and that's what it's failing on | 22:55 |
fultonj | mwhahaha: that's my point is that we got to step5 so therefore the metrics pool was working well enough at that point | 22:55 |
* mwhahaha shrugs | 22:57 | |
fultonj | that's step5 now | 22:57 |
mwhahaha | yea i think it's hanging on that | 22:57 |
mwhahaha | anyway | 22:57 |
mwhahaha | we'll figure it out monday i guess | 22:57 |
fultonj | you mean gnochi no longer gets runing at step4 | 22:57 |
fultonj | if ghnochi now starts running at 5 instead, then that's a problem in my reasoning | 22:57 |
fultonj | yeah monday | 22:57 |
* fultonj going back to scale lab... | 22:58 | |
* weshay is going back to cali | 22:58 | |
openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: update rdocloud dns settings to outside provider https://review.openstack.org/588678 | 22:59 |
openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: update rdo-cloud dns to outside provider https://review.openstack.org/588679 | 23:00 |
weshay | derekh, ^ we're trying that ^ | 23:01 |
derekh | weshay: worth a try, my bmc has "nameserver 38.145.33.91" | 23:03 |
weshay | derekh, ya. that's what we were using | 23:03 |
weshay | maybe that resolved well enough at times, but not all the time? | 23:03 |
*** mschuppert has quit IRC | 23:03 | |
*** tosky has quit IRC | 23:04 | |
weshay | mwhahaha, ya.. no-op jobs for queens https://review.openstack.org/#/c/567224 | 23:04 |
weshay | scen001 is down for sure | 23:04 |
derekh | weshay: it would definitely explain intermittent ipmi errors, coupled with recent cloud stability issues | 23:05 |
weshay | derekh, /me picks an appropriate nirvana song to listen to now | 23:05 |
weshay | this job invents new reasons to drink | 23:07 |
derekh | weshay: I gotta run, afraid I'm not back until Tuesday, hopefully that sorts things out | 23:07 |
weshay | :) | 23:07 |
weshay | take care brotha | 23:07 |
derekh | weshay: who needs new reason, I've been using the same reason for years | 23:07 |
weshay | :) | 23:07 |
derekh | because I wanna | 23:07 |
derekh | ttyl, have a good evening | 23:08 |
*** derekh has quit IRC | 23:08 | |
*** ooolpbot has joined #tripleo | 23:10 | |
ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784017 | 23:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784822 | 23:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1784847 | 23:10 |
ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1785342 | 23:10 |
openstack | Launchpad bug 1784017 in tripleo "Build of instance was re-scheduled: invalid argument: could not find capabilities for domaintype=kvm" [Critical,In progress] - Assigned to Rafael Folco (rafaelfolco) | 23:10 |
*** ooolpbot has quit IRC | 23:10 | |
openstack | Launchpad bug 1784822 in tripleo "legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master fails contact ipmi" [Critical,Triaged] | 23:10 |
openstack | Launchpad bug 1784847 in tripleo "exception introspecting nodes" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 23:10 |
openstack | Launchpad bug 1785342 in tripleo "public rdo-cloud dns settings are causing bmc setup issues intermittently " [Critical,In progress] - Assigned to wes hayutin (weshayutin) | 23:10 |
weshay | bfournie, I'll for sure need to buy you a beer or three | 23:18 |
weshay | ci people need a beer budget | 23:18 |
*** boazel has quit IRC | 23:45 | |
*** rpioso is now known as rpioso|afk | 23:45 | |
bfournie | weshay: nice, I'll see you in denver :-) although I didn't do anything to figure out 1785342 | 23:48 |
*** mjturek has quit IRC | 23:52 | |
weshay | bfournie, it's working thus far https://phx2.cloud.rdoproject.org/dashboard/project/instances/031a8a5b-1eb6-4476-89b9-2375000af709/console | 23:53 |
weshay | 2018-08-03 23:55:10 | Successfully introspected 4 node(s). | 23:55 |
weshay | 2018-08-03 23:55:10 | | 23:55 |
weshay | 2018-08-03 23:55:10 | Introspection completed. | 23:55 |
weshay | 2018-08-03 23:55:10 | + openstack overcloud node pr | 23:55 |
bfournie | weshay: awesome | 23:56 |
weshay | 2018-08-03 23:57:46 | 4 node(s) successfully moved to the "available" state. | 23:57 |
weshay | https://review.openstack.org/#/c/588678/ https://review.openstack.org/#/c/588679/ | 23:58 |
weshay | panda|rover|off, rfolco|off marios ^ | 23:58 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!