| *** gbarros has joined #tripleo | 00:01 | |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
|---|---|---|
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714412 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 00:10 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1714361 in tripleo "undercloud reinstall is unstable" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 00:10 |
| openstack | Launchpad bug 1714412 in tripleo "Upgrade from Ocata to Pike fails to pull containers during ControllerDeployment_Step1" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 00:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,Triaged] | 00:10 |
| *** gbarros has quit IRC | 00:25 | |
| *** ansmith has quit IRC | 00:58 | |
| *** limao has joined #tripleo | 01:00 | |
| *** psahoo has joined #tripleo | 01:01 | |
| *** gbarros has joined #tripleo | 01:02 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714412 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 01:10 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1714361 in tripleo "undercloud reinstall is unstable" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 01:10 |
| openstack | Launchpad bug 1714412 in tripleo "Upgrade from Ocata to Pike fails to pull containers during ControllerDeployment_Step1" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 01:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,Triaged] | 01:10 |
| *** dixiaoli has joined #tripleo | 01:11 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Upgrade CI test - never merge https://review.openstack.org/461000 | 01:12 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Upgrade CI test - never merge https://review.openstack.org/461000 | 01:12 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Fix containerized zaqar-api db_sync https://review.openstack.org/500413 | 01:28 |
| *** gbarros has quit IRC | 01:35 | |
| openstackgerrit | Merged openstack/tripleo-docs master: Document the need to clean the Ceph disks when redeploying https://review.openstack.org/499979 | 01:35 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Switch manila-share to pacemaker version in scenario004/containers https://review.openstack.org/500314 | 01:36 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: enable pingtest on fs019 https://review.openstack.org/498575 | 01:36 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Persist containerized services httpd logs https://review.openstack.org/499235 | 01:37 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Remove bgp-vpn from scenario004-multinode-containers https://review.openstack.org/499626 | 01:37 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: manila: set "neutron_admin_auth_url" correctly https://review.openstack.org/500145 | 01:38 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Unset default value for the DockerCephDaemonImage https://review.openstack.org/500150 | 01:39 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Add clustercheck to service list for scenarios https://review.openstack.org/499133 | 01:40 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: Add scenarios environment files to prepare cmd https://review.openstack.org/500364 | 01:48 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Upgrade CI test - never merge https://review.openstack.org/461000 | 01:49 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Upgrade CI test - never merge https://review.openstack.org/461000 | 01:49 |
| *** fzdarsky__ has joined #tripleo | 01:52 | |
| openstackgerrit | Steve Baker proposed openstack/python-tripleoclient master: Command to discover the versioned tag from latest https://review.openstack.org/498683 | 01:53 |
| *** fzdarsky_ has quit IRC | 01:56 | |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714412 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 02:10 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1714361 in tripleo "undercloud reinstall is unstable" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 02:10 |
| openstack | Launchpad bug 1714412 in tripleo "Upgrade from Ocata to Pike fails to pull containers during ControllerDeployment_Step1" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 02:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,Triaged] | 02:10 |
| EmilienM | stevebaker: hey | 02:16 |
| EmilienM | stevebaker: sounds like https://review.openstack.org/#/c/493391/ is helping. Are we sure --pull-source was really backward compatible? | 02:19 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Switch scenario004-container to run Tempest https://review.openstack.org/500423 | 02:53 |
| *** ramishra has joined #tripleo | 03:01 | |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714412 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 03:10 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1714361 in tripleo "undercloud reinstall is unstable" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 03:10 |
| openstack | Launchpad bug 1714412 in tripleo "Upgrade from Ocata to Pike fails to pull containers during ControllerDeployment_Step1" [Critical,In progress] - Assigned to Steve Baker (steve-stevebaker) | 03:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,Triaged] | 03:10 |
| *** udesale has joined #tripleo | 03:19 | |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Stop using deprecated --pull-source https://review.openstack.org/493391 | 03:22 |
| *** ykarel has joined #tripleo | 03:23 | |
| *** yamahata has joined #tripleo | 03:30 | |
| *** gkadam has joined #tripleo | 03:48 | |
| *** stendulker has joined #tripleo | 03:49 | |
| stevebaker | EmilienM: I must have messed something up in this change, but it may not even be worth fixing now that there are no known uses of --pull-source | 04:02 |
| stevebaker | EmilienM: https://review.openstack.org/#/c/489837/ | 04:02 |
| *** skramaja has joined #tripleo | 04:06 | |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 04:10 |
| openstack | Launchpad bug 1714361 in tripleo "undercloud reinstall is unstable" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 04:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,Triaged] | 04:10 |
| *** pdeore has joined #tripleo | 04:11 | |
| *** links has joined #tripleo | 04:19 | |
| *** janki has joined #tripleo | 04:19 | |
| *** shreshtha has joined #tripleo | 04:21 | |
| *** noslzzp_ has quit IRC | 04:24 | |
| EmilienM | stevebaker: you wanna backport it? | 04:25 |
| stevebaker | EmilienM: yes, we'll need to for ceph images | 04:26 |
| *** dsariel has quit IRC | 04:28 | |
| *** psachin has joined #tripleo | 04:28 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common stable/pike: Upload remove default for pull_source https://review.openstack.org/500438 | 04:30 |
| EmilienM | stevebaker: done ^ | 04:30 |
| EmilienM | stevebaker: i've approved https://review.openstack.org/#/c/500364/ as well, maybe you can double check in case | 04:31 |
| stevebaker | EmilienM: sure, I've just updated this bug https://bugs.launchpad.net/tripleo/+bug/1702764 | 04:31 |
| openstack | Launchpad bug 1702764 in tripleo "container images enable the epel repository" [Critical,Fix released] - Assigned to David Moreau Simard (dmsimard) | 04:31 |
| EmilienM | stevebaker: ack | 04:32 |
| EmilienM | stevebaker: upgrade job timeouts, it's really hard to tell how it works. At least the workflow works again now | 04:33 |
| EmilienM | we'll need to see logs on https://review.openstack.org/#/c/461000/ when job finish, all upgrade job failed | 04:33 |
| *** jaosorior has joined #tripleo | 04:36 | |
| EmilienM | stevebaker: we have a new issue | 04:38 |
| EmilienM | http://logs.openstack.org/00/461000/54/check/gate-tripleo-ci-centos-7-scenario003-multinode-oooq-container-upgrades-nv/d31863a/logs/undercloud/home/jenkins/overcloud_deploy.log.txt.gz#_2017-09-04_02_47_25 | 04:38 |
| EmilienM | esources.ServiceChain: Property error: resources[35].properties: Property DockerMistralExecutorImage not assigned | 04:38 |
| EmilienM | which sounds like the BarbicanApi one we had | 04:38 |
| stevebaker | EmilienM: Is there a bug for it? | 04:38 |
| EmilienM | stevebaker: no I just found it | 04:39 |
| stevebaker | EmilienM: ok, I'll take a look and raise one | 04:39 |
| EmilienM | stevebaker: thank you | 04:40 |
| EmilienM | stevebaker: DockerMistralExecutorImage is in tripleo-common, weird | 04:40 |
| stevebaker | EmilienM: maybe its missing a services entry | 04:40 |
| EmilienM | but you probably know better | 04:40 |
| EmilienM | ok | 04:40 |
| EmilienM | stevebaker: I have a similar issue on scenario002 | 04:41 |
| EmilienM | http://logs.openstack.org/00/461000/54/check/gate-tripleo-ci-centos-7-scenario002-multinode-oooq-container-upgrades-nv/f3a1eac/logs/undercloud/home/jenkins/overcloud_deploy.log.txt.gz#_2017-09-04_02_46_41 | 04:41 |
| EmilienM | resources.ServiceChain: Property error: resources[42].properties: Property DockerEc2ApiConfigImage not assigned | 04:41 |
| EmilienM | and on scenario001 | 04:42 |
| EmilienM | http://logs.openstack.org/00/461000/54/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container-upgrades-nv/702da7a/logs/undercloud/home/jenkins/overcloud_deploy.log.txt.gz#_2017-09-04_02_43_52 | 04:42 |
| EmilienM | resources.ServiceChain: Property error: resources[47].properties: Property DockerPankoConfigImage not assigned | 04:42 |
| EmilienM | stevebaker: I'm wondering if it's because of https://review.openstack.org/#/c/500364/ (I added it in Depends-On) | 04:42 |
| EmilienM | same on scenario004 | 04:43 |
| EmilienM | http://logs.openstack.org/00/461000/54/check/gate-tripleo-ci-centos-7-scenario004-multinode-oooq-container-upgrades-nv/d2c3621/logs/undercloud/home/jenkins/overcloud_deploy.log.txt.gz | 04:43 |
| EmilienM | resources.ServiceChain: Property error: resources[27].properties: Property DockerManilaShareImage not assigned | 04:43 |
| stevebaker | EmilienM: its like random entries fail | 04:43 |
| EmilienM | stevebaker: can you file a bug with them in a single single bug? | 04:43 |
| EmilienM | they aren't random I think | 04:43 |
| EmilienM | they correspond to the scenarios where services are deployed | 04:44 |
| stevebaker | EmilienM: oh, maybe missing some -e arguments to the prepare command | 04:44 |
| EmilienM | stevebaker: we can hold the patch on if you want :) | 04:44 |
| EmilienM | I can -2 to block it, it' sin the gate now | 04:45 |
| stevebaker | EmilienM: which one? | 04:45 |
| EmilienM | stevebaker: https://review.openstack.org/#/c/500364 | 04:45 |
| stevebaker | EmilienM: that change looks like it would fix this problem | 04:46 |
| EmilienM | stevebaker: well, I used Depends-On with this patch | 04:46 |
| EmilienM | stevebaker: the thing is it fails during the deployment | 04:47 |
| EmilienM | stevebaker: not the upgrade | 04:47 |
| EmilienM | stevebaker: and the deployment should deploy ocata | 04:47 |
| stevebaker | EmilienM: ok, that should be enough context for me to investigate | 04:48 |
| EmilienM | stevebaker: so gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv is passing again now, which is good | 04:48 |
| EmilienM | stevebaker: but the upgrade scenarios don't | 04:48 |
| *** dpawar has joined #tripleo | 04:49 | |
| stevebaker | right | 04:49 |
| EmilienM | stevebaker: I'm doing a recheck on https://review.openstack.org/#/c/500145 to see how it works on stable/pike, we'll see how upgrade work on gate-tripleo-ci-centos-7-scenario004-multinode-oooq-container-upgrades-nv | 04:50 |
| EmilienM | stevebaker: without the Depends-on the quickstart-extras patch | 04:50 |
| EmilienM | stevebaker: thanks for the help, I'm going offline in a few | 04:51 |
| stevebaker | no problem | 04:51 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo stable/ocata: Remove extra keystone admin haproxy listen and allow TLS https://review.openstack.org/494947 | 04:52 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates stable/pike: Persist containerized services httpd logs https://review.openstack.org/499235 | 04:55 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: DNM - Run tempest after an upgrade Ocata to Pike https://review.openstack.org/500440 | 05:01 |
| *** garyk1 has joined #tripleo | 05:04 | |
| *** ratailor has joined #tripleo | 05:05 | |
| *** dparkes has quit IRC | 05:08 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 05:10 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1714361 in tripleo "undercloud reinstall is unstable" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 05:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,Triaged] | 05:10 |
| EmilienM | stevebaker: I posted on ML, good evening sir, ttyl | 05:23 |
| *** stendulker_ has joined #tripleo | 05:35 | |
| *** stendulker has quit IRC | 05:35 | |
| *** hjensas has quit IRC | 05:45 | |
| *** jfrancoa has joined #tripleo | 05:59 | |
| *** masco has joined #tripleo | 05:59 | |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/472607 | 06:00 |
| *** shreshtha has quit IRC | 06:01 | |
| *** florianf has joined #tripleo | 06:03 | |
| *** iranzo has joined #tripleo | 06:03 | |
| jaosorior | bandini: hey dude, could you check these two out https://review.openstack.org/#/c/498324/ https://review.openstack.org/#/c/498325/ ? | 06:04 |
| *** hanish has joined #tripleo | 06:04 | |
| jaosorior | jistr|off: ^^ | 06:05 |
| *** ccamacho has joined #tripleo | 06:08 | |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 06:10 |
| openstack | Launchpad bug 1714361 in tripleo "undercloud reinstall is unstable" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 06:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,Triaged] | 06:10 |
| *** dparkes has joined #tripleo | 06:11 | |
| EmilienM | jaosorior: done | 06:16 |
| EmilienM | please verify the version of puppet-rabbitmq in RDO / pike | 06:16 |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-heat-templates master: Mount vhost_sockets directory for vhost-user socket creation https://review.openstack.org/499086 | 06:16 |
| EmilienM | not sure it has the commit you need | 06:16 |
| EmilienM | I'm afk now | 06:16 |
| *** marios has joined #tripleo | 06:17 | |
| *** agurenko has joined #tripleo | 06:21 | |
| *** pgadiya has joined #tripleo | 06:23 | |
| *** dsariel has joined #tripleo | 06:28 | |
| *** limao has quit IRC | 06:29 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates master: Change template names to queens https://review.openstack.org/500451 | 06:32 |
| ccamacho | Hello TripleOers!!!! | 06:32 |
| ccamacho | good morning!!! | 06:32 |
| *** jprovazn has joined #tripleo | 06:34 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates master: Change template names to queens https://review.openstack.org/500451 | 06:36 |
| *** yprokule has joined #tripleo | 06:47 | |
| lvdombrkr | morning folks, any ideas on this ? https://bugs.launchpad.net/tripleo/+bug/1714544 | 06:51 |
| openstack | Launchpad bug 1714544 in tripleo "cant delete stuck with network-isolation" [Medium,Triaged] | 06:51 |
| *** rcernin has quit IRC | 06:52 | |
| *** rcernin has joined #tripleo | 06:52 | |
| Tengu | hello! Small question: what should I check if `openstack overcloud profiles list` returns the intended node<->profile association, if `openstack overcloud profiles match [options]` returns 0, and if an overcloud deploy fails because it can't find any nodes? | 06:54 |
| *** ebarrera has joined #tripleo | 06:55 | |
| *** cschwede_ has joined #tripleo | 06:58 | |
| *** hjensas has joined #tripleo | 06:58 | |
| *** limao has joined #tripleo | 06:59 | |
| *** jpena|off is now known as jpena | 06:59 | |
| *** jlinkes has joined #tripleo | 07:02 | |
| *** yamahata has quit IRC | 07:07 | |
| *** ebarrera has quit IRC | 07:07 | |
| *** ebarrera has joined #tripleo | 07:08 | |
| *** karthiks_afk is now known as karthiks | 07:10 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 07:10 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1714361 in tripleo "undercloud reinstall is unstable" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 07:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,Triaged] | 07:10 |
| *** tesseract has joined #tripleo | 07:16 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates master: Correct template names from ocata to pike. https://review.openstack.org/500460 | 07:18 |
| openstackgerrit | Jan Provaznik proposed openstack/tripleo-heat-templates stable/pike: Escape ceph capabilities for manila client https://review.openstack.org/500462 | 07:18 |
| *** mcornea has joined #tripleo | 07:20 | |
| *** anshul has joined #tripleo | 07:21 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates master: Correct template names from ocata to pike. https://review.openstack.org/500460 | 07:21 |
| *** paramite has joined #tripleo | 07:21 | |
| *** fzdarsky__ is now known as fzdarsky | 07:21 | |
| *** jtomasek has joined #tripleo | 07:23 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates master: Correct template names from ocata to pike. https://review.openstack.org/500465 | 07:23 |
| lvdombrkr | morning folks, any ideas on this ? https://bugs.launchpad.net/tripleo/+bug/1714544 | 07:24 |
| openstack | Launchpad bug 1714544 in tripleo "cant delete stuck with network-isolation" [Medium,Triaged] | 07:24 |
| openstackgerrit | Michael Chapman proposed openstack/tripleo-quickstart-extras master: Add opendaylight to collect-logs https://review.openstack.org/494043 | 07:25 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates master: Change template names to queens https://review.openstack.org/500451 | 07:27 |
| *** jaganathan has joined #tripleo | 07:28 | |
| *** dciabrin has joined #tripleo | 07:29 | |
| *** aufi has joined #tripleo | 07:29 | |
| *** dtantsur|afk is now known as dtantsur | 07:34 | |
| *** jpich has joined #tripleo | 07:35 | |
| *** pcaruana has joined #tripleo | 07:36 | |
| *** pcaruana has quit IRC | 07:36 | |
| *** pcaruana has joined #tripleo | 07:37 | |
| Tengu | anyone having any idea regarding https://bugs.launchpad.net/tripleo/+bug/1714887 ? I'm a bit stuck right now with that issue :(. | 07:37 |
| openstack | Launchpad bug 1714887 in tripleo "[pike] Openstack overcloud deploy failed: not enough nodes" [Undecided,New] | 07:37 |
| *** stendulker_ has quit IRC | 07:37 | |
| *** oidgar has joined #tripleo | 07:37 | |
| *** lucas-afk is now known as lucasagomes | 07:41 | |
| *** rcernin has quit IRC | 07:41 | |
| *** shadower has joined #tripleo | 07:44 | |
| *** egonzalez has joined #tripleo | 07:45 | |
| *** shardy has joined #tripleo | 07:50 | |
| *** brault has joined #tripleo | 07:59 | |
| *** nyechiel has joined #tripleo | 07:59 | |
| *** stendulker_ has joined #tripleo | 08:03 | |
| *** dbecker has joined #tripleo | 08:04 | |
| ccamacho | hey folks quick question, did we have any deep dive session in the las 2 weeks? | 08:06 |
| ccamacho | s/las/last/ | 08:06 |
| lvdombrkr | morning folks, any ideas on this ? https://bugs.launchpad.net/tripleo/+bug/1714544 | 08:06 |
| openstack | Launchpad bug 1714544 in tripleo "cant delete stuck with network-isolation" [Medium,Triaged] | 08:06 |
| shardy | lvdombrkr: to debug we'll need more logs, e.g to figure out the root cause of the nova 500 error | 08:07 |
| *** nyechiel has quit IRC | 08:08 | |
| shardy | can you check the nova and neutron logs to see what the corresponding error before the 500 was? | 08:08 |
| *** shreshtha has joined #tripleo | 08:08 | |
| lvdombrkr | shardy: one sec | 08:09 |
| shardy | lvdombrkr: normally 500 errors indicate a misconfiguration or failure of the services, so also worth confirming all the nova and neutron services are running OK and not in a failed state | 08:09 |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 08:10 |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 08:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,Triaged] | 08:10 |
| *** ykarel is now known as ykarel|lunch | 08:10 | |
| Tengu | shardy: hello :). do you have some time/idea for https://bugs.launchpad.net/tripleo/+bug/1714887 ? sorry to bother you, but as you seem to be more or less present… I'd gladly provide more logs if needed. | 08:12 |
| openstack | Launchpad bug 1714887 in tripleo "[pike] Openstack overcloud deploy failed: not enough nodes" [Undecided,New] | 08:12 |
| lvdombrkr | shardy: i see that the nove-compoute not started :Active: activating (start) | 08:15 |
| lvdombrkr | shardy: in nova logs i see : http://paste.openstack.org/raw/620298/ | 08:17 |
| *** owalsh_ is now known as owalsh | 08:19 | |
| openstackgerrit | Saravanan KR proposed openstack/python-tripleoclient master: Add default kolla conf file for TripleO to build container images https://review.openstack.org/500475 | 08:20 |
| lvdombrkr | shardy: neutron services are up and running | 08:20 |
| shardy | Tengu: Hi, need some more information, added a comment to the bug | 08:20 |
| Tengu | shardy: thanks, I'll check that. | 08:21 |
| shardy | lvdombrkr: so there are rabbit related errors in the log output, is rabbitmq running OK? | 08:21 |
| Tengu | shardy: hmm, the full command is in the latest log entry. will copy it back in the issue. | 08:22 |
| *** matbu has quit IRC | 08:24 | |
| lvdombrkr | shardy : rabbitmq-server.service up and running | 08:24 |
| shardy | Tengu: Ok I see it, so we need the contents of the environment.yaml and the node show / flavor show output | 08:24 |
| shardy | it seems like a mismatch with the flavors you're requesting, but it's hard to be sure without that information | 08:25 |
| lvdombrkr | shardy: and nova compute after restart. service also is up and running | 08:25 |
| Tengu | shardy: hmm, the environment.yaml contains possibly sensitive data. any secure way to share it? | 08:25 |
| shardy | Tengu: I just need to know what *Flavor parameters are in it | 08:26 |
| Tengu | shardy: ok, will add it then. | 08:26 |
| *** derekh has joined #tripleo | 08:26 | |
| *** cylopez has joined #tripleo | 08:29 | |
| *** shadower has quit IRC | 08:29 | |
| *** shadower has joined #tripleo | 08:30 | |
| openstackgerrit | Saravanan KR proposed openstack/python-tripleoclient master: Add default kolla conf file for TripleO to build container images https://review.openstack.org/500475 | 08:30 |
| *** shadower has quit IRC | 08:32 | |
| *** shadower has joined #tripleo | 08:32 | |
| Tengu | shardy: hmm, but shouldn't the check command fails (openstack overcloud profiles *) if there's some mismatch? | 08:33 |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-common master: Added a kolla config file for building contaner images with qemu uid https://review.openstack.org/499053 | 08:34 |
| *** matbu has joined #tripleo | 08:37 | |
| *** nyechiel has joined #tripleo | 08:37 | |
| Tengu | shardy: issue updated. | 08:39 |
| *** cylopez has quit IRC | 08:41 | |
| shardy | Tengu: thanks, I can't see the mismatch atm, but can you please check the nova-scheduler.log on the undercloud, and ensure that nova hypervisor-stats shows the three nodes? | 08:46 |
| *** limao has quit IRC | 08:47 | |
| shardy | Tengu: personally I wouldn't add all the capabilities to the flavors, and would just use the profile to select the nodes | 08:47 |
| shardy | as it's really easy to get one key mismatched then nova won't select the nodes | 08:47 |
| Tengu | shardy: hmm, tha last command shows only 0 in the Value column :/ | 08:47 |
| Tengu | shardy: i.e. dropping the disk_label and boot_mode from the flavor ? | 08:48 |
| shardy | Tengu: Ok, so the nodes are registered in ironic, but not visible to nova | 08:48 |
| shardy | Tengu: yes, personally I find that less error prone, but it sounds like it's not your issue here | 08:48 |
| shardy | Tengu: are all the ironic services running OK? | 08:49 |
| Tengu | shardy: yep, that' what I concluded as well - and OK for the capabilities. I followed some doc for the GPT support, and it was pointing the flavor capabilities as well (https://docs.openstack.org/ironic/pike/install/advanced.html#choosing-the-disk-label) | 08:49 |
| shardy | Tengu: ack, yeah I guess its needed for the ironic nodes, but I don't think you need to specify every capability to select nodes via a nova flavor | 08:51 |
| shardy | just the profile should be enough AFAIK | 08:51 |
| Tengu | shardy: ok | 08:51 |
| *** sshnaidm|off is now known as sshnaidm | 08:54 | |
| *** cylopez has joined #tripleo | 08:55 | |
| Tengu | shardy: how may I check the ironic service status? | 08:56 |
| Tengu | oh. | 08:56 |
| Tengu | hmmm. | 08:56 |
| Tengu | 2017-09-04 09:12:25.880 17595 ERROR nova.servicegroup.drivers.db DBConnectionError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '10.27.100.1' ([Errno 111] ECONNREFUSED)") | 08:56 |
| Tengu | that's bad. | 08:57 |
| openstackgerrit | Gael Chamoulaud proposed openstack-infra/tripleo-ci master: Enable tripleo-validations tests https://review.openstack.org/481080 | 08:57 |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-quickstart master: Add release notes for TripleO-Validations patch. https://review.openstack.org/487744 | 08:57 |
| Tengu | shardy: a tail -f nova-scheduler.log doesn't show any error when I try my deploy command :/ | 08:58 |
| shardy | Tengu: yeah that sounds bad - for any service I start with sudo systemctl | grep ironic, then check each e.g sudo systemctl status openstack-ironic-conductor | 08:58 |
| Tengu | hmm. apparently, no error in the nova*.log in fact, during the deploy. | 09:00 |
| Tengu | the mysql part was probably a temporary issue, not reproducted for now | 09:00 |
| shardy | Tengu: the problem is nova thinks there are zero nodes available | 09:01 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Remove tacker from containers scenario001 https://review.openstack.org/500165 | 09:01 |
| Tengu | shardy: yup.checking the processes, 2s | 09:01 |
| shardy | so you need to figure out why, is openstack-ironic-conductor running? | 09:01 |
| Tengu | shardy: Active: active (running) since Mon 2017-09-04 09:14:14 CEST; 1h 47min ago | 09:01 |
| shardy | Tengu: Ok maybe dtantsur can offer some ideas on what to check if the nodes aren't visible to nova | 09:02 |
| *** mrch has joined #tripleo | 09:02 | |
| dtantsur | shardy, Tengu, we even have a huge guide on that https://docs.openstack.org/ironic/latest/admin/troubleshooting.html#nova-returns-no-valid-host-was-found-error :) | 09:02 |
| Tengu | shardy: thank you for your time | 09:02 |
| openstackgerrit | Merged openstack/tripleo-common master: Add missing OVN container service entries https://review.openstack.org/499388 | 09:03 |
| shardy | dtantsur: thanks, in this case nova hypervisor-stats returns zero nodes | 09:03 |
| Tengu | dtantsur: ah, thanks, I'll check that. | 09:03 |
| honza | I'd really appreciate any help with this *critical* CI bug https://bugs.launchpad.net/tripleo/+bug/1714361 | 09:03 |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 09:03 |
| *** jokke_ has joined #tripleo | 09:04 | |
| Tengu | aahhh | 09:05 |
| Tengu | I think I have something, dtantsur ! with the openstack baremetal node validate <UUID>: Cannot validate image information for node 8520fdcb-23a5-4d42-b301-b81c36d7cdd6 because one or more parameters are missing from its instance_info. Missing are: ['ramdisk', 'kernel', 'image_source'] // Cannot validate image information for node 8520fdcb-23a5-4d42-b301-b81c36d7cdd6 because one or more parameters are | 09:05 |
| Tengu | missing from its instance_info. Missing are: ['ramdisk', 'kernel', 'image_source'] | 09:05 |
| dtantsur | this is fine, I think | 09:06 |
| Tengu | dtantsur: shall we switch to the ironic dedicated channel, or may we continue in here? I'll update the issue if we find the issue. | 09:06 |
| dtantsur | these fields are populated by nova | 09:06 |
| dtantsur | I'm fine either way, I'm in both channels :) | 09:06 |
| Tengu | same for me :). | 09:06 |
| dtantsur | probably the ironic channel is better | 09:06 |
| Tengu | ok, let's switch then. | 09:06 |
| shardy | Thanks for the help dtantsur :) | 09:06 |
| dtantsur | np | 09:07 |
| Tengu | shardy: thank you for the help :) | 09:07 |
| jaosorior | EmilienM: thanks | 09:07 |
| *** hewbrocca_afk is now known as hewbrocca | 09:09 | |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 09:10 |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 09:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,Triaged] | 09:10 |
| mandre | shardy, marios: hi! can one of you verify my theory in https://bugs.launchpad.net/tripleo/+bug/1714905? | 09:11 |
| openstack | Launchpad bug 1714905 in tripleo "Composable scenarios BM -> Containers Upgrade jobs never deploy on BM" [Critical,Triaged] - Assigned to Martin André (mandre) | 09:11 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates master: Allow Sensu to connect to RabbitMQ cluster https://review.openstack.org/495149 | 09:12 |
| mandre | this can explain why the containers upgrade scenario jobs are failing now, but why they were passing before remains a total mystery to me | 09:12 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates master: Allow Sensu to connect to RabbitMQ cluster https://review.openstack.org/495149 | 09:14 |
| marios | mandre: looking | 09:14 |
| mandre | marios: I'm also very interested if you can explain to me how these jobs were passing before :) | 09:15 |
| marios | mandre: are you referring to /environments/composable-upgrade-steps-docker.yaml ? oh i see what you mean. we aren't ever deploying bm now? but really I have not been involved in the ci work (jistr and matbu probably better there), wrt why it was passing before.perhaps sthing changed and we were deploying bm before | 09:15 |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart-extras master: Adding scenarios tests in default regex https://review.openstack.org/491794 | 09:15 |
| *** japestinho has joined #tripleo | 09:16 | |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates master: Allow Sensu to connect to RabbitMQ cluster https://review.openstack.org/495149 | 09:17 |
| shardy | mandre: ack looking | 09:18 |
| mandre | marios, shardy, matbu: the problem is that we set "composable_scenario: scenarioXXX-multinode-containers.yaml" in the featureset file, and that is passed to the deployment for the first overcloud | 09:19 |
| shardy | marios: hey I added a comment to https://review.openstack.org/#/c/499517 - not blocking but I wonder if the fix should actually be in the step->when mangling in tripleoclient? | 09:19 |
| shardy | e.g we should evaluate the step first | 09:19 |
| *** dtantsur is now known as dtantsur|bbl | 09:21 | |
| *** cmyster has quit IRC | 09:21 | |
| marios | shardy: o/ thanks checking | 09:22 |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-quickstart-extras master: Add pre-deployment negative tests for validations https://review.openstack.org/488495 | 09:22 |
| *** tosky has joined #tripleo | 09:22 | |
| shardy | mandre: Yeah I think you're right - something got lost in translation when we added scenarios for containers I think | 09:23 |
| marios | shardy: interesting! do you mean it won't evaluate if the first condition fails? mcornea check shardy comment at https://review.openstack.org/#/c/499517 | 09:23 |
| shardy | mandre: when jistr|off first implemented this there weren't upgrade scenarios, so we just added the -e environments/docker.yaml I think | 09:23 |
| mandre | shardy: alright, I'm working on a patch to set thing straight | 09:23 |
| mcornea | marios: checking | 09:24 |
| shardy | mandre: ack sounds good, thanks! | 09:24 |
| *** akrivoka has joined #tripleo | 09:29 | |
| shardy | marios: yeah that's my assumption but I've not tested it | 09:29 |
| marios | shardy: ack i just added a comment too on the review i mean, would be nice if it does work though | 09:29 |
| *** ebarrera has quit IRC | 09:31 | |
| *** ebarrera has joined #tripleo | 09:32 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-ui master: Imported Translations from Zanata https://review.openstack.org/500265 | 09:34 |
| *** nyechiel has quit IRC | 09:35 | |
| *** ykarel_ has joined #tripleo | 09:36 | |
| *** dixiaoli has quit IRC | 09:38 | |
| *** milan has joined #tripleo | 09:38 | |
| *** ykarel|lunch has quit IRC | 09:38 | |
| *** psahoo has quit IRC | 09:38 | |
| *** gcerami has joined #tripleo | 09:39 | |
| *** cylopez has quit IRC | 09:39 | |
| *** cylopez has joined #tripleo | 09:39 | |
| *** hanish has quit IRC | 09:41 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-ui stable/pike: Imported Translations from Zanata https://review.openstack.org/500267 | 09:41 |
| openstackgerrit | Derek Higgins proposed openstack/tripleo-common master: Add health check command for ironic-pxe image https://review.openstack.org/490892 | 09:44 |
| openstackgerrit | Derek Higgins proposed openstack/tripleo-common master: Fix the path to HEALTHCHECK_SCRIPTS in healthcheck/ironic-api https://review.openstack.org/500495 | 09:44 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-quickstart master: Disable pipelining mode in Ansible config https://review.openstack.org/500497 | 09:46 |
| *** cylopez has quit IRC | 09:46 | |
| lvdombrkr | guys who can look into this isssue? | 09:51 |
| *** limao has joined #tripleo | 09:51 | |
| lvdombrkr | https://bugs.launchpad.net/tripleo/+bug/1714544 | 09:52 |
| openstack | Launchpad bug 1714544 in tripleo "cant delete stuck with network-isolation" [Medium,Triaged] | 09:52 |
| *** lucasagomes is now known as lucas-brb | 09:52 | |
| *** cylopez has joined #tripleo | 09:55 | |
| *** limao has quit IRC | 09:56 | |
| openstackgerrit | Marios Andreou proposed openstack/python-tripleoclient master: Move the step condition to be first when writing upgrade playbook https://review.openstack.org/500498 | 09:58 |
| shardy | lvdombrkr: as we discussed earlier, nova should not throw 500 errors, so I'd suggest retrying the delete and looking carefully at the log output from heat. nova and neutron - something is going badly wrong if a 500 error happens, so the next step is to figure out what by looking at the logs | 09:58 |
| marios | mcornea: posted ontop of your one ^^ | 09:58 |
| mcornea | marios: thanks, will give it a go | 09:58 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates master: Allow Sensu to connect to RabbitMQ cluster https://review.openstack.org/495149 | 10:01 |
| *** tosky has quit IRC | 10:05 | |
| lvdombrkr | shardy: right now there is no mistake about 500 error after i enable rabbitmq-server.service which was inactive | 10:07 |
| lvdombrkr | shardy : now i cant found any errors in logs | 10:08 |
| shardy | lvdombrkr: Ok, but the delete still fails? | 10:08 |
| lvdombrkr | shardy: yes it steal fails on deleting network | 10:08 |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 10:10 |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 10:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,Triaged] | 10:10 |
| lvdombrkr | shardy: i can collect for you all logs which can help somehow | 10:13 |
| lvdombrkr | shardy: but as i mentioned i see no errors | 10:14 |
| lvdombrkr | shardy: i manually have deleted subnet and network | 10:16 |
| lvdombrkr | shardy: maybe i need to delete somehing else | 10:16 |
| shardy | lvdombrkr: please paste the heat-engine log somewhere, there will be clues even if there's no error to grep for I expect | 10:19 |
| *** milan has quit IRC | 10:20 | |
| *** pgadiya has quit IRC | 10:31 | |
| openstackgerrit | Raoul Scarazzini proposed openstack/tripleo-quickstart master: Enable and document tags for all tasks https://review.openstack.org/476114 | 10:31 |
| openstackgerrit | Raoul Scarazzini proposed openstack/tripleo-quickstart-extras master: Restore tags inside the main playbook https://review.openstack.org/473491 | 10:33 |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart master: Removing ceilometer integration tests from scenario001 https://review.openstack.org/500507 | 10:34 |
| lvdombrkr | shardy: https://files.fm/u/fjxjw588 | 10:35 |
| lvdombrkr | shardy : 017-09-04 13:26:42.701 i press delete stack | 10:36 |
| *** psahoo has joined #tripleo | 10:37 | |
| *** ykarel__ has joined #tripleo | 10:38 | |
| *** pblaho has joined #tripleo | 10:38 | |
| Tengu | shardy: me again (since dtantsur|bbl is away) - I'm wondering if I'm not missing a capability for the c2c_baremetal flavor: apparently I added a "capabilities:profile='controller'" for example on the c2c_controller flavor - and the same for c2c_compute. Might be a missing "capabilities:profile='baremetal'" on the c2c_baremetal flavor produce my issue? | 10:39 |
| *** ykarel_ has quit IRC | 10:41 | |
| shardy | lvdombrkr: Perhaps I'm missing something but there are 2 minutes of logs after that timestamp, and the delete is still in progress | 10:44 |
| shardy | we need to see the part where the stack is marked DELETE_FAILED | 10:44 |
| lvdombrkr | shardy: 2017-09-04 12:33:04.454 3858 INFO heat.engine.stack [req-a986a70b-e2e5-4a97-a798-d90799078a89 - - - - -] Stack DELETE FAILED (overcloud-Networks-v4kifr4coiur): Engine went down during stack DELETE | 10:44 |
| shardy | Tengu: I don't think you were using that flavor in your deploy command? | 10:44 |
| shardy | lvdombrkr: Ok, did the undercloud run out of memory and kill heat-engine perhaps? | 10:45 |
| *** ykarel__ is now known as ykarel | 10:45 | |
| Tengu | shardy: nope, not namely at least. I suspect something calls that flavor/capability at some point, like for the first boot in order to "dd" the real image and reboot it. | 10:45 |
| *** udesale has quit IRC | 10:46 | |
| Tengu | that's really weird. I suspected the GPT disk_label to be the culprit, but after having cleaned the flavor and nodes (i.e. rollback in our git), it's not better. really strange. and a bit worrying. | 10:46 |
| shardy | Tengu: if you created that flavor, nothing will use it unless your environment specifies it | 10:46 |
| shardy | the templates default to the generic "baremetal" flavor though | 10:46 |
| Tengu | shardy: ok. so it shouldn't be an issue. | 10:47 |
| *** pgadiya has joined #tripleo | 10:47 | |
| Tengu | I actually do set the flavor name in the environment.yaml | 10:47 |
| Tengu | "just in case". | 10:47 |
| Tengu | but as said: the deploy command doesn't call that flavor namely. | 10:47 |
| shardy | Tengu: did you figure out why nova hypervisor-stats shows zero nodes? | 10:48 |
| Tengu | nope, not yet :(. | 10:48 |
| shardy | That isn't related to the flavor names passed to the deploy command | 10:48 |
| Tengu | I'm trying to dig in nova*.log, but… | 10:48 |
| *** lucas-brb is now known as lucasagomes | 10:48 | |
| Tengu | shardy: at least, the node import does work as expected: Status:SUCCESS. Errors:None for all the three. | 10:50 |
| openstackgerrit | David Sariel proposed openstack/tripleo-validations master: Add containers_sanity_check https://review.openstack.org/483403 | 10:50 |
| lvdombrkr | shardy: heat.engine as a service is up and running, and there is no memory problems during deletaion process, i just checked | 10:51 |
| lvdombrkr | shardy: CPU is 50% usage but RAM is under 30 | 10:51 |
| Tengu | shardy: and nova-compute.log does show nova has knowledge about the newly added nodes. | 10:52 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Enable TLS for rabbitmq's replication traffic https://review.openstack.org/498324 | 10:53 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Rabbitmq: Enable Erlang distribution TLS https://review.openstack.org/498325 | 10:53 |
| Tengu | seems to see that via the following resource: /usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py | 10:53 |
| lvdombrkr | shardy: but looks there is CPU usage issue http://paste.openstack.org/raw/620318/ | 10:54 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo stable/pike: Enable TLS for rabbitmq's replication traffic https://review.openstack.org/500515 | 10:54 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates stable/pike: Rabbitmq: Enable Erlang distribution TLS https://review.openstack.org/500516 | 10:54 |
| *** nyechiel has joined #tripleo | 10:55 | |
| openstackgerrit | Merged openstack/tripleo-common master: Use CephAnsibleDisksConfig when deriving HCI parameters https://review.openstack.org/498561 | 10:57 |
| chandankumar | Are we using novajoin in Tripleo? Who is the best person to contact? | 10:57 |
| Tengu | novajoin is used in case you want to enroll nodes in freeipa | 10:58 |
| Tengu | chandankumar: -^ | 10:58 |
| Tengu | (and I use it, please don't break it ;)) | 10:58 |
| chandankumar | Tengu: I am volunteering for tempest plugin split Queens community goal, Nova-join provides tempest plugin https://github.com/openstack/novajoin-tempest-plugin | 10:59 |
| chandankumar | Tengu: i am looking for volunteer to package it in RDO and integrate in puppet modules and tripleo | 10:59 |
| Tengu | chandankumar: I unfortunately don't remember whom I talked to when I got my novajoin issues. 2s, have some logs. | 11:00 |
| chandankumar | Tengu: no problem thanks for the info :-) | 11:01 |
| Tengu | chandankumar: might want to ping jaosorior I think. that's the (nice) one who helped me back in August. | 11:02 |
| chandankumar | Tengu: Sure :-) | 11:02 |
| jaosorior | chandankumar: you're looking for packaging volunteers? I'll bring it up with my team. | 11:03 |
| chandankumar | jaosorior: yup | 11:04 |
| jaosorior | chandankumar: it's a US holiday today, so it'll have to wait till tomorrow for me to poke most of them. | 11:04 |
| Tengu | :) | 11:04 |
| chandankumar | jaosorior: it is not only packaging it also involves maintaince of the tempest plugin upstream as well as integration in RDO and Upstream | 11:04 |
| chandankumar | jaosorior: i can help in getting packaged in RDO and integration in puppet modules | 11:05 |
| chandankumar | jaosorior: Thanks that will work for me | 11:05 |
| jaosorior | chandankumar: integration with puppet modules for the tempest plugin? or for novajoin itself? | 11:05 |
| shardy | http://paste.openstack.org/raw/620318/ | 11:06 |
| chandankumar | jschlueter: yup | 11:06 |
| chandankumar | jaosorior: it just involves changes in puppet-tempest and not much | 11:07 |
| shardy | lvdombrkr: that doesn't show a CPU usage issue AFAICS but you need to work out why heat-engine went down, that is not expected and it's the reason your stack delete failed | 11:07 |
| shardy | lvdombrkr: maybe check /var/log/messages for clues | 11:07 |
| chandankumar | jaosorior: for example https://github.com/openstack/puppet-tempest/commit/f173ce59ef489043b1f8f98728a98b49ae685359 | 11:07 |
| *** links has quit IRC | 11:07 | |
| shardy | a common reason is the OOM killer on nodes without swap and enough RAM | 11:07 |
| shardy | but there could be other reasons of course | 11:07 |
| jaosorior | chandankumar: I see. Thanks for the link | 11:08 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Use DeployedSSLCertificatePath for public TLS via certmonger https://review.openstack.org/500517 | 11:09 |
| *** nyechiel has quit IRC | 11:09 | |
| *** dougbtv_ has joined #tripleo | 11:09 | |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 11:10 |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 11:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,In progress] - Assigned to Arx Cruz (arxcruz) | 11:10 |
| *** shardy is now known as shardy_lunch | 11:10 | |
| *** pkovar has joined #tripleo | 11:11 | |
| Tengu | hmm. shardy_lunch I might have something: the command `openstack hypervisor show <ironic node UUID>' does show the node, with a "disabled" state. Might be an issue if nova wants to check that state ? | 11:11 |
| Tengu | shardy_lunch: duh, happy meal then ;) | 11:11 |
| *** limao has joined #tripleo | 11:13 | |
| *** pgadiya has quit IRC | 11:15 | |
| *** links has joined #tripleo | 11:20 | |
| *** jlabarre has joined #tripleo | 11:23 | |
| openstackgerrit | Merged openstack/tripleo-common master: Run fluentd container https://review.openstack.org/487038 | 11:31 |
| openstackgerrit | Merged openstack/tripleo-common master: Parse ceph_client_ansible_vars in ceph-ansible workbook https://review.openstack.org/499624 | 11:31 |
| *** snecklifter has joined #tripleo | 11:31 | |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-heat-templates master: Add subnet property to ctlplane network for server resources https://review.openstack.org/473817 | 11:38 |
| *** stendulker_ has quit IRC | 11:40 | |
| lvdombrkr | shardy: thanks, checked messages there is no any helpfull information | 11:41 |
| *** links has quit IRC | 11:42 | |
| *** lucasagomes is now known as lucas-hungry | 11:43 | |
| *** catintheroof has joined #tripleo | 11:43 | |
| *** shardy_lunch is now known as shardy | 11:44 | |
| *** shreshtha has quit IRC | 11:46 | |
| shardy | Tengu: yes openstack hypervisor list should show the ironic nodes as up, and hypervisor show should show them enabled | 11:48 |
| *** jlabarre has quit IRC | 11:49 | |
| *** jlabarre has joined #tripleo | 11:49 | |
| shardy | lvdombrkr: Ok, well there are really only two reasons the engine would go down, either the OOM killer killed it (which should be visible in the syslog) or somebody stopped/restarted the service | 11:50 |
| *** jkilpatr has joined #tripleo | 11:51 | |
| *** jpena is now known as jpena|lunch | 11:51 | |
| jaosorior | flaper87: hey, I saw that you have some patches for deploying openshift with tripleo. How's that going? | 11:51 |
| jaosorior | flaper87: I tried doing it manually, but it failed, probably cause cause of firewall issues. what do you do for that? | 11:52 |
| Tengu | shardy: ok, so that might explain why nova doesn't see them properly. any way to know/understand why they are "disabled" ? | 11:52 |
| flaper87 | jaosorior: getting any specific error I can look at? | 11:52 |
| *** dtantsur|bbl is now known as dtantsur | 11:52 | |
| flaper87 | lemme get some links for ya' | 11:52 |
| jaosorior | flaper87: http://paste.openstack.org/show/620326/ | 11:53 |
| *** links has joined #tripleo | 11:56 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates master: Remove package if service stopped and disabled https://review.openstack.org/479886 | 11:56 |
| shardy | Tengu: I would expect it to be related to the node state in ironic or connectivity between nova and ironic - nova logs should show the reason but I don't have an exact search string for you | 11:56 |
| lvdombrkr | shardy:thanks, interesting that this issue begin only after i deployod overcloud with network-isolation | 11:56 |
| *** jlabarre has quit IRC | 11:56 | |
| lvdombrkr | before i deploy/delete ocercloud few times without any problems | 11:57 |
| shardy | lvdombrkr: well that will use more memory - how much ram does your undercloud have? | 11:57 |
| jaosorior | flaper87: although.... that's weird, it should enable the ports by default: r_openshift_master_firewall_enabled: "{{ os_firewall_enabled | default(True) }}" | 11:57 |
| *** adarazs is now known as adarazs_lunch | 11:57 | |
| Tengu | shardy: hmm. I didn't find anything in nova logs that might point to a reason :'(. | 11:57 |
| Tengu | and ironic show them as "available", without "maintenance" flag nor reason. | 11:57 |
| shardy | Tengu: does openstack hypervisor list show them all as "up" ? | 11:57 |
| Tengu | shardy: state is "up" yes | 11:58 |
| shardy | hrm | 11:58 |
| flaper87 | jaosorior: sorry, slow connection today. You gotta love moving houses. | 11:58 |
| shardy | Tengu: Ok, probably time to go back to the ironic experts then I think, sorry I'm out of ideas atm | 11:58 |
| flaper87 | jaosorior: https://review.openstack.org/#/c/494470/16/extraconfig/services/openshift-master.yaml | 11:58 |
| flaper87 | jaosorior: all the configs I'm setting are in line 111 | 11:58 |
| Tengu | shardy: no problem. I'm trying to understand the links between services, that's why I go back and forth between ironic and nova, poking around in here ;). | 11:59 |
| flaper87 | jaosorior: thanks for your comments, btw. I've got answers for them. Gimme a couple of mins and I'll get back to oyu on that reviwe | 11:59 |
| Tengu | shardy: I'll poke a bit more on the ironic channel then. thanks again for the hints. | 11:59 |
| jaosorior | flaper87: what is openshift_use_dnsmasq used for? | 11:59 |
| jaosorior | flaper87: and enable_excluders | 11:59 |
| *** jlabarre has joined #tripleo | 12:00 | |
| flaper87 | jaosorior: that's to enable dnsmasq in the openshift cluster or not. Openshift depends on a working DNS(ish). It was possible to disable it in 3.5 but it must be enabled in 3.6 | 12:00 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Convert step to integer in when statement for upgrade tasks https://review.openstack.org/499540 | 12:01 |
| flaper87 | jaosorior: the excluders are some extra validations and checks that the installer does | 12:01 |
| jaosorior | ok | 12:01 |
| lvdombrkr | shardy: 8GB, but i monitor ir during deletion process there are 2-3 gb always unused | 12:02 |
| *** gkadam has quit IRC | 12:02 | |
| jaosorior | flaper87: I was using a prettys imilar config. Except that I use heat-admin instead of tripleo-admin. wasn't setting containerized to true, dns nor the excluders | 12:02 |
| jaosorior | flaper87: and I'm trying to do it in the ctlplane network for simplicity. | 12:03 |
| flaper87 | jaosorior: may I ask what you're doing this for? | 12:03 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates master: Remove package if service stopped and disabled https://review.openstack.org/479886 | 12:03 |
| shardy | lvdombrkr: IME 8G is only just enough, 12G is the quickstart default - do you have any swap configured? | 12:04 |
| jaosorior | flaper87: setting it up in a tripleo cluster? For fun :D . If this doesn't work out I'll just do oc cluster up and continue what I was actually gonna try out. | 12:04 |
| *** dparkes has quit IRC | 12:05 | |
| flaper87 | jaosorior: awesome! If you have any other input on the patch, lemme know. Also, you could prolly give my patch a go :P | 12:05 |
| flaper87 | if you want to help testing | 12:05 |
| jaosorior | flaper87: ultimately I want to set up ViaQ (Common Logging) and start investigating what's needed to get TripleO to log there. | 12:05 |
| shardy | lvdombrkr: also please double check grep -i killed /var/log/messages - I think it's unlikely you have much (if any) free memory unless you've tuned the undercloud from its default settings | 12:05 |
| jaosorior | flaper87: well, calling ansible-playbook manually seemed like the easier choice... but I think I'll try your patch, that way I could contribute to it (hopefully). let me give it a try | 12:06 |
| lvdombrkr | shardy: yes 10GB swap | 12:07 |
| jaosorior | flaper87: can you explain to me what you're doing with network manager and what line 89 is doing? from here https://review.openstack.org/#/c/494470/16/extraconfig/services/openshift-master.yaml | 12:08 |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 12:10 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 12:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,In progress] - Assigned to Arx Cruz (arxcruz) | 12:10 |
| lvdombrkr | shardy: grep messages http://paste.openstack.org/raw/620329/ | 12:10 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Stop hardcoding host's config volume path https://review.openstack.org/494163 | 12:13 |
| honza | I'd really appreciate any help or pointers on this *critical* ci bug https://bugs.launchpad.net/tripleo/+bug/1714361 cc apetrich | 12:17 |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 12:17 |
| *** dpawar has quit IRC | 12:18 | |
| jaosorior | honza: have you checked what version is being hosted on RDO ? | 12:18 |
| honza | jaosorior: if i'm not mistaken, instack-undercloud uses puppet-mistral to build it from source | 12:19 |
| honza | jaosorior: rdo seems to have the correct code in the latest build | 12:19 |
| jaosorior | honza: right, but the issue is with the mistral package itself, not the puppet manifests | 12:20 |
| honza | jaosorior: comment #3 in the bug report indicates that rdo is in fact correct | 12:20 |
| jaosorior | honza: puppet-* doesn't build packages from source :/ ; instack-undercloud will just download whatever comes from the configured repository | 12:20 |
| jaosorior | honza: oh, alright | 12:20 |
| honza | jaosorior: sorry, my bad, i couldn't find any references to mistral rpms in the gate logs so i assumed it was built from source | 12:21 |
| jaosorior | honza: seems like the images are around 11 days old https://dashboards.rdoproject.org/rdo-dev so that might be an issue. | 12:21 |
| apetrich | jaosorior, honza as far as I talked to ci sshnaidm told me that we had no promotions in 10 or so days | 12:22 |
| apetrich | jaosorior, aye. | 12:22 |
| apetrich | jaosorior, but what bugs me is that the mistral in https://trunk.rdoproject.org/centos7-master/current-passed-ci/ has the patch | 12:22 |
| honza | jaosorior: apetrich: so... should we bug rdo people about this? | 12:22 |
| sshnaidm | apetrich, can you post the log to the job again please? | 12:23 |
| shardy | yeah please link a recent job failing due to the old mistral | 12:23 |
| apetrich | https://bugs.launchpad.net/tripleo/+bug/1714361 | 12:24 |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 12:24 |
| honza | http://logs.openstack.org/08/469608/13/check/gate-tripleo-ci-centos-7-undercloud-oooq/80a6ddc/ | 12:24 |
| apetrich | shardy, sshnaidm ^ | 12:24 |
| shardy | openstack-mistral-api-5.0.0-1.el7.noarch | 12:24 |
| shardy | http://logs.openstack.org/08/469608/13/check/gate-tripleo-ci-centos-7-undercloud-oooq/80a6ddc/logs/rpm-qa.txt.gz | 12:24 |
| shardy | So it's not using the current-tripleo version AFAICS? | 12:25 |
| shardy | https://trunk.rdoproject.org/centos7-master/current-tripleo/openstack-mistral-api-5.0.0-0.20170823122251.1a8837b.el7.centos.noarch.rpm | 12:25 |
| apetrich | shardy, good point | 12:25 |
| apetrich | shardy, I missed that. let me check that one | 12:25 |
| shardy | IIRC we hit issues like this last cycle just after stable/ocata branched | 12:25 |
| sshnaidm | apetrich, it's consistent repo http://logs.openstack.org/08/469608/13/check/gate-tripleo-ci-centos-7-undercloud-oooq/80a6ddc/logs/undercloud/etc/yum.repos.d/delorean.repo.txt.gz | 12:26 |
| sshnaidm | apetrich, trunk.rdoproject.org/centos7/consistent/delorean.repo | 12:26 |
| jaosorior | flaper87: is there a reason why heat-admin is not used and you call the workflow to create tripleo-admin? | 12:26 |
| * shardy finds the thread | 12:26 | |
| *** dr_gogeta86 has quit IRC | 12:26 | |
| apetrich | shardy, consistent has openstack-mistral-all-5.0.0-0.20170823122251.1a8837b.el7.centos.noarch.rpm | 12:27 |
| *** lucas-hungry is now known as lucasagomes | 12:27 | |
| shardy | apetrich: well that's not what's installed on the undercloud? | 12:27 |
| *** psahoo has quit IRC | 12:27 | |
| apetrich | I think that comes from the images | 12:27 |
| apetrich | anyway until a promotion and new images I don't think we will be able to solve this | 12:28 |
| flaper87 | jaosorior: it doesn't exist in that CI job. It's created by the validation workflow, I believe | 12:28 |
| *** udesale has joined #tripleo | 12:28 | |
| shardy | apetrich: No it comes from here: | 12:29 |
| sshnaidm | shardy, in current-tripleo I think we have only tripleo related: http://logs.openstack.org/08/469608/13/check/gate-tripleo-ci-centos-7-undercloud-oooq/80a6ddc/logs/undercloud/etc/yum.repos.d/delorean-current.repo.txt.gz | 12:29 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements https://review.openstack.org/488148 | 12:29 |
| shardy | http://mirror.dfw.rax.openstack.org:8080/buildlogs.centos/centos/7/cloud/x86_64/openstack-pike/ | 12:29 |
| honza | apetrich: what's needed for a promotion? | 12:29 |
| shardy | instead of from here: | 12:29 |
| shardy | http://mirror.dfw.rax.openstack.org:8080/rdo/centos7/21/a2/21a252272f31950155e4a8086561214b2743c1ae_774c92ec/ | 12:29 |
| honza | huh | 12:29 |
| shardy | http://logs.openstack.org/08/469608/13/check/gate-tripleo-ci-centos-7-undercloud-oooq/80a6ddc/logs/undercloud/etc/yum.repos.d/ | 12:29 |
| jaosorior | flaper87: right, it is. I just wanted to know if there was a reason to use that instead of heat-admin, which should already exist in the overcloud ndoe. | 12:29 |
| jaosorior | *node | 12:29 |
| shardy | like I said, we had the same issue last cycle with the puppet modules | 12:29 |
| shardy | for a while the nvr of stable/pike looks newer than master | 12:30 |
| honza | shardy: what was the solution last time? | 12:30 |
| shardy | we cut a new release of master for all the things | 12:30 |
| shardy | let me find the openstack-dev thread | 12:30 |
| apetrich | shardy, thanks! | 12:30 |
| shardy | it was kind of an unsatisfactory solution, but it worked around the issue | 12:30 |
| *** garyk1 has quit IRC | 12:31 | |
| openstackgerrit | Ana Krivokapic proposed openstack/tripleo-ui master: Fix plan descriptions on plan cards https://review.openstack.org/496381 | 12:32 |
| openstackgerrit | Merged openstack/tripleo-common master: Drop MANIFEST.in - it's not needed by pbr https://review.openstack.org/478976 | 12:32 |
| apetrich | honza, promotion is an integration test of all the recent packages working together. Currently there's an issue with nova (I don't know the lp for it though) that prevents it from passing | 12:32 |
| honza | apetrich: thanks, that's helpful | 12:32 |
| shardy | apetrich: http://lists.openstack.org/pipermail/openstack-dev/2017-March/113529.html has details | 12:34 |
| shardy | I suspect it's the same problem again | 12:34 |
| shardy | it's probably not specific to mistral | 12:34 |
| apetrich | shardy, thanks again | 12:34 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Drop MANIFEST.in - it's not needed by pbr https://review.openstack.org/479266 | 12:35 |
| apetrich | I assumed so too | 12:35 |
| *** skramaja has quit IRC | 12:35 | |
| jaosorior | sshnaidm: is this expected http://logs.openstack.org/15/500515/1/check/gate-tripleo-ci-centos-7-containers-multinode/8323f0a/logs/undercloud/home/jenkins/overcloud_prep_containers.log.txt.gz#_2017-09-04_11_47_05 ? | 12:36 |
| *** adarazs_lunch is now known as adarazs | 12:36 | |
| jaosorior | mandre: ^^ | 12:36 |
| sshnaidm | jaosorior, not really | 12:37 |
| jaosorior | sshnaidm: just saw that in a stable/pike job :/ | 12:37 |
| sshnaidm | jaosorior, I saw those errors before too, seems like it happens sometimes | 12:37 |
| sshnaidm | jaosorior, but today it happens a lot | 12:38 |
| jaosorior | sure seems like it | 12:39 |
| *** dr_gogeta86 has joined #tripleo | 12:41 | |
| *** dr_gogeta86 has joined #tripleo | 12:41 | |
| sshnaidm | http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22404%20Client%20Error%3A%20Not%20Found%5C%22%20%20AND%20build_name%3A*tripleo-ci-*%20AND%20tags%3Aconsole%20AND%20voting%3A1%20AND%20build_status%3AFAILURE | 12:41 |
| sshnaidm | jaosorior, hmm.. I see --namespace docker.io/tripleoupstream, but we have a reverse proxy for this | 12:42 |
| mandre | jaosorior: was the job running with https://review.openstack.org/#/c/493391/? | 12:43 |
| mandre | sshnaidm, jaosorior: there was a lot of changes in this area recently and it's not over yet | 12:44 |
| jaosorior | mandre: I backported those today | 12:46 |
| *** ratailor has quit IRC | 12:48 | |
| sshnaidm | mandre, jaosorior there were changes by stevebaker that I don't understand.. but seems like we stopped to use proxy since then | 12:49 |
| sshnaidm | docker.io shouldn't be used in CI | 12:49 |
| mandre | jaosorior: https://review.openstack.org/#/c/493391/ merged this morning, I think you have the commit, so something must be missing in the stable/pike branch | 12:49 |
| mandre | sshnaidm: infra maintains a proxy, so even if the url shows docker.io we're fetching from infra rather than from docker.io directly | 12:51 |
| sshnaidm | mandre, it's *reverse* proxy, you should mention it, not docker.io | 12:51 |
| mandre | sshnaidm: but the duplicate docker.io thingy is a bug | 12:51 |
| sshnaidm | mandre, it's not transparent, it's kind of mirror host | 12:52 |
| mandre | sshnaidm: right, we set it up in docker config | 12:52 |
| sshnaidm | mandre, you need to set it as a namespace, not "http_proxy" var | 12:53 |
| sshnaidm | pabelanger is welcome to explain to be sure (or to correct me) | 12:54 |
| sshnaidm | pabelanger, is docker proxy a reverse proxy? should we set it as namespace of docker or just "http_proxy" var? | 12:55 |
| mandre | sshnaidm: the proxy is setup correctly in jaosorior's ci job, http://logs.openstack.org/15/500515/1/check/gate-tripleo-ci-centos-7-containers-multinode/8323f0a/logs/undercloud/etc/docker/daemon.json.txt.gz | 12:55 |
| openstackgerrit | Janki Chhatbar proposed openstack/python-tripleoclient master: Add neutron_driver value to prepare command https://review.openstack.org/498743 | 12:55 |
| sshnaidm | mandre, hmm.. I think registry mirror is for docker registries only, not sure if it works with apache reverse proxy which is used in infra. But not to confuse you, I'd like pabelanger to clarify this. | 12:57 |
| *** dparkes has joined #tripleo | 12:57 | |
| shardy | mandre: should we be setting --block-registry in /etc/sysconfig/docker to ensure it only uses the local mirror? | 12:58 |
| shardy | I did that locally because I found small misconfiguration of the local mirror would result in docker silently pulling from docker.io | 12:58 |
| shardy | blocking docker.io enabled debugging those issues much faster IIRC | 12:58 |
| lvdombrkr | shardy: can i delete all stack from sql? | 12:59 |
| mandre | shardy: I don't think we can because in this case we won't be able to populate the registry mirror IIUC | 12:59 |
| shardy | lvdombrkr: yes but that won't help explain why your heat engine got restarted | 12:59 |
| sshnaidm | mandre, blocking settings are for client, not for mirror | 13:00 |
| *** limao_ has joined #tripleo | 13:00 | |
| sshnaidm | mandre, mirror will download it once if it needs to | 13:00 |
| shardy | mandre: but can't we just pull direct from the infra cache? | 13:00 |
| *** jaganathan has quit IRC | 13:01 | |
| shardy | my local docker registry is configured as a pull-through cache, then I don't need the one on the undercloud, or any access to docker.io | 13:01 |
| *** limao has quit IRC | 13:01 | |
| *** jpena|lunch is now known as jpena | 13:01 | |
| shardy | I was assuming the infra setup was similar but it sounds like it's not | 13:01 |
| mandre | sshnaidm: hmm ok, I'll need a fresh look at how all the docker mirror thing works | 13:01 |
| sshnaidm | mandre, shardy folks, I very doubt we use docker registry in infra.. let's clarify it before | 13:02 |
| sshnaidm | oh, us is in holidays today | 13:02 |
| *** leitan has joined #tripleo | 13:03 | |
| sshnaidm | mandre, shardy docker mirror and proxy have completely different settings | 13:03 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Expose account/container/object worker count https://review.openstack.org/497917 | 13:04 |
| openstackgerrit | Janki Chhatbar proposed openstack/tripleo-heat-templates master: Mount folders and log file https://review.openstack.org/500097 | 13:05 |
| *** tosky has joined #tripleo | 13:05 | |
| lvdombrkr | shardy: can you give me some step how do do that? | 13:06 |
| *** oidgar has quit IRC | 13:09 | |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 13:10 |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 13:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,In progress] - Assigned to Arx Cruz (arxcruz) | 13:10 |
| pabelanger | morning | 13:16 |
| pabelanger | http://status.openstack.org/elastic-recheck/#1710533 | 13:16 |
| pabelanger | looks like docker.io is broken on stable/pike | 13:16 |
| *** akrivoka has quit IRC | 13:17 | |
| *** pdeore has quit IRC | 13:18 | |
| pabelanger | mwhahaha: EmilienM: ^ | 13:19 |
| pabelanger | sshnaidm: mandre: just catching up on backscoll give me a second | 13:19 |
| shardy | Could it be related to https://review.openstack.org/#/c/493391/ perhaps? | 13:20 |
| * shardy looks for missing pike tripleoclient patches | 13:21 | |
| sshnaidm | pabelanger, so should we configured like so? http://logs.openstack.org/15/500515/1/check/gate-tripleo-ci-centos-7-containers-multinode/8323f0a/logs/undercloud/etc/docker/daemon.json.txt.gz | 13:21 |
| pabelanger | shardy: possible | 13:21 |
| pabelanger | shardy: https://review.openstack.org/500438/ looks related | 13:22 |
| pabelanger | sshnaidm: yes, that is correct | 13:22 |
| pabelanger | shardy: so, if 493391 is the reason stable/pike broken, there is a gap in testing. Specifically, tripleo-quickstart / triple-quickstart-extras is branchless. So, at least 1 job from each stable branch should be gating on those repos too, so avoid broken code on master from breaking other branches | 13:24 |
| *** oidgar has joined #tripleo | 13:25 | |
| pabelanger | sshnaidm: mandre: it looks like we are looking at the same issue: docker.io/docker.io/tripleoupstream/centos-binary-aodh-api:latest isn't valid syntax. It should be docker.io/tripleoupstream/centos-binary-aodh-api:latest, something merged and broke stable/pike. See 500438 and 493391 above | 13:27 |
| *** akrivoka has joined #tripleo | 13:29 | |
| *** sshnaidm is now known as sshnaidm|mtg | 13:31 | |
| shardy | pabelanger: yeah looks like https://review.openstack.org/#/c/500438/ should fix it? Agree re the t-q-e test coverage | 13:32 |
| pabelanger | Ya, http://logs.openstack.org/38/500438/1/check/gate-tripleo-ci-centos-7-containers-multinode/76fb56c/logs/undercloud/home/jenkins/overcloud_prep_containers.log.txt.gz looks to be correct | 13:32 |
| *** pdeore has joined #tripleo | 13:33 | |
| *** garyk has quit IRC | 13:33 | |
| *** limao_ has quit IRC | 13:35 | |
| mandre | nice combo on https://review.openstack.org/#/c/500438/ | 13:35 |
| mandre | thanks pabelanger | 13:35 |
| *** oidgar has quit IRC | 13:37 | |
| *** dsariel has quit IRC | 13:37 | |
| jaosorior | flaper87: ran the deployment but it seems stuck in 2017-09-04 13:12:15Z [overcloud.AllNodesDeploySteps.WorkflowTasks_Step2_Execution]: CREATE_IN_PROGRESS state changed | 13:41 |
| jaosorior | flaper87: while the engine doesn't seem to be executing anything and is also stuck waiting | 13:41 |
| jaosorior | flaper87: oh, there actually was an error. But for some reason mistral didn't fail | 13:44 |
| jaosorior | flaper87: http://paste.openstack.org/show/620339/ | 13:45 |
| openstackgerrit | Martin André proposed openstack/tripleo-quickstart-extras master: Use different variables for deploy and upgrade scenarios https://review.openstack.org/500546 | 13:46 |
| *** hjensas has quit IRC | 13:46 | |
| *** jlabarre has quit IRC | 13:47 | |
| Tengu | shardy: finally we (well, dtantsur to be honnest) found my issue: nova-compute was indeed deactivated because of the pike nova::compute::consecutive_build_service_disable_threshold thingy - setting up the option doesn't re-activate nova-compute obviously, and I had to do that by hand after an upgrade of the undercloud in order to apply the change. | 13:48 |
| Tengu | shardy: meaning: I can update my issue and say it's a duplicated one. | 13:48 |
| *** jlabarre has joined #tripleo | 13:49 | |
| *** masco has quit IRC | 13:49 | |
| *** oidgar has joined #tripleo | 13:51 | |
| shardy | Tengu: ack, good to hear you found a solution | 13:51 |
| openstackgerrit | Martin André proposed openstack/tripleo-quickstart master: Use different variables for deploy and upgrade scenarios https://review.openstack.org/500552 | 13:53 |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-ui master: Add new lint rule to check for presence of license headers https://review.openstack.org/500553 | 13:53 |
| shardy | mcornea: Hey https://review.openstack.org/#/c/500498/ doesn't have the same error does it? | 13:53 |
| shardy | the error is about a j2 delimiter, not an undefined variable? | 13:54 |
| shardy | marios: ^^ | 13:54 |
| shardy | it should be pretty easy to test manually, e.g when step == 1 and undef.foo == "blah" | 13:54 |
| mcornea | shardy: it's the same that I've been intially seeing - 'dict object' has no attribute 'stdout'\n\nThe error , related to '{{ovs_version.stdout}}', where ovs_version is undefined | 13:55 |
| *** pdeore has quit IRC | 13:55 | |
| shardy | Well the error now says when statements should not include jinja2 templating delimiters ? | 13:56 |
| shardy | which isn't quite the same, unless I'm missing something? | 13:56 |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-validations master: Fix name of controller token validation https://review.openstack.org/500555 | 13:56 |
| shardy | mcornea: ah, actually it's not rendering the undefined variable | 13:56 |
| *** shreshtha has joined #tripleo | 13:57 | |
| shardy | mcornea, marios: Ok sorry ignore me then :) | 13:57 |
| *** jlabarre has quit IRC | 13:57 | |
| mcornea | shardy: I think the jinja2 templating delimiter message is just a warning, the failure comes on the next line | 13:57 |
| shardy | mcornea: ah, yeah I see now, thanks for clarifying | 13:58 |
| shardy | I think it'd work if it was two simple variable comparisons vs a dictionary lookup | 13:58 |
| *** jlabarre has joined #tripleo | 13:59 | |
| Tengu | small note: the following commands just don't take into account what nova sees from ironic: "openstack overcloud profiles list" and "openstack overcloud profiles match <options>" - that might be misleading if you rely on those two in order to ensure you actually can deploy the overcloud (or do the update). | 14:01 |
| shardy | Tengu: yeah it just compares flavors with nodes in ironic, perhaps we could add a validation which checks the hypervisor stats count against the expected number of nodes | 14:01 |
| openstackgerrit | Merged openstack/tripleo-ui master: Imported Translations from Zanata https://review.openstack.org/500265 | 14:01 |
| * shardy thought we had that already... | 14:02 | |
| Tengu | would be good if those two commands could query nova regarding the nodes - for example like "openstack hypervisor stats show". | 14:02 |
| openstackgerrit | Martin André proposed openstack/tripleo-quickstart-extras master: Add scenarios environment files to prepare cmd https://review.openstack.org/500364 | 14:02 |
| openstackgerrit | Martin André proposed openstack/tripleo-quickstart-extras master: Use different variables for deploy and upgrade scenarios https://review.openstack.org/500546 | 14:02 |
| *** ansmith has joined #tripleo | 14:02 | |
| Tengu | darn… now I get another kind of error "can't found enough hosts". | 14:02 |
| shardy | https://github.com/openstack/tripleo-common/blob/master/workbooks/validations.yaml#L765 | 14:04 |
| shardy | so we do already validate hypervisor stats, would be good to understand why that didn't help in this case | 14:04 |
| Tengu | indeed :/ | 14:04 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates master: Upgrade CI test - never merge https://review.openstack.org/461000 | 14:04 |
| *** udesale has quit IRC | 14:05 | |
| shardy | Tengu: feel free to raise a bug if you can provide steps to reproduce, thanks! | 14:05 |
| Tengu | trying to understand what's happening now again -.- | 14:05 |
| Tengu | raahhh. again X(. | 14:07 |
| *** udesale has joined #tripleo | 14:07 | |
| Tengu | shardy: I'm doomed, that's pretty sure. Openstack doesn't like I took a week off, and make me pay. | 14:07 |
| dtantsur | heh | 14:07 |
| EmilienM | Hello | 14:08 |
| dtantsur | Tengu: as to the "profile" commands, that's a good feature request (check nova hypervisors), I can give it a shot, if you file a bug for it | 14:08 |
| Tengu | I fill it right now :) | 14:08 |
| Tengu | dtantsur: in tripleo project? | 14:08 |
| dtantsur | Tengu: yep, these commands are part of tripleoclient | 14:08 |
| Tengu | ok | 14:08 |
| dtantsur | morning EmilienM | 14:10 |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 14:10 |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 14:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,In progress] - Assigned to Arx Cruz (arxcruz) | 14:10 |
| *** tzumainn has joined #tripleo | 14:11 | |
| *** dbellant has joined #tripleo | 14:13 | |
| Tengu | dtantsur: https://bugs.launchpad.net/tripleo/+bug/1714965 (and shardy ) | 14:15 |
| openstack | Launchpad bug 1714965 in tripleo ""profiles" commands shoule check nova hypervisors" [Undecided,New] | 14:15 |
| dtantsur | thanks Tengu | 14:15 |
| Tengu | :) | 14:16 |
| EmilienM | shardy: what's up? | 14:16 |
| *** nyechiel has joined #tripleo | 14:16 | |
| *** aufi has quit IRC | 14:18 | |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-ui master: Add new lint rule to check for presence of license headers https://review.openstack.org/500553 | 14:18 |
| EmilienM | mandre: hey, where are we? | 14:18 |
| *** dparkes has quit IRC | 14:19 | |
| *** gbarros has joined #tripleo | 14:21 | |
| lvdombrkr | folks, any idea about this error when deploying overcloud? i have external network same as managment network | 14:21 |
| lvdombrkr | http://paste.openstack.org/raw/620343/ | 14:21 |
| honza | apetrich: would you mind updating the bug with the latest info? i'm not sure i fully understand it yet | 14:21 |
| *** ykarel has quit IRC | 14:21 | |
| honza | apetrich: ugh, please ignore, email fail, i didn't see the updates from an hour ago :( | 14:22 |
| apetrich | honza, I did or I think I did. In a very typical way I might have forgotten to hit enter | 14:22 |
| honza | apetrich: sorry, my bad | 14:22 |
| apetrich | honza, :) | 14:22 |
| apetrich | honza, you scared me for a second :) | 14:22 |
| openstackgerrit | Janki Chhatbar proposed openstack/tripleo-heat-templates master: Add param to configure snat mechanism https://review.openstack.org/493861 | 14:23 |
| *** marrusl has quit IRC | 14:24 | |
| EmilienM | 404 Client Error: Not Found ("no such id: docker.io/docker.io/tripleoupstream/centos-binary-aodh-api:latest") | 14:27 |
| EmilienM | pabelanger: is that what you talked about? | 14:27 |
| EmilienM | oh, https://review.openstack.org/#/c/500438/ is the fix | 14:27 |
| EmilienM | shardy, pabelanger: re: coverage - yes I agree t-q-e needs more covergae. I'll add it | 14:28 |
| shardy | EmilienM: hey yeah hopefully that pike backport will fix it | 14:28 |
| marios | shardy: ack thanks - would have been a nicer way to fix it if it worked that way and mcornea ++ for the quick testing today | 14:28 |
| pabelanger | EmilienM: ya, testing stable branches on quickstart projects will help that | 14:29 |
| honza | what is nvr? | 14:29 |
| EmilienM | marios: hello, do we have any update on https://bugs.launchpad.net/tripleo/+bug/1713832 ? I'm going to look how often do we hit that today | 14:29 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 14:29 |
| EmilienM | marios: I want to remove my -2 on your patch but before make sure things are stable now | 14:30 |
| marios | hey EmilienM no update yet. it hit twice today. i did look at recent swift/zaqar commits but didn't spot something obvious yet | 14:30 |
| EmilienM | ok | 14:31 |
| EmilienM | I'll keep investigating at logstash | 14:31 |
| marios | EmilienM: i mean here http://status.openstack.org/elastic-recheck/#1713832 | 14:31 |
| EmilienM | marios: and sorry again... really I know it sucks | 14:31 |
| EmilienM | but we have had many issues lately | 14:31 |
| marios | EmilienM: yea OK, and ack on the ^^ and your resply on the upstream list, i will respond there (was waiting to see if i came up with something for the bug this week too) | 14:31 |
| shardy | honza: the name-version.release of the RPM | 14:32 |
| EmilienM | marios: when things are more stable, I'm happy to backport it if really needed | 14:32 |
| honza | shardy: ah, thanks | 14:32 |
| shardy | honza: since we branched pike, the version of the RPM is newer than trunk :( | 14:32 |
| honza | right | 14:33 |
| marios | EmilienM: it is my job to try and get it landed and make the case for it, and it is your job to make sure things are working. it just wasn't clear to me why that patch but i can understand 'validations' were involved | 14:33 |
| shardy | we had the exact same issue last cycle, but for some reason folks weren't super keen to fix it, e.g by mandating an automated initial release right after we branch | 14:33 |
| shardy | I still don't get why we don't just do that tbh | 14:33 |
| marios | EmilienM: lets talk again about it later this week thanks? | 14:33 |
| shardy | but it requires agreement from the release team | 14:33 |
| *** mrch has quit IRC | 14:33 | |
| marios | EmilienM: 'make sure things are working' well thats everyone job, but i mean in the given discussion for the ongoing release for P and ci issues especially | 14:35 |
| EmilienM | marios: ok | 14:35 |
| *** udesale has quit IRC | 14:35 | |
| *** agurenko has quit IRC | 14:35 | |
| *** ccamacho has quit IRC | 14:36 | |
| *** ccamacho has joined #tripleo | 14:37 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/python-tripleoclient master: [WIP] Report node availability from "overcloud profiles list" https://review.openstack.org/500570 | 14:39 |
| openstackgerrit | Ricardo Noriega proposed openstack/tripleo-heat-templates master: Environment to deploy BGPVPN with Bagpipe in a unique file https://review.openstack.org/500571 | 14:39 |
| *** pdeore has joined #tripleo | 14:39 | |
| *** links has quit IRC | 14:40 | |
| Tengu | dtantsur: you're light-speed on that issue :) | 14:41 |
| *** pdeore has quit IRC | 14:43 | |
| dtantsur | Tengu: heh, I'm not done yet, playing around with it right now | 14:45 |
| Tengu | dtantsur: :) | 14:45 |
| openstackgerrit | Ricardo Noriega proposed openstack/tripleo-heat-templates master: Fix NeutronServicePlugins parameter to match ODL L3 feature https://review.openstack.org/500573 | 14:48 |
| openstackgerrit | Ricardo Noriega proposed openstack/tripleo-heat-templates master: Environment to deploy BGPVPN with Bagpipe in a unique file https://review.openstack.org/500571 | 14:49 |
| openstackgerrit | Ricardo Noriega proposed openstack/tripleo-heat-templates master: Environment to deploy BGPVPN with Bagpipe in a unique file https://review.openstack.org/500571 | 14:50 |
| *** dparkes has joined #tripleo | 14:51 | |
| *** sshnaidm|mtg is now known as sshnaidm | 14:53 | |
| *** udesale has joined #tripleo | 14:54 | |
| *** udesale has quit IRC | 14:58 | |
| *** ykarel has joined #tripleo | 15:04 | |
| EmilienM | mandre: adding alert on https://bugs.launchpad.net/tripleo/+bug/1714905 | 15:04 |
| openstack | Launchpad bug 1714905 in tripleo "Composable scenarios BM -> Containers Upgrade jobs never deploy on BM" [Critical,In progress] - Assigned to Martin André (mandre) | 15:04 |
| EmilienM | mandre: thanks for the help | 15:04 |
| openstackgerrit | Ricardo Noriega proposed openstack/tripleo-heat-templates stable/pike: Fix NeutronServicePlugins parameter to match ODL L3 feature https://review.openstack.org/500579 | 15:07 |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 15:10 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714905 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 15:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,In progress] - Assigned to Arx Cruz (arxcruz) | 15:10 |
| openstack | Launchpad bug 1714905 in tripleo "Composable scenarios BM -> Containers Upgrade jobs never deploy on BM" [Critical,In progress] - Assigned to Martin André (mandre) | 15:10 |
| openstackgerrit | Ricardo Noriega proposed openstack/tripleo-heat-templates master: Environment to deploy BGPVPN with Bagpipe in a unique file https://review.openstack.org/500571 | 15:11 |
| lvdombrkr | folks, if i want create network isolation to test tls, did i need add also network-isolation.yaml to deploy command or there will be enought with network-environment.yaml | 15:12 |
| *** ebarrera has quit IRC | 15:15 | |
| EmilienM | should we backport https://review.openstack.org/#/c/494163/ ? (ping mandre, shardy) | 15:16 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common stable/pike: Parse ceph_client_ansible_vars in ceph-ansible workbook https://review.openstack.org/500580 | 15:17 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Add CephExternal role for ceph-ansible https://review.openstack.org/500581 | 15:18 |
| *** cylopez has quit IRC | 15:18 | |
| EmilienM | same question for https://review.openstack.org/#/c/487038/ ( I know dprince was against adding new containers at this stage, but we might consider exceptions) | 15:19 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common stable/pike: Add missing OVN container service entries https://review.openstack.org/500582 | 15:19 |
| *** janki has quit IRC | 15:21 | |
| *** gbarros has quit IRC | 15:21 | |
| openstackgerrit | Merged openstack/tripleo-common stable/pike: Upload remove default for pull_source https://review.openstack.org/500438 | 15:22 |
| Tengu | hmmm. weird. | 15:23 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: scenario001-container: run autoscaling tests as well https://review.openstack.org/500250 | 15:23 |
| Tengu | my nodes got an Instance UUID according to `openstack baremetal node list' - but suddenly, "Message: No valid host was found." - although all lights are green in both profiles and nova. | 15:23 |
| openstackgerrit | Feodor Tersin proposed openstack/os-net-config master: This patch adds initial support for the Contrail vRouter interface https://review.openstack.org/492492 | 15:24 |
| EmilienM | numans: do we have progress on https://review.openstack.org/#/c/494293/ ? | 15:24 |
| numans | EmilienM, no. I am just clueless why the test is blocking. I am trying to run the test locally in my setup | 15:26 |
| numans | EmilienM, I guess you would have noticed, the test ran and was successful. The next thing I would have expected in the log was the test report and then ostestr cleanup called. I will keep you updated | 15:27 |
| *** garyk has joined #tripleo | 15:28 | |
| EmilienM | numans: ok | 15:28 |
| EmilienM | numans: it seems like it's timeouting | 15:28 |
| *** iranzo has quit IRC | 15:28 | |
| numans | EmilienM, that's right. | 15:28 |
| *** dbecker_ has joined #tripleo | 15:30 | |
| *** gbarros has joined #tripleo | 15:31 | |
| EmilienM | ok https://review.openstack.org/500438 has merged | 15:31 |
| EmilienM | pabelanger: once it's built in delorean, we would do recheck on stable/pike and it should work | 15:31 |
| *** noslzzp has joined #tripleo | 15:33 | |
| *** shreshtha has quit IRC | 15:33 | |
| EmilienM | ok we can do recheck on stable/pike now | 15:34 |
| EmilienM | it's built | 15:34 |
| Tengu | o_O duh… apparently novajoin-server sets wrong permissions on its logfiles… | 15:34 |
| *** shreshtha has joined #tripleo | 15:34 | |
| *** dbecker has quit IRC | 15:34 | |
| *** gkadam has joined #tripleo | 15:35 | |
| pabelanger | EmilienM: ok | 15:35 |
| *** gbarros has quit IRC | 15:35 | |
| EmilienM | jaosorior: ^ | 15:36 |
| jaosorior | Tengu: does it? | 15:37 |
| jaosorior | Tengu: what permissions does it set? | 15:37 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Set mode for ansible written files https://review.openstack.org/500585 | 15:38 |
| *** ccamacho has quit IRC | 15:38 | |
| *** garyk has quit IRC | 15:39 | |
| Tengu | jaosorior: the log files belong to nova, while the service runs as novajoin | 15:39 |
| Tengu | jaosorior: that issue caused a really high load on my undercloud vm - correcting the rights solved the load issue. Like, drastically. | 15:40 |
| Tengu | (divided by two) | 15:40 |
| *** dbellant has quit IRC | 15:40 | |
| Tengu | jaosorior: would be good to ensure either python-novajoin package or puppet receipt managing that service applies the right ownership and rights to the log directory/files :) | 15:41 |
| jaosorior | Tengu: investigating | 15:42 |
| Tengu | thank you! | 15:43 |
| *** aufi has joined #tripleo | 15:43 | |
| jaosorior | Tengu: well, it is supposed to use the novajoin user https://github.com/rdo-packages/novajoin-distgit/blob/rpm-master/python-novajoin.spec#L176 | 15:44 |
| Tengu | hmmm. | 15:44 |
| Tengu | interesting. | 15:44 |
| Tengu | I did an `openstack undercloud upgrade' with the new trunk sources for pike. | 15:45 |
| jaosorior | Tengu: and we don't set those permissions in puppet. | 15:45 |
| Tengu | but I can't ensure the rights weren't already broken before that action today - I was a week off, and my colleagues worked on the openstack. | 15:45 |
| Tengu | jaosorior: might be an idea to juste ensure the rights are OK? | 15:45 |
| Tengu | \o/ with a lower load, the deploy now seems to find its node ! | 15:46 |
| Tengu | yeaaahhhh | 15:46 |
| EmilienM | jaosorior: please see my comment: https://review.openstack.org/#/c/500507 | 15:47 |
| EmilienM | jaosorior: I would avoid any +2 regarding ignoring tests in the middle of a release | 15:48 |
| EmilienM | we actually need more tests | 15:48 |
| jaosorior | EmilienM: thought ceilometer API was deprecated. | 15:48 |
| EmilienM | jaosorior: not related at all | 15:49 |
| EmilienM | jaosorior: we run autoscaling tests without ceilometer-api | 15:49 |
| jaosorior | EmilienM: ah | 15:49 |
| jaosorior | well shit | 15:49 |
| jaosorior | ok, thanks for the -2 then | 15:49 |
| EmilienM | jaosorior: please read the bug report | 15:49 |
| EmilienM | jaosorior: we need to be super careful in CI reviews | 15:49 |
| EmilienM | if we stop testing autoscaling, we're ignoring a bunch of services | 15:49 |
| jaosorior | Tengu: is it possible for you to check the spec file that was used for installing novajoin? | 15:49 |
| Tengu | dtantsur: shardy for information: the patch works fine, and after finding the huge load cause, all the undercloud API seems to work as expected, returning information in time in order to let the deploy flow. | 15:50 |
| Tengu | jaosorior: where may I find that? | 15:50 |
| dtantsur | nice! | 15:50 |
| Tengu | jaosorior: note: today, I did the upgrade, and it upgraded packages as well. But as said: not sure it broke the rights this morning, or if it was broken earlier. | 15:50 |
| jaosorior | Tengu: rpm2cpio myrpm.src.rpm | cpio -civ '*.spec' | 15:50 |
| *** marios has quit IRC | 15:51 | |
| jaosorior | Tengu: I think it might have been broken earlier. | 15:51 |
| Tengu | duh… I had to run a yum clean all, jaosorior :/. sorry. | 15:51 |
| Tengu | (disk space issues on the undercloud vm) | 15:51 |
| jaosorior | Tengu: we used to use the nova user, and switched to having a novajoin user | 15:51 |
| Tengu | jaosorior: ah! that's probably the root cause. nova on Ocata maybe ? | 15:51 |
| jaosorior | Tengu: yep | 15:52 |
| Tengu | that's it | 15:52 |
| Tengu | jaosorior: we upgraded the undercloud from ocata to pike | 15:52 |
| Tengu | the processus should check/ensure the logs belong to the right user then | 15:52 |
| jaosorior | Tengu: funky the rpm didn't change the permissions. | 15:52 |
| Tengu | either with puppet or something else. | 15:52 |
| Tengu | yup, and uncool. | 15:52 |
| *** dparkes has quit IRC | 15:54 | |
| Tengu | jaosorior: care to check that? I might not be the only one in that situation. | 15:54 |
| Tengu | jaosorior: ah!! the specfile takes care of the directory, not its content! | 15:54 |
| EmilienM | shardy: if you're still around, can you look https://bugs.launchpad.net/tripleo/+bug/1714857 ? maybe a user mistake | 15:55 |
| openstack | Launchpad bug 1714857 in tripleo "Network isolation fails for me on stable/pike" [Medium,Triaged] | 15:55 |
| Tengu | jaosorior: -> if a logfile already exists, the service is restarted after the updated, changing its running user, but the logfile still belongs to the old one | 15:55 |
| Tengu | jaosorior: mystery solved I think? | 15:55 |
| Tengu | jaosorior: and as the logrotate has a weekly occurrence, we might keep a broken novajoin for a week. | 15:56 |
| Tengu | jaosorior: more over, the size does matter in the logrotate - as the logfile isn't written anymore, it won't rotate. | 15:56 |
| Tengu | jaosorior: do you want an issue on launchpad/tripleo ? :) | 15:57 |
| jaosorior | Tengu: yeah please | 15:57 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Stop hardcoding host's config volume path https://review.openstack.org/500590 | 15:57 |
| Tengu | jaosorior: writing it with all the details. | 15:58 |
| *** lucasagomes is now known as lucas-afk | 15:58 | |
| *** dtantsur is now known as dtantsur|bbl | 16:01 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Allow upgrade tasks to run when looping through steps https://review.openstack.org/499517 | 16:02 |
| *** pblaho has quit IRC | 16:02 | |
| *** egonzalez has quit IRC | 16:02 | |
| Tengu | jaosorior: https://bugs.launchpad.net/tripleo/+bug/1714991 here you are | 16:03 |
| openstack | Launchpad bug 1714991 in tripleo "[pike] novajoin user" [Undecided,New] | 16:03 |
| lvdombrkr | folks, trying deploy overcloud with network-isolation and get this error, any ideas? http://paste.openstack.org/raw/620356/ | 16:03 |
| Tengu | lvdombrkr: you might want to check `openstack stack failures list overcloud --long' for more information | 16:04 |
| jaosorior | Tengu: thanks! | 16:04 |
| Tengu | jaosorior: I think you have all the information? feel free to comment for more. I have to leave now | 16:04 |
| shardy | EmilienM: ack, looking | 16:05 |
| Tengu | overcloud controller is "active". That's a good news. | 16:05 |
| jaosorior | Tengu: have a good one! | 16:05 |
| Tengu | :) thanks. cu! | 16:05 |
| *** catintheroof has quit IRC | 16:05 | |
| *** catintheroof has joined #tripleo | 16:06 | |
| *** jlinkes has quit IRC | 16:06 | |
| *** catintheroof has quit IRC | 16:06 | |
| shardy | EmilienM: I think I see the issue, will comment | 16:06 |
| lvdombrkr | Tengu: thanks but there is not much helpful information i think http://paste.openstack.org/raw/620357/ | 16:06 |
| *** tzumainn has quit IRC | 16:07 | |
| *** yprokule has quit IRC | 16:08 | |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714905 | 16:10 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 16:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,In progress] - Assigned to Arx Cruz (arxcruz) | 16:10 |
| openstack | Launchpad bug 1714905 in tripleo "Composable scenarios BM -> Containers Upgrade jobs never deploy on BM" [Critical,In progress] - Assigned to Martin André (mandre) | 16:10 |
| EmilienM | shardy: thanks | 16:11 |
| *** nyechiel has quit IRC | 16:11 | |
| *** mcornea has quit IRC | 16:11 | |
| *** anshul has quit IRC | 16:13 | |
| openstackgerrit | Or Idgar proposed openstack/puppet-tripleo master: Adapting Octavia api to work with containerized environment https://review.openstack.org/500593 | 16:15 |
| *** Goneri has joined #tripleo | 16:18 | |
| rnoriega | hello guys, quick question! tripleo quickstart take the images from current-passed-ci?? or from current-tripleo-rdo?? | 16:18 |
| *** mcornea has joined #tripleo | 16:19 | |
| rnoriega | just trying to figure out why one of my patches is not included installing master with oooq :-) | 16:19 |
| shardy | rnoriega: see https://bugs.launchpad.net/tripleo/+bug/1714361 | 16:19 |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 16:19 |
| mandre | EmilienM: regarding https://review.openstack.org/#/c/494163/, I feel like it shouldn't be backported, it's not strictly needed for the containers to work (but makes our life easier if we need to debug the docker-puppet step) | 16:19 |
| mandre | EmilienM: if you look at the review we've even waited for master to be open to development again :D | 16:21 |
| *** mcornea has quit IRC | 16:21 | |
| mandre | EmilienM: that said, it's simple enough that it's relatively safe to backport | 16:22 |
| rnoriega | shardy, that might be it. Thanks. However, I see the last promotion of RDO is from a week ago... | 16:24 |
| shardy | rnoriega: Yeah I don't think the promotion helps if you have delorean repos overlaying a pike-testing repo | 16:24 |
| shardy | there are discussions about rdo level workarounds, and perhaps we can raise the issue (again) with the release team tomorrow | 16:25 |
| shardy | last cycle we had the discussion and there wasn't great consensus around fixing it by just automating a new tag on master right after we branch | 16:25 |
| EmilienM | mandre: I backported https://review.openstack.org/#/c/494163/ to make our life easier if we have to debug stable version | 16:25 |
| *** oidgar has quit IRC | 16:26 | |
| EmilienM | mandre: thx for the feedback | 16:26 |
| rnoriega | shardy, I need to clarify the pipeline in my head...but images in tripleo are build from delorean RPMs, right? | 16:26 |
| mandre | EmilienM: have opstool asked for an exception regarding fluentd container? afaik the containerized fluent service is not yet ready so I wouldn't add the image to pike | 16:26 |
| shardy | rnoriega: yeah but we also pull in some deps from delorean-pike-testing | 16:26 |
| EmilienM | mandre: they didn't so no backport for now | 16:26 |
| shardy | rnoriega: right now that contains RPMs which are newer than the ones built by delorean from trunk | 16:27 |
| shardy | :( | 16:27 |
| *** cylopez has joined #tripleo | 16:27 | |
| rnoriega | shardy, ewwww | 16:27 |
| shardy | Yeah, I linked the ML thread from the exact same saga we had a few months ago | 16:27 |
| shardy | it's a recurring problem | 16:28 |
| rnoriega | shardy, I see... thanks! | 16:28 |
| rnoriega | shardy, anyway, the rpm installed by oooq for tht is: openstack-tripleo-heat-templates-7.0.0-0.20170824035705.c9c4d7e.el7.centos.noarch | 16:28 |
| rnoriega | shardy, which is the same as in here: https://trunk.rdoproject.org/centos7/current-passed-ci/ | 16:28 |
| lvdombrkr | folks, my overcloud deployment fails on this,any ideas? http://paste.openstack.org/raw/620359/ | 16:29 |
| shardy | rnoriega: ah, perhaps a different issue then | 16:29 |
| *** cschwede_ has quit IRC | 16:31 | |
| lvdombrkr | i dont understood why script is waching in for intreface ir [ERROR] No interfaces defined in config: /etc/os-net-config/config.json | 16:33 |
| *** hewbrocca is now known as hewbrocca_afk | 16:36 | |
| lvdombrkr | ' | 16:38 |
| lvdombrkr | ? | 16:38 |
| *** aufi has quit IRC | 16:38 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Allow upgrade tasks to run when looping through steps https://review.openstack.org/500596 | 16:41 |
| *** stendulker has joined #tripleo | 16:42 | |
| *** jpich has quit IRC | 16:45 | |
| *** tesseract has quit IRC | 16:48 | |
| lvdombrkr | folks, anyone can help, my deploymaint fails on http://paste.openstack.org/raw/620359/ , i dont understood what info script need from /etc/os-net-config/config.json | 16:48 |
| *** gkadam has quit IRC | 16:50 | |
| *** gkadam has joined #tripleo | 16:51 | |
| *** ebarrera has joined #tripleo | 16:53 | |
| *** dtantsur|bbl is now known as dtantsur | 16:54 | |
| *** jfrancoa has quit IRC | 16:54 | |
| *** milan has joined #tripleo | 16:55 | |
| pabelanger | EmilienM: mwhahaha: sshnaidm: something isn't right here: http://logs.openstack.org/33/499133/3/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq/91cedb7/console.html | 16:56 |
| pabelanger | that jobs shouldn't be hitting mirror.centos.org | 16:56 |
| EmilienM | true | 16:56 |
| EmilienM | let me look | 16:57 |
| EmilienM | we need to check NODEPOOL_CENTOS_MIRROR | 16:57 |
| *** derekh has quit IRC | 16:57 | |
| EmilienM | pabelanger: it's odd we don't have all logs | 16:58 |
| EmilienM | but we have http://logs.openstack.org/33/499133/3/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq/91cedb7/logs/subnode-2/home/jenkins/repo_setup.log.txt.gz | 16:58 |
| EmilienM | pabelanger: see the script, do you see anything wrong? http://logs.openstack.org/33/499133/3/check/gate-tripleo-ci-centos-7-scenario001-multinode-oooq/91cedb7/logs/subnode-2/home/jenkins/repo_setup.sh.txt.gz | 16:59 |
| pabelanger | looking | 17:00 |
| EmilienM | ah | 17:01 |
| EmilienM | sudo yum install -y yum-plugin-priorities; | 17:01 |
| EmilienM | sshnaidm: ^ I think we should install this package AFTER having the mirrors | 17:01 |
| pabelanger | yup | 17:01 |
| *** stendulker_ has joined #tripleo | 17:01 | |
| sshnaidm | yum install -y yum-plugin-priorities | 17:01 |
| pabelanger | that will have to be moved after yum repos are configured | 17:01 |
| *** Goneri has quit IRC | 17:02 | |
| *** stendulker has quit IRC | 17:02 | |
| pabelanger | EmilienM: in zuulv3, we'd move that logic to a pre-run. Which will setup mirrors even before your jobs start | 17:03 |
| pabelanger | EmilienM: we'll go over it at PTG | 17:03 |
| sshnaidm | pabelanger, that would be nice | 17:04 |
| sshnaidm | EmilienM, yeah, will move it.. | 17:04 |
| *** garyk has joined #tripleo | 17:05 | |
| *** stendulker_ has quit IRC | 17:07 | |
| *** stendulker_ has joined #tripleo | 17:07 | |
| *** gkadam has quit IRC | 17:07 | |
| *** catintheroof has joined #tripleo | 17:07 | |
| EmilienM | pabelanger: nice | 17:07 |
| EmilienM | sshnaidm: thanks! | 17:07 |
| EmilienM | sshnaidm: I can help as well, let me know | 17:08 |
| *** fzdarsky is now known as fzdarsky|afk | 17:08 | |
| sshnaidm | pabelanger, but resolving error is still an issue, it shouldn't be there | 17:09 |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714905 | 17:10 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 17:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,In progress] - Assigned to Arx Cruz (arxcruz) | 17:10 |
| openstack | Launchpad bug 1714905 in tripleo "Composable scenarios BM -> Containers Upgrade jobs never deploy on BM" [Critical,In progress] - Assigned to Martin André (mandre) | 17:10 |
| lvdombrkr | folks, anyone can help, my deploymaint fails on http://paste.openstack.org/raw/620359/ , i dont understood what info script need from /etc/os-net-config/config.json | 17:10 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart master: Install yum priorities after repo set up https://review.openstack.org/500601 | 17:10 |
| sshnaidm | pabelanger, EmilienM ^^ | 17:10 |
| EmilienM | sshnaidm: that was fast :) | 17:10 |
| *** garyk has quit IRC | 17:11 | |
| EmilienM | sshnaidm: I'm SOO happy we don't branch oooq and oooq-extras :) | 17:11 |
| openstackgerrit | Raoul Scarazzini proposed openstack/tripleo-quickstart-extras master: Add the ability to load custom yaml(s) https://review.openstack.org/500602 | 17:11 |
| *** jkilpatr has quit IRC | 17:17 | |
| *** jpena is now known as jpena|off | 17:19 | |
| *** sshnaidm is now known as sshnaidm|off | 17:22 | |
| *** tosky has quit IRC | 17:30 | |
| *** shreshtha has quit IRC | 17:32 | |
| *** psachin has quit IRC | 17:33 | |
| *** cylopez has quit IRC | 17:33 | |
| *** dpawar has joined #tripleo | 17:33 | |
| *** achadha has joined #tripleo | 17:34 | |
| *** ramishra has quit IRC | 17:38 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common stable/ocata: Remove machine-id from image https://review.openstack.org/493685 | 17:47 |
| *** stendulker_ has quit IRC | 17:49 | |
| *** gcerami has quit IRC | 17:49 | |
| Tengu | lvdombrkr: hm, I hoped it would show more information. Sorry, no idea on that one :/ | 17:52 |
| Tengu | lvdombrkr: at least it points something with the network configuration, you might want to check the logs on the instances maybe. | 17:52 |
| Tengu | i.e. missing setting, or something like that. | 17:52 |
| lvdombrkr | Tengu: thanks, can you look into this : http://paste.openstack.org/raw/620359/ | 17:54 |
| lvdombrkr | Tengu: i dont understood why script is taking information from /etc/os-net-config/config | 17:55 |
| *** paramite has quit IRC | 17:55 | |
| *** tosky has joined #tripleo | 17:55 | |
| lvdombrkr | Tengu: and if i go into /etc/os-net-config/config i see undercloud interface config not undercloud | 17:56 |
| Tengu | lvdombrkr: you might have overlooked network interfaces, or maybe they aren't properly recognized? Sorry, I'm also a beginner with all of that, and not that used to the network debugging part (that part was working as expected for me) | 17:57 |
| *** rcernin has joined #tripleo | 17:57 | |
| *** dtantsur is now known as dtantsur|afk | 17:57 | |
| lvdombrkr | Tengu: thanks anyway | 17:59 |
| openstackgerrit | Merged openstack/tripleo-validations master: Add separate fail-if-no-hosts plugin https://review.openstack.org/500067 | 18:00 |
| openstackgerrit | Merged openstack/tripleo-validations master: Updated from global requirements https://review.openstack.org/500041 | 18:00 |
| *** dpawar has quit IRC | 18:02 | |
| *** milan has quit IRC | 18:04 | |
| *** achadha has quit IRC | 18:06 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo master: metadata.json: prepare for 8.0.0 release (queens) https://review.openstack.org/500607 | 18:09 |
| *** jlabarre has quit IRC | 18:09 | |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714905 | 18:10 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 18:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,In progress] - Assigned to Arx Cruz (arxcruz) | 18:10 |
| openstack | Launchpad bug 1714905 in tripleo "Composable scenarios BM -> Containers Upgrade jobs never deploy on BM" [Critical,In progress] - Assigned to Martin André (mandre) | 18:10 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo stable/pike: Prepare 7.4.0 (pike-rc2) https://review.openstack.org/500608 | 18:10 |
| openstackgerrit | Numan Siddique proposed openstack/tripleo-quickstart master: DO NOT REVIEW : TESTING ONLY https://review.openstack.org/500609 | 18:10 |
| *** jlabarre has joined #tripleo | 18:11 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-ui stable/pike: Prepare 7.4.0 (pike-rc2) https://review.openstack.org/500610 | 18:12 |
| *** ykarel has quit IRC | 18:14 | |
| openstackgerrit | Martin André proposed openstack/tripleo-quickstart master: Use different variables for deploy and upgrade scenarios https://review.openstack.org/500552 | 18:18 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Change references from nsx_v3 to nsx https://review.openstack.org/498142 | 18:25 |
| openstackgerrit | Merged openstack/tripleo-common master: Add clustercheck healthcheck https://review.openstack.org/497468 | 18:25 |
| *** kristaps_ has joined #tripleo | 18:28 | |
| EmilienM | bandini: do you need https://review.openstack.org/#/c/497468/ in stable/pike? | 18:29 |
| *** StevenK_ has joined #tripleo | 18:33 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-ui master: Prepare tripleo-ui for queens cycle https://review.openstack.org/500619 | 18:35 |
| *** kbyrne_ has joined #tripleo | 18:36 | |
| *** leifmadsen_ has joined #tripleo | 18:36 | |
| *** saneax___ has joined #tripleo | 18:37 | |
| *** kbyrne has quit IRC | 18:37 | |
| *** kambiz has quit IRC | 18:37 | |
| *** saneax has quit IRC | 18:37 | |
| *** lvdombrkr has quit IRC | 18:37 | |
| *** leifmadsen has quit IRC | 18:37 | |
| *** StevenK has quit IRC | 18:37 | |
| *** kbyrne_ is now known as kbyrne | 18:37 | |
| *** kambiz has joined #tripleo | 18:37 | |
| *** artom_ has joined #tripleo | 18:39 | |
| *** ecerquei_ has joined #tripleo | 18:39 | |
| *** bandini has quit IRC | 18:40 | |
| *** trown has quit IRC | 18:42 | |
| *** migi has quit IRC | 18:42 | |
| *** shadower has quit IRC | 18:42 | |
| *** ecerquei has quit IRC | 18:42 | |
| *** bandini has joined #tripleo | 18:42 | |
| *** trown has joined #tripleo | 18:42 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common stable/pike: Add missing OVN container service entries https://review.openstack.org/500582 | 18:42 |
| *** migi has joined #tripleo | 18:42 | |
| *** shadower has joined #tripleo | 18:42 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common stable/pike: Parse ceph_client_ansible_vars in ceph-ansible workbook https://review.openstack.org/500580 | 18:43 |
| *** artom has quit IRC | 18:46 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common stable/pike: Make curl healthchecks work with internal TLS https://review.openstack.org/500149 | 18:48 |
| EmilienM | pabelanger: please review https://review.openstack.org/#/c/499821/ | 18:49 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Add support for Dell EMC VMAX Manila Backend https://review.openstack.org/499199 | 18:50 |
| pabelanger | EmilienM: oh, ya. I was to work on that. | 18:50 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Add support for Dell EMC Isilon Manila backend https://review.openstack.org/499195 | 18:50 |
| pabelanger | EmilienM: will push up an update shortly | 18:50 |
| *** jprovazn has quit IRC | 18:55 | |
| EmilienM | pabelanger: thx | 18:57 |
| *** gbarros has joined #tripleo | 19:08 | |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714905 | 19:10 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 19:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,In progress] - Assigned to Arx Cruz (arxcruz) | 19:10 |
| openstack | Launchpad bug 1714905 in tripleo "Composable scenarios BM -> Containers Upgrade jobs never deploy on BM" [Critical,In progress] - Assigned to Martin André (mandre) | 19:10 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Upgrade CI test on stable/pike - never merge https://review.openstack.org/500625 | 19:10 |
| *** gbarros has quit IRC | 19:11 | |
| *** pkovar has quit IRC | 19:17 | |
| *** pabelanger has quit IRC | 19:17 | |
| *** pabelanger has joined #tripleo | 19:17 | |
| *** shardy has quit IRC | 19:18 | |
| openstackgerrit | Merged openstack/python-tripleoclient stable/pike: Filter out disabled services from prepare command https://review.openstack.org/500225 | 19:22 |
| *** akrivoka has quit IRC | 19:24 | |
| *** catintheroof has quit IRC | 19:26 | |
| *** artom_ has quit IRC | 19:27 | |
| *** artom_ has joined #tripleo | 19:28 | |
| *** akrivoka has joined #tripleo | 19:30 | |
| *** catintheroof has joined #tripleo | 19:30 | |
| *** gbarros has joined #tripleo | 19:32 | |
| *** gbarros has quit IRC | 19:33 | |
| pabelanger | EmilienM: need a break, stepping away for a bit, but will get the query today. | 19:35 |
| pabelanger | EmilienM: good news is, I don't think I see any new failures in elastic-recheck | 19:35 |
| pabelanger | now that stable/pike is working again, things should just merge | 19:35 |
| EmilienM | pabelanger: same thing, doing a break now | 19:36 |
| EmilienM | pabelanger: yeah, thx | 19:36 |
| EmilienM | pabelanger: we still need to figure out some alerts in progress | 19:36 |
| EmilienM | but folks are away now | 19:36 |
| pabelanger | ya | 19:36 |
| *** dbecker_ has quit IRC | 19:37 | |
| *** dparkes has joined #tripleo | 19:37 | |
| EmilienM | jaosorior: https://bugs.launchpad.net/tripleo/+bug/1714991 triaged and assigned to you | 19:42 |
| openstack | Launchpad bug 1714991 in tripleo "[pike] novajoin user" [High,Triaged] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 19:42 |
| EmilienM | stevebaker: I think mandre sent you an update about status of upgrade jobs, I'll be online on my evening to see where we are now | 19:44 |
| *** ebarrera has quit IRC | 19:45 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/pike: Remove tacker from containers scenario001 https://review.openstack.org/500166 | 19:48 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/pike: Separate config_volume for ringbuilder https://review.openstack.org/499457 | 20:03 |
| openstackgerrit | Merged openstack/tripleo-common stable/pike: Updated from global requirements https://review.openstack.org/498181 | 20:03 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/pike: Updated from global requirements https://review.openstack.org/498182 | 20:03 |
| *** dougbtv_ has quit IRC | 20:07 | |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714905 | 20:10 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 20:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,In progress] - Assigned to Arx Cruz (arxcruz) | 20:10 |
| openstack | Launchpad bug 1714905 in tripleo "Composable scenarios BM -> Containers Upgrade jobs never deploy on BM" [Critical,In progress] - Assigned to Martin André (mandre) | 20:10 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/pike: Use list_concat in place of yaql https://review.openstack.org/500172 | 20:14 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/pike: Fix containerized zaqar-api db_sync https://review.openstack.org/500413 | 20:15 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/pike: Configure Zaqar trust notifier https://review.openstack.org/499894 | 20:15 |
| *** ebarrera has joined #tripleo | 20:32 | |
| *** dsariel has joined #tripleo | 20:35 | |
| stevebaker | EmilienM: ok | 20:36 |
| *** rcernin has quit IRC | 20:45 | |
| *** catintheroof has quit IRC | 20:46 | |
| mandre | EmilienM, stevebaker: this looks much better than before, https://review.openstack.org/#/c/500552/, now gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv seems to report a real error | 20:51 |
| *** dougbtv_ has joined #tripleo | 20:58 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/pike: Manually set healthchecks for _cron services https://review.openstack.org/500147 | 20:59 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/pike: Stop hardcoding host's config volume path https://review.openstack.org/500590 | 20:59 |
| openstackgerrit | Merged openstack/puppet-tripleo stable/pike: Prepare 7.4.0 (pike-rc2) https://review.openstack.org/500608 | 20:59 |
| openstackgerrit | Merged openstack/puppet-tripleo master: metadata.json: prepare for 8.0.0 release (queens) https://review.openstack.org/500607 | 20:59 |
| *** dsariel has quit IRC | 21:05 | |
| *** florianf has quit IRC | 21:07 | |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714905 | 21:10 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 21:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,In progress] - Assigned to Arx Cruz (arxcruz) | 21:10 |
| openstack | Launchpad bug 1714905 in tripleo "Composable scenarios BM -> Containers Upgrade jobs never deploy on BM" [Critical,In progress] - Assigned to Martin André (mandre) | 21:10 |
| *** leitan has quit IRC | 21:19 | |
| *** pcaruana has quit IRC | 21:20 | |
| *** leitan has joined #tripleo | 21:20 | |
| *** leitan has quit IRC | 21:20 | |
| *** leitan has joined #tripleo | 21:20 | |
| *** leitan has quit IRC | 21:20 | |
| *** leitan has joined #tripleo | 21:21 | |
| *** artom_ has quit IRC | 21:22 | |
| *** artom_ has joined #tripleo | 21:22 | |
| openstackgerrit | Merged openstack/tripleo-ui stable/pike: Prepare 7.4.0 (pike-rc2) https://review.openstack.org/500610 | 21:23 |
| openstackgerrit | Merged openstack/tripleo-ui master: Prepare tripleo-ui for queens cycle https://review.openstack.org/500619 | 21:23 |
| pabelanger | EmilienM: how can I find out why [overcloud.AllNodesDeploySteps.ControllerDeployment_Step3]: CREATE_FAILED CREATE aborted | 21:23 |
| pabelanger | http://logs.openstack.org/80/500580/2/check/gate-tripleo-ci-centos-7-scenario004-multinode-oooq-container/c6b2779/logs/undercloud/home/jenkins/overcloud_deploy.log.txt.gz | 21:23 |
| pabelanger | I've seen that a few times now | 21:23 |
| *** leitan has quit IRC | 21:25 | |
| EmilienM | pabelanger: let me look | 21:31 |
| EmilienM | pabelanger: it's tricky, but let me show | 21:31 |
| EmilienM | pabelanger: well, in fact it's a timeout | 21:32 |
| EmilienM | CREATE_FAILED Create timed out | 21:32 |
| EmilienM | I think we need to create a create with both [overcloud.AllNodesDeploySteps.ControllerDeployment_Step3]: CREATE_FAILED Resource CREATE failed: Operation cancelled and [overcloud]: CREATE_FAILED Create timed out | 21:32 |
| EmilienM | and call the bug "step3 timeouts" | 21:32 |
| EmilienM | pabelanger: ^ | 21:33 |
| EmilienM | mandre: good to know Depends-on works :D | 21:33 |
| pabelanger | EmilienM: okay, I've seen that more then once. Let me get search logstash | 21:33 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Switch manila-share to pacemaker version in scenario004/containers https://review.openstack.org/500314 | 21:34 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Remove bgp-vpn from scenario004-multinode-containers https://review.openstack.org/499626 | 21:35 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo stable/pike: Use TLS proxy for Redis' internal TLS https://review.openstack.org/499995 | 21:35 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Add clustercheck to service list for scenarios https://review.openstack.org/499133 | 21:35 |
| EmilienM | pabelanger: ok | 21:35 |
| EmilienM | pabelanger: I'm offline a bit now | 21:35 |
| pabelanger | EmilienM: message:"[overcloud.AllNodesDeploySteps.ControllerDeployment_Step3]: CREATE_FAILED Resource CREATE failed: Operation cancelled" AND tags:"console" AND voting:1 | 21:36 |
| pabelanger | 22 hits | 21:36 |
| pabelanger | since 09-02 | 21:36 |
| EmilienM | yeah | 21:37 |
| EmilienM | let's create a query with that | 21:37 |
| pabelanger | limited to gate-tripleo-ci-centos-7-scenario004-multinode-oooq-container | 21:37 |
| pabelanger | k, I'll create a bug now | 21:37 |
| *** dougbtv_ has quit IRC | 21:38 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates stable/pike: Persist containerized services httpd logs https://review.openstack.org/499235 | 21:38 |
| *** jlabarre has quit IRC | 21:38 | |
| EmilienM | pabelanger: I'm back in few hours | 21:38 |
| pabelanger | EmilienM: k | 21:39 |
| pabelanger | bug 1715029 | 21:40 |
| openstack | bug 1715029 in tripleo "[overcloud.AllNodesDeploySteps.ControllerDeployment_Step3]: CREATE_FAILED Resource CREATE failed: Operation cancelled" [Undecided,New] https://launchpad.net/bugs/1715029 | 21:40 |
| pabelanger | creating e-r query now | 21:40 |
| *** jlabarre has joined #tripleo | 21:41 | |
| *** jtomasek has quit IRC | 21:44 | |
| *** dougbtv_ has joined #tripleo | 21:49 | |
| *** achadha has joined #tripleo | 22:08 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 22:10 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714905 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 22:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,In progress] - Assigned to Arx Cruz (arxcruz) | 22:10 |
| openstack | Launchpad bug 1714905 in tripleo "Composable scenarios BM -> Containers Upgrade jobs never deploy on BM" [Critical,In progress] - Assigned to Martin André (mandre) | 22:10 |
| stevebaker | EmilienM: Is this job meant to be ocata->pike? because I think it is currently pike-bm -> pike-containers gate-tripleo-ci-centos-7-scenario001-multinode-oooq-container-upgrades-nv | 22:13 |
| *** akrivoka has quit IRC | 22:16 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-heat-templates master: Updated from global requirements https://review.openstack.org/488148 | 22:19 |
| pabelanger | EmilienM: http://status.openstack.org/elastic-recheck/#1715029 | 22:20 |
| pabelanger | large uptick | 22:20 |
| pabelanger | good news, its recent regression | 22:21 |
| pabelanger | so, should be easy to track down | 22:21 |
| *** achadha has quit IRC | 22:26 | |
| *** jlabarre has quit IRC | 22:28 | |
| *** jlabarre has joined #tripleo | 22:29 | |
| EmilienM | stevebaker: good question, I think on master it deploys pike to containers | 22:41 |
| EmilienM | stevebaker: but I'm more interested by stable/pike runtimes | 22:42 |
| EmilienM | pabelanger: looking | 22:42 |
| EmilienM | pabelanger: mhh ok | 22:42 |
| EmilienM | stevebaker: we need to figure why https://review.openstack.org/#/c/500625/ fails (upgrade jobs) | 22:46 |
| stevebaker | EmilienM: yep, for the scenario001 job its because mongod service stop fails because there is no mongod service | 22:47 |
| stevebaker | EmilienM: mongodb is disabled by default now for bm, but still enabled by default for containers, so I think that is one thing that needs a fix | 22:48 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-common master: Define ceph image in overcloud_containers.yaml.j2 https://review.openstack.org/499822 | 22:52 |
| EmilienM | stevebaker: ok, good to know thx | 22:53 |
| EmilienM | stevebaker: I would like to get gate-tripleo-ci-centos-7-containers-multinode-upgrades-nv first if possible | 22:53 |
| EmilienM | since it's the most basic one | 22:53 |
| EmilienM | I'll try to look asap | 22:53 |
| *** leitan has joined #tripleo | 22:53 | |
| stevebaker | I'm seeing lots of db sync errors in that one | 22:54 |
| EmilienM | ouch | 22:54 |
| EmilienM | I go for run and I'll look | 22:54 |
| stevebaker | ok | 22:54 |
| EmilienM | let me know if you find simething | 22:54 |
| *** ebarrera has quit IRC | 22:55 | |
| pabelanger | EmilienM: okay, EOD for me. Have an early flight in the morning. I'll read backscroll in the morning | 22:57 |
| *** rwsu has joined #tripleo | 22:59 | |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1713832 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714361 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714755 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1714905 | 23:10 |
| openstack | Launchpad bug 1713832 in tripleo "Object PUT failed for zaqar_subscription" [Critical,In progress] - Assigned to Marios Andreou (marios-b) | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1714361 in tripleo "mistral on gates seems old and does not have the required patchs" [Critical,Triaged] - Assigned to Adriano Petrich (apetrich) | 23:10 |
| openstack | Launchpad bug 1714755 in tripleo "CI: telemetry tempest tests fail for scenario 001 in periodic jobs" [Critical,In progress] - Assigned to Arx Cruz (arxcruz) | 23:10 |
| openstack | Launchpad bug 1714905 in tripleo "Composable scenarios BM -> Containers Upgrade jobs never deploy on BM" [Critical,In progress] - Assigned to Martin André (mandre) | 23:10 |
| *** tosky has quit IRC | 23:23 | |
| *** lblanchard has joined #tripleo | 23:42 | |
| *** leitan has quit IRC | 23:47 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-heat-templates master: Containerized mongodb, disable by default, fix upgrade https://review.openstack.org/500646 | 23:49 |
| openstackgerrit | Steve Baker proposed openstack/tripleo-heat-templates master: Upgrade CI test - never merge https://review.openstack.org/461000 | 23:50 |
| *** leitan has joined #tripleo | 23:55 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!