| arxcruz | weshay: mwhahaha this might be related https://bugs.launchpad.net/tripleo/+bug/1702257 | 00:00 |
|---|---|---|
| openstack | Launchpad bug 1702257 in tripleo "VM's cannot have ssh key injected because can't access metadata service" [Critical,Fix released] | 00:00 |
| weshay | arxcruz, shouldn't that job be killed and run on rdo-cloud now? | 00:04 |
| * weshay looking at the bug, thanks for pointing it out | 00:04 | |
| Slower | weshay: got it thanks! | 00:04 |
| *** achadha has joined #tripleo | 00:06 | |
| *** radeks has joined #tripleo | 00:06 | |
| arxcruz | weshay: not sure... | 00:06 |
| *** achadha_ has joined #tripleo | 00:06 | |
| weshay | arxcruz, ya.. no more periodic jobs upstream man | 00:06 |
| arxcruz | weshay: this is not periodic job | 00:07 |
| weshay | really? /me looks again | 00:07 |
| weshay | arxcruz, http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-nonha-tempest-oooq-master/154fe22/logs/oooq/undercloud/home/jenkins/tempest/tempest.html.gz | 00:07 |
| arxcruz | weshay: which one you're looking? from the bug ? | 00:07 |
| arxcruz | weshay: I was talking about http://logs.openstack.org/74/467074/7/gate/legacy-tripleo-ci-centos-7-scenario001-multinode-oooq/451d2a8/logs/undercloud/home/zuul/tempest_output.log.txt.gz#_2017-11-06_23_29_16 | 00:08 |
| weshay | arxcruz, ya the gate failure | 00:08 |
| weshay | I know | 00:08 |
| weshay | arxcruz, this http://logs.openstack.org/periodic/periodic-tripleo-ci-centos-7-ovb-nonha-tempest-oooq-master/ | 00:08 |
| weshay | needs to move asap | 00:08 |
| weshay | tomorrow | 00:09 |
| weshay | :) | 00:09 |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 00:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 00:10 |
| arxcruz | weshay: ok, anyway, it might be some random problem in the infra? it's pretty hard to debug, if you don't have the same environment :/ | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 00:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 00:10 |
| *** achadha has quit IRC | 00:10 | |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 00:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 00:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 00:10 |
| arxcruz | gotta go, tty tomorrow | 00:10 |
| weshay | arxcruz, k.. ping me tomorrow w/ the project-config review to nuke that job and rdo review to add it | 00:10 |
| tonyb | mwhahaha: Should I open a bug for the case that no VIPs are assigend ig not using docker-ha ? | 00:12 |
| *** noslzzp has joined #tripleo | 00:13 | |
| openstackgerrit | Ian Main proposed openstack/tripleo-quickstart master: DNM, update undercloud install options for containers https://review.openstack.org/517445 | 00:14 |
| Slower | weshay: that look right? % | 00:14 |
| *** anilvenkata has quit IRC | 00:14 | |
| openstackgerrit | Ian Main proposed openstack/tripleo-quickstart-extras master: DNM, update the undercloud install for containers https://review.openstack.org/517444 | 00:15 |
| openstackgerrit | Ian Main proposed openstack/tripleo-quickstart-extras master: DNM, update the undercloud install for containers https://review.openstack.org/517444 | 00:17 |
| *** bobh has joined #tripleo | 00:24 | |
| *** anilvenkata has joined #tripleo | 00:27 | |
| openstackgerrit | Oliver Walsh proposed openstack/puppet-tripleo master: Unset MountFlags in docker.service systemd directives https://review.openstack.org/518179 | 00:31 |
| *** gbarros has quit IRC | 00:31 | |
| *** radeks has quit IRC | 00:32 | |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Set bind mount propegatation to shared for /var/lib/nova. https://review.openstack.org/518180 | 00:34 |
| *** bobh has quit IRC | 00:35 | |
| *** fzdarsky has quit IRC | 00:35 | |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Set bind mount propegatation to shared for /var/lib/nova. https://review.openstack.org/518180 | 00:35 |
| openstackgerrit | Ian Main proposed openstack/python-tripleoclient master: Support to drive undercloud deploy via undercloud.conf https://review.openstack.org/511350 | 00:35 |
| openstackgerrit | Ian Main proposed openstack/python-tripleoclient master: Pre-run validations and verifications for the undercloud https://review.openstack.org/513856 | 00:35 |
| *** achadha has joined #tripleo | 00:39 | |
| *** achadha_ has quit IRC | 00:42 | |
| *** achadha has quit IRC | 00:43 | |
| openstackgerrit | Oliver Walsh proposed openstack/puppet-tripleo master: Unset MountFlags in docker.service systemd directives https://review.openstack.org/518179 | 00:43 |
| *** gbarros has joined #tripleo | 00:45 | |
| weshay | Slower, yup | 00:48 |
| openstackgerrit | Ricardo Noriega proposed openstack/tripleo-heat-templates master: Using stevedore alias for BGPVPN Service Plugin https://review.openstack.org/511277 | 00:52 |
| *** chlong has joined #tripleo | 00:57 | |
| *** jkilpatr has joined #tripleo | 00:58 | |
| *** zshi_ has joined #tripleo | 01:04 | |
| *** owalsh_ has joined #tripleo | 01:09 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 01:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 01:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 01:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 01:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 01:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 01:10 |
| *** owalsh has quit IRC | 01:11 | |
| *** cshastri has joined #tripleo | 01:12 | |
| *** owalsh_ is now known as owalsh | 01:15 | |
| *** moshele has joined #tripleo | 01:18 | |
| *** gbarros_ has joined #tripleo | 01:20 | |
| *** gbarros has quit IRC | 01:20 | |
| *** noslzzp has quit IRC | 01:22 | |
| *** liverpooler has joined #tripleo | 01:27 | |
| openstackgerrit | Ian Main proposed openstack/tripleo-quickstart master: DNM, update undercloud install options for containers https://review.openstack.org/517445 | 01:28 |
| *** ecerquei has joined #tripleo | 01:30 | |
| *** gbarros_ has quit IRC | 01:31 | |
| *** fragatina has quit IRC | 01:35 | |
| *** zshi_ has quit IRC | 01:51 | |
| *** jlabarre has quit IRC | 01:56 | |
| *** rwsu has joined #tripleo | 02:01 | |
| *** zshi has joined #tripleo | 02:06 | |
| *** radeks has joined #tripleo | 02:09 | |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 02:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 02:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 02:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 02:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 02:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 02:10 |
| *** zshi has quit IRC | 02:14 | |
| *** radeks has quit IRC | 02:16 | |
| *** zshi has joined #tripleo | 02:19 | |
| *** jkilpatr has quit IRC | 02:27 | |
| *** bobh has joined #tripleo | 02:34 | |
| *** ecerquei has quit IRC | 02:37 | |
| *** liverpooler has quit IRC | 02:39 | |
| *** openstack has joined #tripleo | 02:43 | |
| *** ChanServ sets mode: +o openstack | 02:43 | |
| *** bobh has quit IRC | 02:48 | |
| *** nguyentrihai has quit IRC | 02:48 | |
| *** rodrigods has quit IRC | 02:50 | |
| *** gbarros has joined #tripleo | 02:51 | |
| *** links has joined #tripleo | 02:52 | |
| *** yamahata has quit IRC | 02:53 | |
| *** moshele has quit IRC | 02:54 | |
| *** rodrigods has joined #tripleo | 02:58 | |
| *** bkopilov has quit IRC | 03:07 | |
| *** apetrich has quit IRC | 03:09 | |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 03:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 03:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 03:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 03:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 03:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 03:10 |
| *** radeks has joined #tripleo | 03:11 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: create subunit test results for the undercloud sanity check https://review.openstack.org/517657 | 03:13 |
| openstackgerrit | Keith Schincke proposed openstack/tripleo-heat-templates master: Set file ACLs for Ceph keyrings for non-containerized deployment https://review.openstack.org/509020 | 03:18 |
| *** agopi has joined #tripleo | 03:19 | |
| *** achadha has joined #tripleo | 03:21 | |
| *** thrash is now known as thrash|g0ne | 03:24 | |
| *** gbarros has quit IRC | 03:26 | |
| *** achadha has quit IRC | 03:26 | |
| *** gbarros has joined #tripleo | 03:29 | |
| *** psachin has joined #tripleo | 03:43 | |
| *** itlinux has joined #tripleo | 03:47 | |
| *** brault has joined #tripleo | 03:51 | |
| *** itlinux has quit IRC | 03:54 | |
| *** radeks has quit IRC | 03:55 | |
| *** brault has quit IRC | 03:55 | |
| openstackgerrit | Ian Main proposed openstack/tripleo-quickstart master: DNM, update undercloud install options for containers https://review.openstack.org/517445 | 03:59 |
| *** ykarel|afk has joined #tripleo | 04:08 | |
| *** udesale has joined #tripleo | 04:10 | |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 04:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 04:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 04:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 04:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 04:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 04:10 |
| *** masco has joined #tripleo | 04:11 | |
| *** mdnadeem has joined #tripleo | 04:13 | |
| *** dbecker has quit IRC | 04:24 | |
| *** ramishra has joined #tripleo | 04:27 | |
| *** pdeore has joined #tripleo | 04:29 | |
| *** skramaja has joined #tripleo | 04:32 | |
| *** pdeore has quit IRC | 04:34 | |
| *** fragatina has joined #tripleo | 04:36 | |
| *** chlong has quit IRC | 04:36 | |
| *** chlong has joined #tripleo | 04:37 | |
| *** dbecker has joined #tripleo | 04:38 | |
| *** fragatina has quit IRC | 04:38 | |
| *** fragatina has joined #tripleo | 04:38 | |
| *** fragatina has quit IRC | 04:39 | |
| *** fragatina has joined #tripleo | 04:40 | |
| *** shreshtha has joined #tripleo | 04:41 | |
| *** yamahata has joined #tripleo | 04:43 | |
| *** bkopilov has joined #tripleo | 04:44 | |
| *** radeks has joined #tripleo | 04:49 | |
| *** jaganathan has joined #tripleo | 04:52 | |
| *** gbarros has quit IRC | 04:55 | |
| *** moshele has joined #tripleo | 04:59 | |
| *** ratailor has joined #tripleo | 05:01 | |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Fix overcloud-prep-network when overcloud_nodes is None https://review.openstack.org/517261 | 05:05 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/pike: mysql: Only set certificate specs if TLS everywhere is enabled https://review.openstack.org/517742 | 05:05 |
| openstackgerrit | Merged openstack/puppet-tripleo stable/pike: Certmonger: Only notify haproxy class if it's defined https://review.openstack.org/517743 | 05:05 |
| openstackgerrit | Merged openstack/puppet-tripleo master: certmonger/HAProxy: don't use principal if CA is local https://review.openstack.org/517039 | 05:05 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Rely on container setup for haproxy's certificate user and group https://review.openstack.org/517176 | 05:05 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Galera: add support for encrypted SST https://review.openstack.org/507852 | 05:05 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Don't rewrite resource_registry values that aren't paths https://review.openstack.org/517040 | 05:05 |
| *** cylopez has joined #tripleo | 05:05 | |
| *** cylopez has quit IRC | 05:06 | |
| *** cylopez has joined #tripleo | 05:07 | |
| *** cylopez has quit IRC | 05:07 | |
| *** cylopez has joined #tripleo | 05:09 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 05:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 05:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 05:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 05:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 05:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 05:10 |
| *** moshele has quit IRC | 05:10 | |
| *** janki has joined #tripleo | 05:17 | |
| *** anilvenkata has quit IRC | 05:19 | |
| *** anilvenkata has joined #tripleo | 05:19 | |
| *** gbarros has joined #tripleo | 05:31 | |
| *** gbarros has quit IRC | 05:38 | |
| *** chlong has quit IRC | 05:39 | |
| *** achadha has joined #tripleo | 05:39 | |
| *** anilvenkata has quit IRC | 05:40 | |
| *** pdeore has joined #tripleo | 05:43 | |
| *** gbarros has joined #tripleo | 05:43 | |
| *** achadha has quit IRC | 05:44 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Remove certificate before updating it https://review.openstack.org/517984 | 05:46 |
| *** gbarros has quit IRC | 05:46 | |
| *** anilvenkata has joined #tripleo | 05:47 | |
| *** dparkes has quit IRC | 05:47 | |
| *** gbarros has joined #tripleo | 05:47 | |
| Tengu | hello there :) | 05:48 |
| *** pchavva has joined #tripleo | 05:48 | |
| Tengu | o_____O wow. patch still being tested since yesterday?! | 05:50 |
| Tengu | amazing. 16 hours in the CI, still 2h to go. | 05:50 |
| jaosorior | yep | 05:51 |
| jaosorior | it's a bummer :/ | 05:51 |
| jaosorior | BUT in good news, the JSON stuff merged. Now we just need to make it ELK friendly | 05:51 |
| Tengu | it will fail, pretty sure. in addition, it apparently needs a rebase against master :D | 05:51 |
| Tengu | jaosorior: cool! I should get the stuff by this week hopefully. | 05:52 |
| Tengu | but I don't think it will require many changes. | 05:52 |
| jaosorior | excellent | 05:52 |
| Tengu | it's even possible newer versions of kibana/elasticsearch will be able to handle list of line inside single json object. | 05:53 |
| jaosorior | what I'm looking at there is that, it doesn't really output the context we need (project, user, domain, region, etc.) | 05:53 |
| jaosorior | So this change is needed https://review.openstack.org/#/c/517894/ | 05:53 |
| jaosorior | it will actually add what we need to the extra section | 05:53 |
| Tengu | hmm, I think I saw that in the outputs you passed me yesterday. | 05:53 |
| jaosorior | BUT, there's a catch | 05:53 |
| jaosorior | right now it outputs the token | 05:54 |
| jaosorior | like, the full keystone token | 05:54 |
| Tengu | erf | 05:54 |
| Tengu | session take-over and so on | 05:54 |
| jaosorior | right | 05:54 |
| jaosorior | it's a matter of filtering it | 05:54 |
| jaosorior | so what I want to figure out is, what I can output instead | 05:54 |
| Tengu | shouldn't be hard… ? lemme check your patch | 05:54 |
| jaosorior | Tengu: I think the filtering should go here: https://github.com/openstack/oslo.context/blob/master/oslo_context/context.py#L348 | 05:55 |
| Tengu | hmm, line 228, you might be able to filter it just before calling the _dictify_context? | 05:55 |
| jaosorior | get_logging_values should only give logging-safe values. | 05:55 |
| Tengu | ah, even better | 05:56 |
| jaosorior | but yeah, I want to output the token ID instead | 05:56 |
| *** moshele has joined #tripleo | 05:56 | |
| jaosorior | so that's what I wanna figure out, if I can get it. | 05:56 |
| *** pchavva has quit IRC | 05:56 | |
| Tengu | where is it called in the formatters? | 05:56 |
| Tengu | wait, checking the code. | 05:56 |
| Tengu | "oslo_context" ?? | 05:57 |
| Tengu | ah | 05:57 |
| Tengu | oslo | 05:57 |
| Tengu | not ocata -.-' | 05:57 |
| Tengu | #coffeeee | 05:57 |
| *** radeks has quit IRC | 05:57 | |
| *** anilvenkata has quit IRC | 05:57 | |
| Tengu | ok, that get_logging_values is called directly from the _dictify_context. cool. | 05:58 |
| *** dpawar has joined #tripleo | 05:58 | |
| jaosorior | lol | 05:59 |
| jaosorior | yep | 05:59 |
| Tengu | jaosorior: hmm. the get_logging_values() calls the to_dict that adds the auth_token | 05:59 |
| Tengu | it's the method just above | 05:59 |
| Tengu | https://github.com/openstack/oslo.context/blob/master/oslo_context/context.py#L340 | 05:59 |
| Tengu | you might want to change that line in order to get only the plain auth_id ? | 06:00 |
| Tengu | unless that to_dict is also called from some other place that actually need the auth_token, of course. | 06:01 |
| *** fragatina has quit IRC | 06:02 | |
| jaosorior | Tengu: it is | 06:02 |
| Tengu | in that case, you'll need to override that dict entry using another values.update() or even values.del(auth_token) (or smth like that, not 100% sure for the del) | 06:02 |
| jaosorior | Tengu: some projects call the to_dict to copy it out | 06:02 |
| jaosorior | that's why I was suggesting removing auth_token from the get_logging_values | 06:02 |
| jaosorior | what would be wrong with the del? | 06:03 |
| jaosorior | that's what I thought I would do | 06:03 |
| Tengu | ok. well, just add a line after the 355 that pops out the auth_token and another one that adds auth_id | 06:03 |
| Tengu | ah, not sure it's "del", maybe "delete" :) | 06:03 |
| Tengu | but yep, "deleting" that entry and pushing another one, or just updating its content so that current parser won't complain is the right way to go I think. | 06:04 |
| Tengu | the latter will ensure users parsing the log output won't get hard errors due to the lack of "auth_token" presence in the log. | 06:05 |
| jaosorior | Tengu: I think keystone had a library function that censored the auth_token | 06:06 |
| jaosorior | so that might be another option | 06:06 |
| Tengu | hmm. doesn't seem to be the case… ? | 06:07 |
| *** pkovar has joined #tripleo | 06:07 | |
| Tengu | unless you're thinking about that one: https://github.com/openstack/keystone/blob/2b6ec143976eb61e58535437606f45524e65cb9b/keystone/models/token_model.py#L83 | 06:07 |
| Tengu | but not really an "censore" function. just a plain getter. | 06:08 |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 06:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 06:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 06:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 06:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 06:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 06:10 |
| jaosorior | Tengu: they don't. only for passwords, not tokens | 06:11 |
| jaosorior | ok | 06:11 |
| *** hjensas has joined #tripleo | 06:13 | |
| *** hjensas has quit IRC | 06:13 | |
| *** hjensas has joined #tripleo | 06:13 | |
| jaosorior | Tengu: I'm starting to think the token might just be the token | 06:16 |
| jaosorior | and I can't get a token ID from it | 06:16 |
| *** anilvenkata has joined #tripleo | 06:16 | |
| Tengu | erf | 06:16 |
| Tengu | the token isn't an object? | 06:16 |
| jaosorior | that's what I'm researching | 06:16 |
| jaosorior | Tengu: yep, auth_token is the direct value from the header X-Auth-Token | 06:17 |
| Tengu | duh | 06:17 |
| Tengu | hmmm. is that information really revelant in logs? | 06:18 |
| jaosorior | I think we'll survive without the token id | 06:18 |
| jaosorior | just had the wild idea that it would be useful to spot stolen tokens | 06:19 |
| jaosorior | but whatever | 06:19 |
| Tengu | hmm. | 06:19 |
| *** pchavva has joined #tripleo | 06:20 | |
| Tengu | lemme check where that self.auth_token comes from. | 06:21 |
| *** abregman has joined #tripleo | 06:22 | |
| Tengu | ah, ok, yes, we can find it at the top: https://github.com/openstack/oslo.context/blob/master/oslo_context/context.py#L45 // and probably then https://github.com/openstack/oslo.context/blob/master/oslo_context/context.py#L193 is called with that header as well. | 06:22 |
| Tengu | maybe there's a call in keystone that can create the token *object* from its header? as it's in the DB… | 06:24 |
| Tengu | jaosorior: https://github.com/openstack/keystone/blob/master/keystone/token/persistence/core.py#L52 ? | 06:25 |
| Tengu | unless token_id is the "real, proper" name of the argument, of course. | 06:25 |
| *** yprokule has joined #tripleo | 06:28 | |
| jaosorior | right but remember that the token is filled up for every request that is made to an openstack service | 06:28 |
| jaosorior | so we don't want to do an extra keystone call there :/ | 06:28 |
| jaosorior | I think we'll have to do without the token id | 06:29 |
| Tengu | true. | 06:29 |
| Tengu | yup. | 06:29 |
| *** pkovar has quit IRC | 06:30 | |
| Tengu | *headshot* - CI failed me. | 06:31 |
| jaosorior | noooo | 06:31 |
| Tengu | hell yeah | 06:33 |
| Tengu | now have to understand why. | 06:33 |
| Tengu | http://logs.openstack.org/03/517903/1/gate/puppet-openstack-unit-4.8-centos-7/02025c9/job-output.txt.gz | 06:33 |
| Tengu | a git clone apparently failed. | 06:34 |
| Tengu | http://logs.openstack.org/03/517903/1/gate/puppet-openstack-unit-4.8-centos-7/02025c9/job-output.txt.gz#_2017-11-07_05_49_53_278978 | 06:34 |
| Tengu | unless it's a non-voting, the merge won't happen. | 06:34 |
| jaosorior | Tengu: is that link working even? | 06:35 |
| jaosorior | the opendaylight git repo | 06:35 |
| Tengu | no idea. | 06:35 |
| Tengu | don't even know what's this project. | 06:35 |
| Tengu | but seeing a 404 isn't that good | 06:36 |
| Tengu | ah. git clone https://git.opendaylight.org/gerrit/integration/packaging/puppet-opendaylight actually works. | 06:36 |
| jaosorior | dafuq | 06:36 |
| *** morazi has joined #tripleo | 06:36 | |
| Tengu | like you say. | 06:37 |
| Tengu | -.- | 06:37 |
| Tengu | I'm pretty sure the CI has some hidden flag that says "fail on him every time" | 06:37 |
| *** dparkes has joined #tripleo | 06:37 | |
| jaosorior | Tengu: you wouldn't be the only one flagged by CI https://review.openstack.org/#/q/owner:jaosorior%2540redhat.com+status:open :/ | 06:38 |
| Tengu | XD | 06:38 |
| Tengu | we hit some threshold, like "you submited too many patches, let the others work a bit, take a nap" XD | 06:38 |
| Tengu | jaosorior: ah, did you see Doug answer regarding a release note for the oslo.log? | 06:42 |
| jaosorior | Tengu: yep, adding it now | 06:42 |
| Tengu | cool | 06:42 |
| *** morazi has quit IRC | 06:42 | |
| Tengu | so. time to make a cup of tea. | 06:43 |
| *** etingof has quit IRC | 06:49 | |
| *** gbarros has quit IRC | 06:50 | |
| *** marios has joined #tripleo | 06:51 | |
| *** gbarros has joined #tripleo | 06:51 | |
| *** suuuper has joined #tripleo | 06:52 | |
| *** iranzo has joined #tripleo | 06:55 | |
| *** gbarros has quit IRC | 06:59 | |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/472607 | 07:00 |
| *** etingof has joined #tripleo | 07:03 | |
| *** jfrancoa has joined #tripleo | 07:09 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 07:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 07:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 07:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 07:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 07:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 07:10 |
| *** pgadiya has joined #tripleo | 07:11 | |
| openstackgerrit | Janki Chhatbar proposed openstack/tripleo-heat-templates stable/pike: Env files for ODL deployments https://review.openstack.org/518260 | 07:11 |
| *** dpawar has quit IRC | 07:14 | |
| *** jprovazn has joined #tripleo | 07:17 | |
| *** spectr has quit IRC | 07:22 | |
| openstackgerrit | Ian Main proposed openstack/tripleo-quickstart master: DNM, update undercloud install options for containers https://review.openstack.org/517445 | 07:27 |
| *** pdeore has quit IRC | 07:28 | |
| *** dpawar has joined #tripleo | 07:29 | |
| *** pdeore has joined #tripleo | 07:30 | |
| *** bogdando has joined #tripleo | 07:32 | |
| *** pchavva has quit IRC | 07:42 | |
| *** ykarel|afk is now known as ykarel|lunch | 07:47 | |
| *** morazi has joined #tripleo | 08:05 | |
| openstackgerrit | Juan Badia Payno proposed openstack/tripleo-heat-templates master: [WIP] logging: merge fluentd-client and fluentd-base https://review.openstack.org/501457 | 08:05 |
| *** apetrich has joined #tripleo | 08:06 | |
| *** ebarrera has joined #tripleo | 08:06 | |
| *** pcaruana has joined #tripleo | 08:06 | |
| *** mcornea has joined #tripleo | 08:07 | |
| *** dsariel__ has quit IRC | 08:07 | |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 08:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 08:10 |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 08:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 08:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 08:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 08:10 |
| openstackgerrit | Juan Badia Payno proposed openstack/tripleo-heat-templates master: [WIP] logging: merge fluentd-client and fluentd-base https://review.openstack.org/501457 | 08:11 |
| *** brault has joined #tripleo | 08:12 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Disable live migration over TLS https://review.openstack.org/517927 | 08:15 |
| *** aufi has joined #tripleo | 08:15 | |
| *** brault has quit IRC | 08:16 | |
| *** tesseract has joined #tripleo | 08:23 | |
| openstackgerrit | Tuan Luong-Anh proposed openstack/tripleo-docs master: Using --option ARGUMENT https://review.openstack.org/516244 | 08:24 |
| *** trown|outtypewww has quit IRC | 08:26 | |
| *** trown has joined #tripleo | 08:27 | |
| *** zoli is now known as zli | 08:28 | |
| *** zli is now known as zoli | 08:28 | |
| *** jpena|off is now known as jpena | 08:30 | |
| *** chem has joined #tripleo | 08:30 | |
| *** brault has joined #tripleo | 08:30 | |
| openstackgerrit | Marius Cornea proposed openstack/tripleo-heat-templates master: Update ansible before the ansible upgrade tasks https://review.openstack.org/517025 | 08:32 |
| *** oidgar has joined #tripleo | 08:35 | |
| *** ccamacho has joined #tripleo | 08:36 | |
| *** adac has joined #tripleo | 08:36 | |
| jaosorior | ccamacho: could you check this out https://review.openstack.org/#/c/517863/ ? | 08:38 |
| jaosorior | Tengu: was it the apache patch or the sshd? | 08:41 |
| Tengu | https://review.openstack.org/#/c/517903/ sshd | 08:41 |
| Tengu | the apache apparently hits some issues in the CI, I'd rather not stress about it and rebase it a bit later if needed. | 08:42 |
| jaosorior | Tengu: it's almost merging | 08:42 |
| Tengu | I think it hits some issues with the cinder volumes. | 08:42 |
| Tengu | jaosorior: ahhh, so that's the expected behavior then. ok. will be merged tomorrow then. | 08:42 |
| jaosorior | the sshd one | 08:42 |
| jaosorior | lol | 08:42 |
| jaosorior | maybe even today (if it passes the gate) | 08:42 |
| jaosorior | it's the first in the queue now | 08:42 |
| Tengu | for the apache one, I'll wait for the "urgent tripleo tasks" to be solved. | 08:43 |
| *** chem has quit IRC | 08:44 | |
| *** chem has joined #tripleo | 08:45 | |
| *** amoralej|off is now known as amoralej | 08:47 | |
| jaosorior | bogdando, bandini, ccamacho: Could you check this out https://review.openstack.org/#/c/517615/ ? | 08:47 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Add option for Neutron containers to log to stdout/stderr https://review.openstack.org/510464 | 08:48 |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart-extras master: Fix tempest html generation file https://review.openstack.org/517906 | 08:51 |
| *** gkadam has joined #tripleo | 08:53 | |
| *** ykarel|lunch is now known as ykarel | 08:53 | |
| Tengu | hmmm. stacktraces in neutron logs due to oslo not being able to treat rabbitmq messages apparently. nice one. | 08:57 |
| bogdando | folks, could we backport this UX thing https://review.openstack.org/#/c/517613 ? | 08:58 |
| bogdando | let's pike undercloud containers guide working finally | 08:58 |
| Tengu | oh. fun. | 08:58 |
| Tengu | 2017-11-07 08:58:07.168 215451 WARNING oslo.messaging._drivers.impl_rabbit [-] Unexpected error during heartbeart thread processing, retrying...: error: [Errno 111] ECONNREFUSED | 08:58 |
| Tengu | wondering if something's not fishy in the cluster… | 08:58 |
| jaosorior | Tengu: what commit? | 08:59 |
| Tengu | jaosorior: no one. jsut seeing that in my current install… since last weekend apparently. | 08:59 |
| jaosorior | dafuq | 08:59 |
| Tengu | we didn't do any update since the deploy, sooo… | 08:59 |
| Tengu | have to dig a bit. | 08:59 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: Remove certificate before updating it https://review.openstack.org/517984 | 09:00 |
| Tengu | this seems to create some issues in the overcloud network - a project sees its routers ports as DOWN, although we can ping the public one from the outside. | 09:00 |
| Tengu | pretty sure it's kind of linked. | 09:00 |
| Tengu | jaosorior: the other log is funny as well: 2017-11-07 08:57:26.297 216255 ERROR oslo.messaging._drivers.impl_rabbit [-] Failed to consume message from queue: 'NoneType' object has no attribute 'itervalues' | 09:00 |
| *** ffiore has joined #tripleo | 09:00 | |
| jaosorior | whaaaaat | 09:00 |
| Tengu | but not related to your current work :] | 09:00 |
| jaosorior | Tengu: could be fixed though | 09:01 |
| Tengu | :Þ anything could be fixed ;) | 09:02 |
| *** jpich has joined #tripleo | 09:02 | |
| Tengu | jaosorior: but I'm unable to find in the oslo.messaging code where it raises this error :/ | 09:06 |
| Tengu | I have the driver code here: https://github.com/openstack/oslo.messaging/blob/master/oslo_messaging/_drivers/impl_rabbit.py but it's… well, it's long ;). | 09:06 |
| Tengu | guess I'll open an issue on launchpad. | 09:07 |
| Tengu | ah. aha | 09:09 |
| Tengu | guess I have it in the stacktrace. | 09:09 |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 09:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 09:10 |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 09:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 09:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 09:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 09:10 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common-tempest-plugin master: Remove PEP8 Errors https://review.openstack.org/518280 | 09:12 |
| *** lucas-afk is now known as lucasagomes | 09:17 | |
| jaosorior | bandini: if you have some time can you check this one too https://review.openstack.org/#/c/517312/ ? | 09:18 |
| jaosorior | and this https://review.openstack.org/#/c/516982/ | 09:18 |
| *** etingof has quit IRC | 09:22 | |
| Tengu | jaosorior: I have some really nice errors in pcs status as well in fact :D | 09:24 |
| Tengu | I think galera is in an unconsistent state… galera_monitor_10000 on controller-2 'unknown error' (1): call=97, status=complete, exitreason='local node <controller-2> is started, but not in primary mode. Unknown state.', galera_monitor_10000 on controller-0 'unknown error' (1): call=87, status=complete, exitreason='local node <controller-0> is started, but not in primary mode. Unknown state.', | 09:24 |
| jaosorior | what the hell | 09:24 |
| jaosorior | dciabrin: ^^ | 09:24 |
| Tengu | http://paste.openstack.org/show/625685/ <- weird. | 09:25 |
| dciabrin | hmm, | 09:25 |
| Tengu | so basically that might explain some issues we currently have I guess. | 09:27 |
| dciabrin | Tengu, can you check whether "ps -ef | grep mysql" return something on controller-1 and see /var/log/mysqld.log for the exact state of galera? | 09:27 |
| dciabrin | Tengu, this could be some network issue, galera noticing it and switching to "Non-Primary" mode, which is a error condition for pacemaker monitoring | 09:28 |
| Tengu | dciabrin: hmm, maybe, but I doubt it: the nodes are in the same rack, connected through the same switch | 09:28 |
| Tengu | dciabrin: http://paste.openstack.org/show/625686/ | 09:29 |
| Tengu | and http://paste.openstack.org/show/625687/ | 09:30 |
| Tengu | so apparently it's up… | 09:30 |
| *** dsariel__ has joined #tripleo | 09:31 | |
| Tengu | hmm, it's more galera on controller-0 and -2 in fact I think | 09:31 |
| dciabrin | Tengu, I'm more interested of what happened around 00:42 | 09:31 |
| *** masco has quit IRC | 09:31 | |
| dciabrin | yeah both node showed a hiccup at the same time | 09:31 |
| dciabrin | and normally since there are galera Master in pcs status, pacemaker should start galera on controller-1 as well | 09:32 |
| Tengu | hmm. | 09:32 |
| Tengu | lemme check in the elk. | 09:32 |
| dciabrin | Tengu, also "pcs resource failcount show galera" to see if pacemaker blocked galera from starting on controller-1 | 09:32 |
| Tengu | 2 on -0 and 1 on -2 | 09:33 |
| Tengu | nothing got -1 | 09:33 |
| ccamacho | hello d0ugal good morning :) | 09:33 |
| d0ugal | Hy | 09:34 |
| d0ugal | Hi! | 09:34 |
| ccamacho | quick question, do you know if there is an easy way of getting the data in undercloud-passwords.conf from a mistral action? I just need the db password :P | 09:34 |
| ccamacho | thanks man and sorry for being a pain | 09:34 |
| ccamacho | maybe they are loaded somewhere I can get them | 09:35 |
| *** etingof has joined #tripleo | 09:35 | |
| d0ugal | ccamacho: undercloud_db_password? | 09:35 |
| ccamacho | g0ugal yeahp | 09:36 |
| Tengu | dciabrin: hmm nothing in journalctl on controller-2, will check the two others | 09:36 |
| d0ugal | ccamacho: I think there is a way, just finding it. | 09:36 |
| ccamacho | thanks man | 09:36 |
| d0ugal | ccamacho: I would have thought this action: https://github.com/openstack/tripleo-common/blob/a6b0682bb4dd1e0d613b5d1a64dc1bfe8d465a87/workbooks/plan_management.yaml#L360 | 09:38 |
| d0ugal | ccamacho: but I don't see it - does this value come from a Heat parameter initially? | 09:38 |
| ccamacho | d0ugal, there is a Heat parameter called MysqlRootPassword, but I want the undercloud db password, I think we use instack-undercloud to configure it instead of THT | 09:41 |
| *** derekh has joined #tripleo | 09:42 | |
| *** masco has joined #tripleo | 09:43 | |
| d0ugal | ccamacho: oh, interesting. Maybe this isn't something that is available in Mistral yet. | 09:46 |
| *** bogdando has quit IRC | 09:46 | |
| *** tosky has joined #tripleo | 09:46 | |
| *** bogdando has joined #tripleo | 09:46 | |
| d0ugal | ccamacho: looks like it comes from hiera. So puppet generated? | 09:48 |
| ccamacho | d0ugal neat ;P the thing is that im able to run the mysqldump with elevated privileges but from previous chats, is not recommended to run elevated actions, so, that's the reason Im trying to retrieve the root db passoword for the undercloud | 09:48 |
| ccamacho | yeahp | 09:48 |
| ccamacho | its configured by puppet | 09:48 |
| d0ugal | ccamacho: so this is a bit of a pain to solve | 09:49 |
| ccamacho | jaja | 09:49 |
| d0ugal | ccamacho: basically, we need instack-undercloud to put the password somewhere that Mistral can access it. | 09:49 |
| bogdando | jistr: hi, could you revise https://review.openstack.org/#/c/513669/ ? | 09:49 |
| ccamacho | ok so, there are some additional bits to work on | 09:49 |
| d0ugal | ccamacho: https://github.com/openstack/instack-undercloud/blob/6939ac649d5206acc08c953994ed7aa338d9dea7/instack_undercloud/undercloud.py#L1495-L1499 | 09:49 |
| d0ugal | ccamacho: we do the same with the undercloud_ceilometer_snmpd_password | 09:50 |
| ccamacho | d0ugal you rock man!!!! | 09:50 |
| ccamacho | thanks!!! | 09:50 |
| d0ugal | ccamacho: so you could add it there potentially and copy the steps for that password | 09:50 |
| ccamacho | Ill dig further from that point | 09:50 |
| d0ugal | ccamacho: np | 09:50 |
| ccamacho | d0ugal++ | 09:50 |
| openstackgerrit | Thomas Herve proposed openstack/tripleo-common master: Make some tweaks to swift logging https://review.openstack.org/498930 | 09:51 |
| bogdando | jistr: I posted update to the ml as well http://lists.openstack.org/pipermail/openstack-dev/2017-November/124277.html | 09:53 |
| janki | I am facing https://bugzilla.redhat.com/show_bug.cgi?id=1496203. Is there any upstream bug for this and a workaround? | 10:01 |
| openstack | bugzilla.redhat.com bug 1496203 in instack-undercloud "tripleo quickstart fails with error lvs: lvm lvs --help: Invalid units specification" [Unspecified,New] - Assigned to jslagle | 10:01 |
| *** salmankhan has joined #tripleo | 10:02 | |
| *** etingof has quit IRC | 10:02 | |
| *** gfidente has joined #tripleo | 10:02 | |
| *** gfidente has quit IRC | 10:02 | |
| *** gfidente has joined #tripleo | 10:02 | |
| janki | jaosorior, mandre marios ^ | 10:05 |
| bogdando | jaosorior: PTAL once again https://review.openstack.org/#/c/513669 :) | 10:08 |
| jistr | bogdando: looks good, +2'd :) | 10:08 |
| bogdando | jistr: thanks | 10:08 |
| *** SlickNik has quit IRC | 10:09 | |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 10:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 10:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 10:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 10:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 10:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 10:10 |
| bogdando | jistr: posted a little update | 10:14 |
| bogdando | so the reset issue will only be a problem for the follow-up patch statefull services like mysql | 10:15 |
| bogdando | and should not be an issue for that patch | 10:15 |
| bogdando | hope that's ok | 10:15 |
| *** etingof has joined #tripleo | 10:15 | |
| bogdando | the t-h-t docker/service part perhaps will be reworked with semanage fcontext | 10:16 |
| *** SlickNik has joined #tripleo | 10:19 | |
| *** ckyriakidou has joined #tripleo | 10:22 | |
| *** jchhatbar has joined #tripleo | 10:26 | |
| *** janki has quit IRC | 10:26 | |
| *** stendulker has joined #tripleo | 10:26 | |
| openstackgerrit | Merged openstack/puppet-tripleo master: Ensure sshd has proper configuration for its HostKey. https://review.openstack.org/517903 | 10:31 |
| *** SlickNik has quit IRC | 10:32 | |
| *** abregman has quit IRC | 10:32 | |
| *** pgadiya has quit IRC | 10:33 | |
| *** janki has joined #tripleo | 10:34 | |
| *** jchhatbar has quit IRC | 10:35 | |
| *** gfidente has quit IRC | 10:36 | |
| *** mrch has joined #tripleo | 10:39 | |
| *** fragatina has joined #tripleo | 10:39 | |
| *** SlickNik has joined #tripleo | 10:40 | |
| openstackgerrit | Carlos Camacho proposed openstack/instack-undercloud master: WIP load undercloud password to a mistral environment https://review.openstack.org/518292 | 10:44 |
| *** links has quit IRC | 10:45 | |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Refactor cellv2 host discovery logic to avoid races https://review.openstack.org/513867 | 10:51 |
| amoralej | could i get some eyes on https://review.openstack.org/#/c/518037/ ? | 10:53 |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Update default cell_v2 cell when it already exists https://review.openstack.org/506595 | 10:53 |
| *** links has joined #tripleo | 10:57 | |
| *** ykarel is now known as ykarel|afk | 10:57 | |
| jistr | trown, adarazs|rover, panda|ruck: hi, please consider reviewing these patches if/when you have some bandwidth https://review.openstack.org/516958 https://review.openstack.org/499680 | 11:02 |
| *** jaganathan has quit IRC | 11:03 | |
| *** ckyriakidou has quit IRC | 11:05 | |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 11:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 11:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 11:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 11:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 11:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 11:10 |
| *** stendulker has quit IRC | 11:11 | |
| *** rfolco has joined #tripleo | 11:15 | |
| adarazs|rover | jistr: first one +2 (needs gates), the second is submitted. thanks! | 11:15 |
| jistr | adarazs|rover: thanks! | 11:15 |
| *** pblaho has quit IRC | 11:20 | |
| openstackgerrit | Guoqiang Ding proposed openstack/puppet-tripleo master: Update the doc link https://review.openstack.org/518297 | 11:22 |
| *** pkovar has joined #tripleo | 11:24 | |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-common master: Update ceph-ansible playbook path in parameter default for update https://review.openstack.org/515442 | 11:24 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-common master: Healthcheck for nova_migration_target container https://review.openstack.org/518298 | 11:24 |
| *** agopi has quit IRC | 11:25 | |
| *** benone has quit IRC | 11:26 | |
| *** benone has joined #tripleo | 11:27 | |
| *** akane has joined #tripleo | 11:28 | |
| *** udesale has quit IRC | 11:28 | |
| jaosorior | jistr: could you check this out https://review.openstack.org/#/c/517615/ ? | 11:29 |
| *** links has quit IRC | 11:30 | |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-common master: Healthcheck for nova_migration_target container https://review.openstack.org/518298 | 11:31 |
| *** benone has quit IRC | 11:32 | |
| *** benone has joined #tripleo | 11:34 | |
| *** shreshtha has quit IRC | 11:35 | |
| jaosorior | marios: do you know how the networks are rendered? | 11:36 |
| jaosorior | the j2 files | 11:36 |
| jaosorior | jistr: ^^ | 11:36 |
| jaosorior | bandini: ^^ | 11:36 |
| marios | jaosorior: there is tripleo-common code that does that sec | 11:37 |
| marios | jaosorior: you can also do tox -e templates in the tht directory and all the .j2 -> .yaml | 11:37 |
| marios | fyi | 11:37 |
| jaosorior | marios: where is the list of .j2 files gotten from? | 11:38 |
| jaosorior | marios: seems this commit https://review.openstack.org/#/c/486260/ broke TLS everywhere as we're not rendering xtraconfig/nova_metadata/krb-service-principals.j2.yaml | 11:38 |
| *** benone has quit IRC | 11:39 | |
| marios | jaosorior: see here for the j2-> yaml https://github.com/openstack/tripleo-common/blob/9c4e6a0c7ee3770d67cb5fcb709dcaec36b5511d/tripleo_common/actions/templates.py#L239 | 11:40 |
| *** benone has joined #tripleo | 11:40 | |
| jaosorior | marios: so, it basically fetches all the files from the swift container and renders them if needed? | 11:43 |
| *** links has joined #tripleo | 11:43 | |
| jaosorior | d0ugal, marios: how are the t-h-t files added to the plan? | 11:44 |
| marios | jaosorior: yeah that sounds right, i've only looked at specific bits of this for whatever i was working on at the time and its been a while. the latest additions are the network ones i believe. that review you pointed at is a few months old. could it be you don't have updated tripleo-common for whateer reason. and so the network .j2 code isn't there? | 11:45 |
| *** benone has quit IRC | 11:45 | |
| *** salmankhan has quit IRC | 11:45 | |
| marios | jaosorior: from the depends on in that review the tripleo-common here https://review.openstack.org/#/c/476682/ | 11:45 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Allow SELinux shared context for containers state https://review.openstack.org/517383 | 11:46 |
| * d0ugal reads | 11:46 | |
| *** benone has joined #tripleo | 11:47 | |
| jaosorior | marios: well, the file isn't *.network.j2.yaml. But instead it's *.j2.yaml, so it should work | 11:47 |
| d0ugal | jaosorior: not sure I follow the question, but maybe you got he answer already | 11:48 |
| d0ugal | jaosorior: files are uploaded directly to swift iirc | 11:48 |
| jaosorior | d0ugal: so... there's this bug https://bugs.launchpad.net/tripleo/+bug/1730650 | 11:48 |
| openstack | Launchpad bug 1730650 in tripleo "cannot use enable-internal-tls.yaml due to /usr/share/openstack-tripleo-heat-templates/extraconfig/nova_metadata/krb-service-principals.yaml doesn't exist" [Critical,Triaged] | 11:48 |
| jaosorior | d0ugal: which basically is that this file https://github.com/openstack/tripleo-heat-templates/blob/master/extraconfig/nova_metadata/krb-service-principals.j2.yaml is not getting rendered | 11:49 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Test containers with SELinux Enforcing https://review.openstack.org/517384 | 11:49 |
| jaosorior | marios: do you think this file https://github.com/openstack/tripleo-heat-templates/blob/master/extraconfig/nova_metadata/krb-service-principals.j2.yaml should have been with the extension network.j2.yaml instead? | 11:49 |
| *** dtantsur|afk is now known as dtantsur | 11:49 | |
| d0ugal | jaosorior: looks like it | 11:50 |
| jaosorior | or not, actually, the files from the same commit don't have the network.j2.yaml extension | 11:51 |
| jaosorior | those are different ones | 11:51 |
| jaosorior | ok, I'm pretty confused :/ | 11:51 |
| d0ugal | jaosorior: I didn't realise we had three different cases .role.j2.yaml, .network.j2.yaml and .j2.yaml. That is pretty confusing | 11:51 |
| d0ugal | https://github.com/openstack/tripleo-common/blob/master/tripleo_common/actions/templates.py#L240 | 11:52 |
| *** radeks has joined #tripleo | 11:52 | |
| *** salmankhan has joined #tripleo | 11:52 | |
| d0ugal | I don't know this code well at all, sorry | 11:54 |
| jaosorior | neither do I | 11:54 |
| jaosorior | d0ugal: any idea who would? | 11:54 |
| *** abregman has joined #tripleo | 11:54 | |
| d0ugal | jaosorior: I think shardy wrote most of it | 11:54 |
| d0ugal | or maybe rbrady | 11:55 |
| *** raildo has joined #tripleo | 11:55 | |
| jaosorior | mandre: are you acquainted with this code by any chance https://github.com/openstack/tripleo-common/blob/master/tripleo_common/actions/templates.py#L240 ? | 11:56 |
| d0ugal | that is a whopping function | 11:57 |
| *** bogdando has quit IRC | 11:58 | |
| jaosorior | :/ | 11:59 |
| *** benone has quit IRC | 11:59 | |
| *** jpena is now known as jpena|lunch | 12:00 | |
| *** benone has joined #tripleo | 12:01 | |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart-extras master: Fix tempest html generation file https://review.openstack.org/517906 | 12:02 |
| *** lucasagomes is now known as lucas-hungry | 12:04 | |
| *** jkilpatr has joined #tripleo | 12:04 | |
| *** benone has quit IRC | 12:06 | |
| *** ykarel|afk is now known as ykarel | 12:07 | |
| *** benone has joined #tripleo | 12:08 | |
| Tengu | mandre: my sshd configuration stuff has been merged in master :). Shall we take it to stable/pike as well? | 12:09 |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729328 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 12:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 12:10 |
| openstack | Launchpad bug 1729328 in tripleo "Container images build in RDOcloud are randomly failing to be pushed to localhost" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 12:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 12:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 12:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 12:10 |
| amoralej | panda|ruck, i think we can remove tags to https://bugs.launchpad.net/tripleo/+bug/1729328 | 12:10 |
| amoralej | we can keep it open for some days, but i think we are not hitting it anymore | 12:10 |
| *** benone has quit IRC | 12:12 | |
| *** thrash|g0ne is now known as thrash | 12:18 | |
| *** brault has quit IRC | 12:18 | |
| *** gfidente has joined #tripleo | 12:19 | |
| *** gfidente has quit IRC | 12:19 | |
| *** gfidente has joined #tripleo | 12:19 | |
| *** benone has joined #tripleo | 12:20 | |
| *** iranzo has quit IRC | 12:20 | |
| *** pdeore has quit IRC | 12:20 | |
| *** morazi has quit IRC | 12:21 | |
| *** abregman is now known as abregman|mtg | 12:21 | |
| *** janki has quit IRC | 12:22 | |
| *** cshastri has quit IRC | 12:23 | |
| *** iranzo has joined #tripleo | 12:24 | |
| *** iranzo has quit IRC | 12:24 | |
| *** iranzo has joined #tripleo | 12:24 | |
| jistr | jaosorior: +2'd, i'll leave the +A up to you (not sure if you want to see the caserver job passing there) | 12:24 |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Add option for ceph ansible playbook https://review.openstack.org/515436 | 12:25 |
| *** pblaho has joined #tripleo | 12:26 | |
| *** radeks has quit IRC | 12:26 | |
| *** pkovar has quit IRC | 12:29 | |
| *** ansmith has quit IRC | 12:31 | |
| slagle | jistr: hey, have a few? i'm having a problem with the undercloud tasks in deploy-steps | 12:32 |
| jistr | slagle: yea, what's the issue? | 12:32 |
| slagle | jistr: when i'm trying to run it all via the mistral workflow, the mistral user is the one that runs ansible-playbook | 12:32 |
| slagle | that fails on all localhost tasks b/c the mistral user can't sudo | 12:33 |
| *** benone has quit IRC | 12:33 | |
| slagle | and we have to run with --become | 12:33 |
| *** mdnadeem has quit IRC | 12:33 | |
| *** pradk has joined #tripleo | 12:33 | |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Add option for ceph ansible playbook https://review.openstack.org/515436 | 12:33 |
| slagle | so...not sure how to address that other than be explicit about become:true only on some tasks | 12:33 |
| slagle | except even that won't work i don't think | 12:34 |
| *** benone has joined #tripleo | 12:34 | |
| *** catintheroof has joined #tripleo | 12:34 | |
| jistr | slagle: i was assuming that we won't need sudo generally, but have it available if needed as e.g. here https://github.com/openstack/tripleo-heat-templates/blob/master/extraconfig/services/kubernetes-master.yaml#L72-L73 | 12:35 |
| jistr | slagle: though that should be provisional, and removed when we have kubespray packaged | 12:35 |
| *** salmankhan has quit IRC | 12:36 | |
| slagle | jistr: that's not going to work though, b/c that task is for localhost, so ansible skips ssh and just runs directly as the mistral user | 12:36 |
| slagle | and the sudo will fail | 12:36 |
| jistr | slagle: yea... did you come across any other tasks that need sudo? | 12:37 |
| *** links has quit IRC | 12:37 | |
| slagle | jistr: well, right now, almost everything in host-prep-tasks and deploy-steps | 12:37 |
| slagle | it's not explicit on the tasks, b/c it's assume you run with --become | 12:37 |
| slagle | jistr: so right now what's failing for me is just the gather_facts | 12:38 |
| slagle | b/c i don't have any other actual undercloud tasks | 12:38 |
| *** salmankhan has joined #tripleo | 12:38 | |
| *** benone has quit IRC | 12:39 | |
| jistr | slagle: i'm a bit confused -- i think host-prep-tasks and deploy-steps are executed on the overcloud though, over ssh, so that shouldn't be an issue? | 12:39 |
| slagle | jistr: that's not an issue. it just requires sudo...so i must run with --become...so then gather_facts on the undercloud fails | 12:39 |
| *** achadha has joined #tripleo | 12:39 | |
| slagle | b/c --become is global | 12:40 |
| jistr | ah i see | 12:40 |
| slagle | i can override that on the gather_facts underclodu task by setting become:false | 12:40 |
| slagle | but aren't you going to need sudo for other things to install kube? | 12:40 |
| *** dpawar has quit IRC | 12:40 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Send ARA statistics to Graphite server https://review.openstack.org/479882 | 12:41 |
| jistr | yea we could use `become` on play level rather than playbook level then | 12:41 |
| *** benone has joined #tripleo | 12:42 | |
| jistr | slagle: currently the host_prep_tasks do require sudo, but i don't think they necessarily need to. It's only for cloning kubespray into /usr/share. Once it's packaged, and assuming it's preinstalled, i don't think we'd need sudo on the undercloud. | 12:43 |
| jistr | slagle: err s/host_prep_tasks/undercloud tasks/ | 12:43 |
| slagle | not even to start any services or containers? | 12:44 |
| slagle | or does none of that actually run on the underclodu? | 12:44 |
| slagle | i guess it wouldn't | 12:44 |
| *** achadha has quit IRC | 12:44 | |
| jistr | slagle: yea currently nothing like that happens on undercloud. We basically just use it as a point from where we can run the nested ansible-playbook command. | 12:45 |
| *** catintheroof has quit IRC | 12:45 | |
| jistr | gfidente: does ceph-ansible need sudo on the host where it's running from? (undercloud) | 12:45 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Set become:false for undercloud plays https://review.openstack.org/518323 | 12:46 |
| gfidente | jistr no it should not | 12:46 |
| gfidente | jistr used to run fine as mistral | 12:46 |
| openstackgerrit | James Slagle proposed openstack/tripleo-quickstart master: fs010: use --config-download from tripleoclient https://review.openstack.org/514736 | 12:46 |
| *** benone has quit IRC | 12:46 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-common master: WIP add UC backup actions https://review.openstack.org/517610 | 12:47 |
| slagle | jistr: ok, https://review.openstack.org/#/c/518323/ | 12:47 |
| *** benone has joined #tripleo | 12:48 | |
| jistr | slagle: lgtm. I think i just need to clone kubespray into the workdir rather than /usr/share and we should be good. Ideally it would be packaged anyway. | 12:49 |
| *** abishop has joined #tripleo | 12:49 | |
| jistr | gfidente: thanks for confirming | 12:50 |
| *** aditya_r has joined #tripleo | 12:52 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates master: Remove deprecated Telemetry services from roles data https://review.openstack.org/509645 | 12:54 |
| *** ffiore has quit IRC | 12:55 | |
| *** jlabarre has joined #tripleo | 12:56 | |
| *** catintheroof has joined #tripleo | 12:57 | |
| *** lucas-hungry is now known as lucasagomes | 12:57 | |
| *** ffiore has joined #tripleo | 12:58 | |
| *** benone has quit IRC | 13:00 | |
| *** rlandy has joined #tripleo | 13:02 | |
| *** benone has joined #tripleo | 13:02 | |
| *** dpawar has joined #tripleo | 13:03 | |
| *** jcoufal has joined #tripleo | 13:04 | |
| *** ratailor has quit IRC | 13:08 | |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 13:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 13:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 13:10 |
| dtantsur | folks, could you please review https://review.openstack.org/#/c/516893/ ? this is a very hot topic nowadays. | 13:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 13:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 13:10 |
| *** shreshtha has joined #tripleo | 13:11 | |
| *** skramaja has quit IRC | 13:12 | |
| *** ramishra has quit IRC | 13:12 | |
| *** benone has quit IRC | 13:13 | |
| *** ecerquei has joined #tripleo | 13:14 | |
| *** ramishra has joined #tripleo | 13:14 | |
| *** benone has joined #tripleo | 13:15 | |
| *** aditya_r has quit IRC | 13:15 | |
| *** aditya_r has joined #tripleo | 13:15 | |
| *** fragatina has quit IRC | 13:16 | |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart-extras master: Adding the possibility to pass the log url to tempestmail https://review.openstack.org/518327 | 13:16 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-common master: Truncate the zaqar message to 512 kbytes https://review.openstack.org/518328 | 13:17 |
| *** abregman|mtg is now known as abregman | 13:17 | |
| jaosorior | jistr: right now the TLS everywhere job would be broken by this https://bugs.launchpad.net/tripleo/+bug/1730650 :/ REALLY need to fix that and I don't really know how | 13:18 |
| openstack | Launchpad bug 1730650 in tripleo "cannot use enable-internal-tls.yaml due to /usr/share/openstack-tripleo-heat-templates/extraconfig/nova_metadata/krb-service-principals.yaml doesn't exist" [Critical,Triaged] | 13:18 |
| jaosorior | slagle: are you acquainted with the code that renders the j2.yaml files? | 13:19 |
| *** benone has quit IRC | 13:19 | |
| slagle | i'm acquainted | 13:20 |
| *** benone has joined #tripleo | 13:21 | |
| *** jmelvin has joined #tripleo | 13:21 | |
| jaosorior | slagle: so, the TLS everywhere workflow got broken because this file: /usr/share/openstack-tripleo-heat-templates/extraconfig/nova_metadata/krb-service-principals.yaml is not being rendered. It was set up to be a template in this commit https://review.openstack.org/#/c/486260/ . Is there a list somewhere where I need to add it to tripleo-common so it gets rendered? or what's missing? | 13:21 |
| slagle | jistr: dang, another problem. /home/mistral doesn't exist on the uc and ansible tries to use it for localhost tasks | 13:21 |
| jaosorior | slagle: this would be the bug report: https://bugs.launchpad.net/tripleo/+bug/1730650 | 13:22 |
| openstack | Launchpad bug 1730650 in tripleo "cannot use enable-internal-tls.yaml due to /usr/share/openstack-tripleo-heat-templates/extraconfig/nova_metadata/krb-service-principals.yaml doesn't exist" [Critical,Triaged] | 13:22 |
| slagle | jistr: i think we can override that with remote_tmp | 13:22 |
| *** pcaruana has quit IRC | 13:22 | |
| jistr | slagle: yea we should be able to. I think social hit something similar when he was testing the improved ansible mistral action. | 13:23 |
| slagle | jaosorior: you shouldn't have to do anything explicitly. fwiw, it is getting rendered for me with tools/process-templates.py | 13:24 |
| slagle | jaosorior: so maybe it's something just on the tripleo-common side | 13:24 |
| jaosorior | slagle: maybe that's the case. Do you know if calling would trigger such rendering? | 13:25 |
| jaosorior | mandre: ^^ | 13:25 |
| jaosorior | slagle: * if calling openstack overcloud container image prepare | 13:25 |
| slagle | jaosorior: no. i don't think that will do it | 13:26 |
| *** benone has quit IRC | 13:26 | |
| adarazs|rover | mwhahaha & sshnaidm: I have a question regarding the overcloud deploy timeout without any visible error thing that I'm about to open a bug for. | 13:26 |
| jaosorior | slagle: so that seems to be the issue. Perhaps it should do the rendering. | 13:26 |
| adarazs|rover | mwhahaha, sshnaidm: so Sagi found this: http://logs.openstack.org/12/517412/3/gate/legacy-tripleo-ci-centos-7-nonha-multinode-oooq/7eacd85/logs/undercloud/var/log/mistral/executor.log.txt.gz#_2017-11-05_14_36_01_209 | 13:26 |
| slagle | jaosorior: why would that do the rendering? | 13:27 |
| adarazs|rover | mwhahaha, sshnaidm: and you're saying in the email that "This is related to our log collection where we try and query for the overcloud stats." -> you mean log collection in the deployment, or the quickstart's collect-logs role? because that doesn't run until much later. | 13:27 |
| jaosorior | slagle: cause if not, it fails if you pass an environment file that contains a file that needs rendering, since the file wouldn't be there. | 13:27 |
| adarazs|rover | mwhahaha, sshnaidm: otherwise this might be a real error. | 13:27 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-common master: Truncate the zaqar message to 512 kbytes https://review.openstack.org/518328 | 13:27 |
| slagle | jaosorior: perhaps. i don't really think that's how that command is meant to be used. so you've probably discovered a new use case | 13:28 |
| slagle | jaosorior: that command doesn't even take a templates dir | 13:28 |
| *** panda|ruck is now known as panada|ruck|brb | 13:28 | |
| *** benone has joined #tripleo | 13:29 | |
| *** ffiore has quit IRC | 13:29 | |
| slagle | jaosorior: i'm not sure. maybe it should. mandre or sbaker are the experts here :) | 13:29 |
| jaosorior | slagle: right. perhaps the command shouldn't fail if it receives an unexistent path in the definition | 13:29 |
| jaosorior | slagle: that makes sense. Thanks. | 13:29 |
| sshnaidm | adarazs|rover, I really have no idea what mistral does there, but I suspect it just tries to get info about stack overcloud and doesn't find it. | 13:30 |
| adarazs|rover | sshnaidm: yeah, but I don't see that error in other jobs that are successful. so that's the only error I see there as the overcloud-deploy just hangs. | 13:30 |
| sshnaidm | jpich, hey | 13:30 |
| adarazs|rover | sshnaidm: so I think that might be useful to include it in the bug. | 13:31 |
| jpich | sshnaidm: hey | 13:31 |
| sshnaidm | jpich, can you please paste me whole log with tripleo quickstart command? | 13:31 |
| sshnaidm | jpich, until it fails | 13:31 |
| jpich | sshnaidm: Sure! Does it save to a log somewhere or should I just grab what came out on the terminal? | 13:31 |
| trown | sshnaidm: issue is in the security groups \o/ | 13:32 |
| trown | sshnaidm: added default sec group to both nodes and I can ping across the tunnel | 13:32 |
| sshnaidm | adarazs|rover, may be, but if I understood mwhahaha right it's just a symptom.. although I didn't find any other errors that could give a clue what's wrong there | 13:32 |
| adarazs|rover | yep, I will include it... | 13:32 |
| sshnaidm | trown, woot \o/ | 13:33 |
| sshnaidm | trown, damn, but I allowed pings there, why didn't it work?? | 13:33 |
| *** benone has quit IRC | 13:33 | |
| *** chem has quit IRC | 13:33 | |
| trown | I think it is the rules alowing "ALLOW IPv4 from default" | 13:34 |
| trown | sshnaidm: since that is the only one missing from your group | 13:34 |
| *** brault has joined #tripleo | 13:34 | |
| *** benone has joined #tripleo | 13:35 | |
| trown | sshnaidm: this one bothered me all night :P ... tunnel was pretty clearly configured correctly | 13:35 |
| *** cylopez1 has joined #tripleo | 13:35 | |
| *** cylopez has quit IRC | 13:36 | |
| jrist | we really need this fix pushed up today if possible - can we get an exception to the +2 rule for TripleO-UI, mwhahaha ? https://review.openstack.org/#/c/517024/ | 13:36 |
| jrist | it works, I just tested the fix | 13:36 |
| sshnaidm | trown, I've almost broke my brain trying to understand what's wrong.. will try to fix it now | 13:37 |
| *** ramishra has quit IRC | 13:37 | |
| *** chem has joined #tripleo | 13:39 | |
| *** liverpooler has joined #tripleo | 13:39 | |
| jrist | slagle: can I beg for a +A on that bug from you? https://review.openstack.org/#/c/517024/ | 13:39 |
| slagle | sure. i'll wait. | 13:40 |
| sshnaidm | jpich, you can run like myscript.sh -opts -args ... 2>&1 | tee my_lovely.log and then paste the my_lovely.log | 13:40 |
| jpich | sshnaidm: Paste is truncating it, I sent it by email | 13:40 |
| slagle | jrist: wth programming language is this anyway? | 13:40 |
| sshnaidm | jpich, got it | 13:40 |
| *** akane has quit IRC | 13:41 | |
| *** aputtur_ has joined #tripleo | 13:41 | |
| jrist | slagle: :( | 13:41 |
| jrist | slagle: one of the great languages | 13:41 |
| *** cylopez1 has quit IRC | 13:42 | |
| slagle | too many semi-colons | 13:42 |
| *** ansmith has joined #tripleo | 13:42 | |
| *** benone has quit IRC | 13:45 | |
| jrist | nice disclaimer | 13:46 |
| *** bkopilov has quit IRC | 13:46 | |
| *** ffiore has joined #tripleo | 13:46 | |
| *** panada|ruck|brb is now known as panda|ruck | 13:47 | |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-validations master: Fix typos in pacemaker module documentation https://review.openstack.org/518330 | 13:48 |
| sshnaidm | tripleoclient fails in unittests in gate: FAIL: tripleoclient.tests.v1.overcloud_deploy.test_overcloud_deploy.TestDeployOvercloud.test_answers_file http://logs.openstack.org/49/517349/2/gate/legacy-tripleo-ci-centos-7-scenario002-multinode-oooq/4dff5cd/logs/delorean_logs/8b/d8/8bd8715fee9392b07d03b6dad960c31420aaf4b1_dev/build.log.txt.gz | 13:49 |
| sshnaidm | panda|ruck, adarazs|rover ^^ | 13:50 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates master: Add Octavia to scenario002 (containers only) https://review.openstack.org/518331 | 13:50 |
| *** ffiore has quit IRC | 13:51 | |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates master: Add Octavia to scenario002 (containers only) https://review.openstack.org/518331 | 13:52 |
| fultonj | is anyone able to review https://review.openstack.org/#/c/516224/ ? thanks | 13:52 |
| *** thrash is now known as thrash|biab | 13:54 | |
| *** jpena|lunch is now known as jpena | 13:54 | |
| abishop | slagle: gfidente and I would like you to review https://review.openstack.org/506715 | 13:54 |
| *** zshi has quit IRC | 13:55 | |
| *** yprokule_ has joined #tripleo | 13:55 | |
| adarazs|rover | sshnaidm: huh, I wonder how that is possible, when the change must have passed the same unittest in the check gate before o_o | 13:55 |
| *** amoralej is now known as amoralej|lunch | 13:55 | |
| mwhahaha | reminder the meeting is in 5 mins. if you have somethign you want to talk about please make sure it's on the agenda https://etherpad.openstack.org/p/tripleo-meeting-items | 13:56 |
| *** ffiore has joined #tripleo | 13:56 | |
| mwhahaha | also, i hate daylight savings | 13:56 |
| adarazs|rover | mwhahaha: yay for UTC :) | 13:56 |
| sshnaidm | jpich, can you please paste your "hosts" and "ssh.config.ansible" files? it's in /home/jpichon/.quickstart/ | 13:56 |
| adarazs|rover | sshnaidm: are you trying to debug the devmode failure that you forwarded as an email? | 13:56 |
| sshnaidm | adarazs|rover, I'll look if it's simple to fix | 13:57 |
| adarazs|rover | sshnaidm: ack. | 13:57 |
| *** yprokule has quit IRC | 13:57 | |
| *** yprokule_ is now known as yprokule | 13:57 | |
| adarazs|rover | sshnaidm: let me know if we should open a bug for it. | 13:57 |
| jpich | sshnaidm: No 'hosts' file and ssh.config.ansible appears to be empty | 13:58 |
| *** udesale has joined #tripleo | 13:59 | |
| sshnaidm | jpich, hmm... | 13:59 |
| slagle | sdoran: hi. do you think /tmp is generally safe to use as a config value for ansible's remote_tmp? | 14:00 |
| mwhahaha | #startmeeting tripleo | 14:00 |
| mwhahaha | #topic agenda | 14:00 |
| mwhahaha | * Review past action items | 14:00 |
| mwhahaha | * One off agenda items | 14:00 |
| mwhahaha | * Squad status | 14:00 |
| mwhahaha | * Bugs & Blueprints | 14:00 |
| mwhahaha | * Projects releases or stable backports | 14:00 |
| slagle | sdoran: the context is that we are running ansible as a user that has no home dir, and there are some localhost tasks, so we need to override remote_tmp | 14:00 |
| openstack | Meeting started Tue Nov 7 14:00:36 2017 UTC and is due to finish in 60 minutes. The chair is mwhahaha. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| mwhahaha | * Specs | 14:00 |
| mwhahaha | * open discussion | 14:00 |
| mwhahaha | Anyone can use the #link, #action and #info commands, not just the moderatorǃ | 14:00 |
| mwhahaha | Hi everyone! who is around today? | 14:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| *** openstack changes topic to " (Meeting topic: tripleo)" | 14:00 | |
| openstack | The meeting name has been set to 'tripleo' | 14:00 |
| *** openstack changes topic to "agenda (Meeting topic: tripleo)" | 14:00 | |
| beagles | o/ | 14:01 |
| * mwhahaha watches as everyone disappears | 14:01 | |
| panda|ruck | o/ | 14:01 |
| *** tzumainn has joined #tripleo | 14:01 | |
| ccamacho | hola! | 14:01 |
| fultonj | o/ | 14:01 |
| jkilpatr | o/ | 14:01 |
| abishop | o/ | 14:01 |
| jpich | o/ | 14:01 |
| jkilpatr | should I bring up quickstart commits here or wait for the TripleO CI meeting? | 14:01 |
| oidgar | o/ | 14:01 |
| slagle | hi | 14:02 |
| mwhahaha | jkilpatr: wait until the open discussion in the meeting plz | 14:02 |
| jkilpatr | sure | 14:02 |
| owalsh | o/ | 14:02 |
| jfrancoa | o/ | 14:03 |
| mwhahaha | ok lets get started | 14:03 |
| mwhahaha | #topic review past action items | 14:03 |
| *** openstack changes topic to "review past action items (Meeting topic: tripleo)" | 14:03 | |
| *** catintheroof has quit IRC | 14:03 | |
| mwhahaha | EmilienM to prepare an etherpad for tripleo onboarding session in Sydney - DONE | 14:03 |
| mwhahaha | well i assume it's done since he's in Sydney | 14:03 |
| mwhahaha | marios, matbu, chem provide doc/status of upgrade workflow | 14:03 |
| mwhahaha | marios, matbu, chem - any update? | 14:04 |
| chem | mwhahaha: yes, there is a review hold on | 14:04 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Fix devmode by right order of playbooks https://review.openstack.org/518336 | 14:04 |
| *** akrivoka has joined #tripleo | 14:04 | |
| chem | mwhahaha: https://review.openstack.org/#/c/517916/ | 14:04 |
| chem | mwhahaha: this is the skeleton | 14:04 |
| mwhahaha | cool | 14:05 |
| marios | mwhahaha: o/ thought there wouldn't be a meeting /me late ... not that i know of but sounds like chem knows more | 14:05 |
| mwhahaha | #action team to review upgrades developer docs https://review.openstack.org/#/c/517916/ | 14:05 |
| mwhahaha | gfidente put together issues around multiple service instances | 14:05 |
| chem | mwhahaha: we plan on filling this as we go, maybe adding TODO and merging this one, no sure how to proceed | 14:05 |
| jrist | o/ | 14:05 |
| mwhahaha | chem: yea let's get the skeleton merged and iterate | 14:05 |
| chem | mwhahaha: ack | 14:06 |
| gfidente | mwhahaha I added just two lines into the integration sqad etherpad https://etherpad.openstack.org/p/tripleo-integration-squad-status | 14:06 |
| *** akrivoka has quit IRC | 14:06 | |
| jfrancoa | mwhahaha: chem: yes, that's the best way to proceed I think | 14:06 |
| chem | jfrancoa: ^ | 14:06 |
| *** akrivoka has joined #tripleo | 14:06 | |
| mwhahaha | gfidente: ok thanks | 14:06 |
| mwhahaha | mwhahaha to move medium bugs to queens-3 - DONE | 14:06 |
| gfidente | mwhahaha but I'd like to get some feedback from people about those and how to approach it | 14:07 |
| mwhahaha | i moved all the unstarted medium bugs to queens-3 | 14:07 |
| gfidente | https://etherpad.openstack.org/p/tripleo-integration-squad-status lines 10 > 12 | 14:07 |
| mwhahaha | gfidente: ok probably wouldn't hurt to solicit feedback via the ML | 14:07 |
| gfidente | mwhahaha ack | 14:07 |
| mwhahaha | #action gfidente to send a note requesting feedback on the ML about multiple service instances issues | 14:08 |
| mwhahaha | ci squad to start gathering gate failure metrics and information | 14:08 |
| *** cdearborn_ has joined #tripleo | 14:08 | |
| mwhahaha | weshay, adarazs|rover, panda|ruck: any updates on the metrics? | 14:08 |
| matbu | o/ | 14:09 |
| *** anshul has quit IRC | 14:09 | |
| panda|ruck | mwhahaha: sova has a adedicated page for the gate jobs now | 14:09 |
| panda|ruck | mwhahaha: we have an aggregate RSS to look at the failures too | 14:09 |
| mwhahaha | panda|ruck: cool, please make sure to communicate this information so others can follow | 14:10 |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 14:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730671 | 14:10 |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 14:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 14:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,Triaged] | 14:10 |
| openstack | Launchpad bug 1730671 in tripleo "overcloud installation times out without useful errors in the logs" [Critical,Triaged] | 14:10 |
| panda|ruck | mwhahaha: ok, I'll send an email | 14:10 |
| mwhahaha | panda|ruck: thanks | 14:10 |
| mwhahaha | #topic one off agenda items | 14:10 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-meeting-items | 14:10 |
| *** openstack changes topic to "one off agenda items (Meeting topic: tripleo)" | 14:10 | |
| *** catintheroof has joined #tripleo | 14:10 | |
| mwhahaha | the agenda is empty, so unless anyone has anything they wish to bring up now I'll move on | 14:11 |
| jkilpatr | I just wanted to bring up a quickstart patch. | 14:11 |
| *** bobh has joined #tripleo | 14:11 | |
| mwhahaha | jkilpatr: sure which one | 14:11 |
| jkilpatr | https://review.openstack.org/#/c/497950/20 | 14:11 |
| jkilpatr | I'm not really sure where it belongs. But it needs to live somewhere. | 14:12 |
| jkilpatr | if that has to be my own repo fine, if it's in extras great. Just want a verdict on that. | 14:12 |
| mwhahaha | jkilpatr: personally i think quickstart-extras is the correct place | 14:12 |
| mwhahaha | let's get some more folks to weigh in but it seems like something useful for ci/developers | 14:13 |
| *** dpawar has quit IRC | 14:13 | |
| jkilpatr | yup, trying to get a nonvoting job running so it would be great if I didn'th ave to cherry pick it in. | 14:13 |
| fultonj | where else could it go? | 14:14 |
| panda|ruck | will this be confined to CI or will it be useful for customers too ? | 14:14 |
| trown | seems like there are 2 things in that patch | 14:14 |
| jkilpatr | fultonj, in theory you can pull in arbitrary repos containing extra roles for quickstart. | 14:14 |
| mwhahaha | it's a dev/qe thing for the most part | 14:14 |
| trown | there is an update role, and a disruption role... | 14:14 |
| mwhahaha | customers want no distruptions but we need to be able to test that is true | 14:15 |
| jkilpatr | trown, yes there was no role for trying the various stack settings change possibilities. I can split that out if you like? | 14:15 |
| jkilpatr | it's a semantics issue. | 14:15 |
| mwhahaha | trown: panda|ruck, please take the comments to the review | 14:15 |
| trown | sure | 14:15 |
| panda|ruck | ok | 14:15 |
| mwhahaha | ok moving on to squad status | 14:16 |
| trown | seems like a good idea... we just dont have wall time for even upgrades atm | 14:16 |
| trown | let alone multi upgrades with special code | 14:16 |
| mwhahaha | it could be a periodic | 14:16 |
| mwhahaha | but anyway | 14:16 |
| mwhahaha | #topic Squad status | 14:16 |
| mwhahaha | ci | 14:16 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-ci-squad-scrum | 14:16 |
| mwhahaha | upgrade | 14:16 |
| *** openstack changes topic to "Squad status (Meeting topic: tripleo)" | 14:16 | |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status | 14:16 |
| mwhahaha | containers | 14:16 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-containers-squad-status | 14:16 |
| mwhahaha | integration | 14:16 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-integration-squad-status | 14:16 |
| mwhahaha | ui/cli | 14:16 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status | 14:16 |
| mwhahaha | validations | 14:16 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-validations-squad-status | 14:16 |
| mwhahaha | networking | 14:16 |
| mwhahaha | #link https://etherpad.openstack.org/p/tripleo-networking-squad-status | 14:16 |
| mwhahaha | jrist: ui/cli status missing updates | 14:17 |
| *** yamahata has quit IRC | 14:17 | |
| jrist | sorry, will update | 14:17 |
| mwhahaha | workflows - thrash|biab please provide a status when you have a chance | 14:17 |
| mwhahaha | everyone else it looks like statuses have been updated, please take a look ad review | 14:18 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Send ARA statistics to Graphite server https://review.openstack.org/479882 | 14:18 |
| mwhahaha | moving on | 14:18 |
| mwhahaha | #topic bugs & blueprints | 14:19 |
| mwhahaha | #link https://launchpad.net/tripleo/+milestone/queens-2 | 14:19 |
| mwhahaha | For Queens we currently have 70 (+1) blueprints and about 521 (+24) open bugs. 254 queens-2 and 267 queens-3. | 14:19 |
| *** openstack changes topic to "bugs & blueprints (Meeting topic: tripleo)" | 14:19 | |
| mwhahaha | so it seems last week hasn't been a good week for bugs as we're +24 | 14:19 |
| mwhahaha | also CI has been hosed for some time | 14:19 |
| mwhahaha | the queue keeps failing with timeouts so we need to work on critical bugs | 14:20 |
| mwhahaha | any other bug related items? | 14:20 |
| mwhahaha | sounds like nope | 14:21 |
| mwhahaha | #topic projects releases or stable backports | 14:21 |
| *** openstack changes topic to "projects releases or stable backports (Meeting topic: tripleo)" | 14:21 | |
| *** aufi has quit IRC | 14:21 | |
| mwhahaha | Any stable backports people need eyes on? | 14:21 |
| mwhahaha | I think EmilienM is working through some release issues if they haven't already been resolved | 14:22 |
| mwhahaha | I believe we'll be lining up another release for stable stuff next week depending on the status of previous ones | 14:22 |
| *** aditya_r has quit IRC | 14:23 | |
| mwhahaha | moving on | 14:24 |
| mwhahaha | #topic specs | 14:24 |
| mwhahaha | #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open | 14:24 |
| *** openstack changes topic to "specs (Meeting topic: tripleo)" | 14:24 | |
| *** anshul has joined #tripleo | 14:25 | |
| mwhahaha | please take some time to review the openspecs and if you have an open one with comments, please update it | 14:25 |
| mwhahaha | dsneddon, lhinds, ccamacho: looks like you folks have some open specs with -1s | 14:26 |
| mwhahaha | moving on | 14:27 |
| mwhahaha | #topic open discussion | 14:27 |
| *** openstack changes topic to "open discussion (Meeting topic: tripleo)" | 14:27 | |
| ccamacho | mwhahaha yeah Ill ask you some questions after the meeting thanks | 14:27 |
| *** thrash|biab is now known as thrash | 14:27 | |
| mwhahaha | so CI... | 14:27 |
| mwhahaha | adarazs|rover, panda|ruck: do we have updates on where we're at with CI and why we're running into so many timeouts? | 14:28 |
| *** benone has joined #tripleo | 14:28 | |
| adarazs|rover | mwhahaha: nope. I just opened the bug for it, as I didn't find anything more enlightening after looking around in the logs. | 14:28 |
| adarazs|rover | here's the bug: https://bugs.launchpad.net/tripleo/+bug/1730671 | 14:29 |
| openstack | Launchpad bug 1730671 in tripleo "overcloud installation times out without useful errors in the logs" [Critical,Triaged] | 14:29 |
| mwhahaha | ok so the gate is 24+ hours behind and it seems that we're hitting a timeout consistently on some jobs | 14:29 |
| *** lblanchard has joined #tripleo | 14:29 | |
| mwhahaha | i'll take a look at that bug and see if i can add some more information | 14:30 |
| *** hewbrocc` is now known as hewbrocca | 14:30 | |
| adarazs|rover | mwhahaha: thanks! | 14:30 |
| panda|ruck | I really need to understand why the zuul queue can grow indefinitely. Chasing gate failures is something we certainly have to do, but it looks like a bug that we ad and add changes to a queue that grow the possibility of getting a job to fail and reset everything again | 14:31 |
| mwhahaha | so at this point we're pretty much blocked again in the gate unless we can figure out what's timing out | 14:31 |
| mwhahaha | panda|ruck: it grows because people are approving stuff or rechecking. and when a reset occurs that's another 2+ hours of not advancing | 14:31 |
| mwhahaha | we probably need a stop order on all aprovals until we figure out what is causing the timeouts | 14:32 |
| mwhahaha | unless a patch is going to fix a known blocker bug I think we need to -2 for now | 14:32 |
| panda|ruck | mwhahaha: well, zuul could just say "i'll teke the first five and chenge only them together, so the risk of getting a reset is lower" | 14:32 |
| *** masco has quit IRC | 14:32 | |
| Tengu | hmm there's also the random issue with the volumes | 14:33 |
| panda|ruck | this kind of optimization zuul is doing, doesn't work very well with our rate of failures | 14:33 |
| mwhahaha | well the argument is that we shouldn't have this rate of failures | 14:33 |
| mwhahaha | cause we shouldn't | 14:33 |
| mwhahaha | so anyway i'm going to send a note about not aproving anything else and we may have to clear the queue to right this | 14:34 |
| adarazs|rover | I mean if you have a 99% chance of passing and you try to run it 20 times you're already at only an 80% chance of passing. | 14:34 |
| adarazs|rover | and 99% is quite ideal. | 14:34 |
| *** dhill_ has quit IRC | 14:34 | |
| mwhahaha | #action mwhahaha send a note about CI to ML and propsing no more merging of items not specifically critical CI bugs | 14:35 |
| panda|ruck | we can keep the rate low, putting a lot of effeort, but I don't think the situation is going to improve in the long run | 14:35 |
| jaosorior | Tengu: got logs for it? | 14:35 |
| Tengu | jaosorior: there's two issues about that already. 2s, getting logs. | 14:35 |
| Tengu | jaosorior: Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" and "Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error"" - I regularly hit the first one on my apache review. | 14:36 |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] https://launchpad.net/bugs/1729253 - Assigned to Martin André (mandre) | 14:36 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] https://launchpad.net/bugs/1730111 | 14:36 |
| ccamacho | mwhahaha what about patches/bug fixes already +2 but waiting for merge? | 14:36 |
| ccamacho | I mean +A | 14:36 |
| *** aufi has joined #tripleo | 14:36 | |
| mwhahaha | ccamacho: unless it's a CI bug no we shouldn't recheck | 14:36 |
| ccamacho | ack | 14:37 |
| mwhahaha | ok anything else? | 14:37 |
| mwhahaha | sounds like nope | 14:40 |
| mwhahaha | thanks everyone | 14:40 |
| mwhahaha | #endmeeting | 14:40 |
| *** openstack changes topic to "CI status: Yellow, jobs timing out http://ow.ly/MTTI30gomCe | http://tripleo.org/ | https://docs.openstack.org/tripleo-docs/latest/" | 14:40 | |
| openstack | Meeting ended Tue Nov 7 14:40:38 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:40 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/tripleo/2017/tripleo.2017-11-07-14.00.html | 14:40 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/tripleo/2017/tripleo.2017-11-07-14.00.txt | 14:40 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/tripleo/2017/tripleo.2017-11-07-14.00.log.html | 14:40 |
| *** dhill_ has joined #tripleo | 14:41 | |
| Tengu | jaosorior: I think you already spoke about the iscsi issue some.. hmm.. weeks ago? I think you spoke about a possible service to restart or something like that. | 14:43 |
| Tengu | well, not "the" iscsi isse, more likely "an" iscsi issue | 14:43 |
| jaosorior | oh :/ | 14:43 |
| Tengu | not sure they are the same. | 14:44 |
| Tengu | that one I didn't hit I think. more the "no valid backend" - note that it might be related at some level… | 14:44 |
| *** jcoufal has quit IRC | 14:45 | |
| *** dsariel has joined #tripleo | 14:45 | |
| *** dsariel__ has quit IRC | 14:47 | |
| *** pcaruana has joined #tripleo | 14:47 | |
| openstackgerrit | Lee Yarwood proposed openstack/tripleo-quickstart-extras master: Allow the configuration of virtualbmc on Newton https://review.openstack.org/509218 | 14:50 |
| openstackgerrit | Lee Yarwood proposed openstack/tripleo-quickstart master: Allow the confgiuration of virtualbmc on Newton https://review.openstack.org/509219 | 14:50 |
| *** ramishra has joined #tripleo | 14:50 | |
| *** aufi_ has joined #tripleo | 14:50 | |
| *** amoralej|lunch is now known as amoralej | 14:51 | |
| openstackgerrit | Merged openstack/tripleo-ui master: Fetch stacks only when current plan changes https://review.openstack.org/517024 | 14:51 |
| *** brault has quit IRC | 14:51 | |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart-extras master: Fix tempest html generation file https://review.openstack.org/517906 | 14:54 |
| *** aufi has quit IRC | 14:54 | |
| openstackgerrit | Rajini Karthik proposed openstack/puppet-tripleo stable/ocata: Support for Dell EMC VMAX ISCSI Cinder Driver https://review.openstack.org/518345 | 14:58 |
| openstackgerrit | Jason E. Rist proposed openstack/tripleo-ui stable/pike: Fetch stacks only when current plan changes https://review.openstack.org/518346 | 14:58 |
| *** brault has joined #tripleo | 14:58 | |
| *** artom has quit IRC | 14:59 | |
| * jrist offers slagle a beer for +A ^ | 14:59 | |
| jrist | (backport) | 15:00 |
| honza | slagle: I love your comment about js, I'm going to do code review just like that from now on | 15:00 |
| jrist | "this looks amazing if I knew anything about python or yaml" | 15:00 |
| *** ramishra has quit IRC | 15:00 | |
| honza | +2 | 15:01 |
| *** spectr has joined #tripleo | 15:01 | |
| *** yprokule_ has joined #tripleo | 15:04 | |
| *** ykarel is now known as ykarel|away | 15:06 | |
| openstackgerrit | Rajini Karthik proposed openstack/tripleo-heat-templates stable/ocata: Add support for Dell EMC VMAX ISCSI Backend https://review.openstack.org/518352 | 15:07 |
| *** ramishra has joined #tripleo | 15:07 | |
| *** yprokule has quit IRC | 15:08 | |
| *** yprokule_ is now known as yprokule | 15:08 | |
| *** anshul has quit IRC | 15:08 | |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 15:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730671 | 15:10 |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 15:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 15:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,In progress] - Assigned to Attila Darazs (adarazs) | 15:10 |
| openstack | Launchpad bug 1730671 in tripleo "overcloud installation times out without useful errors in the logs" [Critical,Triaged] | 15:10 |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-common master: Update ceph-ansible playbook path in parameter default for update https://review.openstack.org/515442 | 15:11 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-docs master: Add cleanup checks in the upgrade doc https://review.openstack.org/389609 | 15:11 |
| *** amodi has joined #tripleo | 15:11 | |
| mandre | jaosorior: still need help on the missing yaml file with the prepare command? | 15:13 |
| therve | mandre, Can I remove the alert on 1727406 ? Still missing the pike merge? | 15:13 |
| mandre | therve: yeah, we're still missing the pike backport | 15:14 |
| jaosorior | mandre: yeah, but we can do that tomorrow, no biggie | 15:14 |
| *** ykarel|away has quit IRC | 15:15 | |
| *** moshele has quit IRC | 15:15 | |
| therve | Please review https://review.openstack.org/#/c/518021/ pike backport! | 15:15 |
| *** chlong has joined #tripleo | 15:15 | |
| *** lblanchard has quit IRC | 15:15 | |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-common master: Octavia post deployment mistral wrapper https://review.openstack.org/515402 | 15:15 |
| udesale | bnemec, hi you there? | 15:16 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates stable/pike: Set metric procssing delay for metricd https://review.openstack.org/515684 | 15:17 |
| *** amodi has quit IRC | 15:17 | |
| jrist | honza: can't you just +A since that's a backport? | 15:18 |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-common master: Update ceph-ansible playbook path in parameter default for update https://review.openstack.org/515442 | 15:18 |
| *** suuuper has quit IRC | 15:18 | |
| bnemec | udesale: Yes, I was just reading your email. | 15:18 |
| honza | jrist: doesn't it still need another +2? | 15:19 |
| jrist | I don't think so? | 15:19 |
| jrist | I could be wrong. | 15:19 |
| honza | jrist: A+++ | 15:19 |
| udesale | bnemec, thanks, may be I missing something | 15:19 |
| bnemec | honza: Not if it's proposed by a core. | 15:19 |
| honza | jrist: you jumped the gun :D | 15:19 |
| honza | bnemec: TIL | 15:19 |
| bnemec | honza: jrist: We actually have a policy that explains when it's okay to do expedited approvals: https://specs.openstack.org/openstack/tripleo-specs/specs/policy/expedited-approvals.html | 15:19 |
| * jrist hugs bnemec | 15:20 | |
| *** anshul has joined #tripleo | 15:20 | |
| honza | haha | 15:20 |
| * honza reads | 15:20 | |
| *** adac has quit IRC | 15:20 | |
| honza | jrist: would still need another core, even if i proposed it | 15:21 |
| jrist | ah | 15:21 |
| openstackgerrit | Cédric Jeanneret proposed openstack/puppet-tripleo master: Create dedicated "apache" base profile https://review.openstack.org/513274 | 15:21 |
| Tengu | (rebased) | 15:21 |
| fultonj | beagles: i saw your comment in https://review.openstack.org/#/c/515402 about a /home/mistral issue. what's the issue? | 15:25 |
| fultonj | i can tell you that that directory doesn't exist and won't exist unless something is changed in the mistral package. | 15:26 |
| bnemec | udesale: Were there any errors when you deployed the OVB environment? I'm going to guess that the problem is the path in the extra role environment file. | 15:27 |
| fultonj | i've had ansible looking for it in the past, but have fixed that by passing an override. | 15:27 |
| beagles | fultonj, having trouble sorting out how to execute ansible on the undercloud node - we were using ssh but that isn't tenable as it tries to manage /home/mistral/.ssh | 15:27 |
| bnemec | It looks like quickstart makes all of the template paths absolute, so the relative path there may be problematic. | 15:27 |
| *** mrch has quit IRC | 15:27 | |
| beagles | fultonj, so what we are trying now is a local ansible connection - but we still need to do some TEMP variable modifications since it wants to use /home/mistral/.ansible (or similar) | 15:27 |
| jpich | sshnaidm: Deployment still failed (IPMI failures), but I got further at least - thanks for the quick patch! | 15:29 |
| jpich | sshnaidm: The IPMI failures ("instackenv validator found 2 errors Failure to contact below addresses IPMI Address" etc) seem to be happening for others as well, fwiw | 15:29 |
| sshnaidm | trown, can it be related to ovb playbooks too? ^^ | 15:30 |
| *** aputtur_ has quit IRC | 15:31 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/python-tripleoclient master: Support to drive undercloud deploy via undercloud.conf https://review.openstack.org/511350 | 15:32 |
| trown | sshnaidm: hmm ya... I think we didnt do that before | 15:32 |
| trown | sshnaidm: well we did it in dev mode playbook | 15:32 |
| *** adac has joined #tripleo | 15:34 | |
| *** mwhahaha changes topic to "CI status: RED, jobs timing out http://ow.ly/MTTI30gomCe please do not approve/recheck things not related to critical CI bugs | http://tripleo.org/ | https://docs.openstack.org/tripleo-docs/latest/" | 15:34 | |
| *** bkopilov has joined #tripleo | 15:34 | |
| *** yprokule has quit IRC | 15:34 | |
| jpich | This is using devmode.sh | 15:36 |
| *** ansmith has quit IRC | 15:37 | |
| honza | jpich: do you have +2 on tripleo-ui? do you feel comfortable doing +2 on a tiny backport patch? | 15:37 |
| fultonj | beagles: the playbook should run in a workdir https://github.com/openstack/tripleo-common/blob/master/tripleo_common/actions/ansible.py#L255-L259 | 15:38 |
| udesale | bnemec, yep there was error but I lost it, it was related with polling, I will reproduce it again | 15:38 |
| jpich | honza: I don't but using common sense etc etc, if it's a small backport it seems like something I could help with | 15:38 |
| beagles | fultonj, is there a way to set the ANSIBLE_LOCAL_TEMP and ANSIBLE_REMOTE_TEMP to point into that work directory? | 15:38 |
| udesale | bnemec, does it creates separate stack for extra node? | 15:39 |
| honza | jpich: https://review.openstack.org/#/c/518346/1 | 15:39 |
| honza | jpich: up to you, no pressure | 15:39 |
| honza | jrist: ^^ | 15:39 |
| bnemec | udesale: Yes, exactly. I think that's probably why you have the main nodes but not the extra node. | 15:39 |
| udesale | bnemec, does it create* | 15:39 |
| jrist | jpich: small backport, huge impact | 15:39 |
| fultonj | beagles: i think you can use extra_env_variables for that | 15:39 |
| udesale | bnemec, ok it created only one stack then | 15:40 |
| fultonj | beagles: looks like you passed some here... | 15:40 |
| fultonj | https://review.openstack.org/#/c/515402/12/workbooks/octavia_post.yaml | 15:40 |
| fultonj | line 34 | 15:41 |
| fultonj | I recommend making a dedicated temp dir for these | 15:41 |
| fultonj | with file actions | 15:41 |
| fultonj | not just /tmp | 15:41 |
| beagles | fultonj, okay | 15:42 |
| sshnaidm | bnemec, hi, do you know what is the problem with pep8 in this patch? https://review.openstack.org/#/c/476708/ we need it quite urgently for this sprint.. | 15:42 |
| bnemec | udesale: Maybe try again with the extra node resource_registry entry set to: OS::OVB::BaremetalPorts: /home/centos/.quickstart/openstack-virtual-baremetal/templates/baremetal-ports-extra-node-port-security.yaml | 15:42 |
| beagles | fultonj, actually isn't that a chicken and egg thing | 15:42 |
| beagles | fultonj, or does mistral have file actions? | 15:43 |
| fultonj | beagles: yes, mistral has them | 15:43 |
| fultonj | beagles: https://github.com/openstack/tripleo-common/blob/master/workbooks/ceph-ansible.yaml#L46-L49 | 15:43 |
| fultonj | and at the end... | 15:43 |
| fultonj | https://github.com/openstack/tripleo-common/blob/master/workbooks/ceph-ansible.yaml#L92 | 15:43 |
| fultonj | you probably want one for remote and one for local | 15:44 |
| beagles | fultonj, PERFECT! | 15:44 |
| beagles | fultonj, yeah might be best | 15:44 |
| beagles | ++fultonj | 15:44 |
| bnemec | sshnaidm: There's a parameter description mismatch in the new files. | 15:44 |
| openstackgerrit | Andreas Scheuring proposed openstack/diskimage-builder master: Add zipl element as s390x architecture bootloader https://review.openstack.org/443548 | 15:44 |
| fultonj | beagles: https://github.com/openstack/tripleo-common/blob/master/workbooks/ceph-ansible.yaml#L12-L20 | 15:44 |
| bnemec | sshnaidm: I believe that's fixed in the latest version of the generator tool. You probably just need to regenerate the templates with it. | 15:44 |
| fultonj | ^ you might want to try those other env variables | 15:45 |
| *** toure_biab is now known as toure | 15:45 | |
| beagles | fultonj, k | 15:45 |
| trown | sshnaidm: trying your patch with https://github.com/slagle/traas/blob/master/templates/undercloud.yaml#L40 added to see if it will "just work" | 15:46 |
| sshnaidm | trown, yeah, me too :) | 15:46 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras master: Multinode provision role https://review.openstack.org/516795 | 15:47 |
| *** ccamacho has quit IRC | 15:49 | |
| *** ansmith has joined #tripleo | 15:49 | |
| *** ffiore has quit IRC | 15:51 | |
| sshnaidm | bnemec, I'm not sure it was generated with this tool.. trown ? | 15:52 |
| bnemec | sshnaidm: https://review.openstack.org/#/c/476708/14/ci/environments/network/multiple-nics/README | 15:52 |
| sshnaidm | bnemec, I think it's just copy-paste from tripleo-ci | 15:52 |
| sshnaidm | oh, I see | 15:53 |
| *** agopi has joined #tripleo | 15:53 | |
| *** ffiore has joined #tripleo | 15:53 | |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Do not failed update if no container registry is set https://review.openstack.org/518361 | 15:53 |
| trown | sshnaidm: it was generated | 15:54 |
| trown | bnemec: can I not manually patch it? I have to go refigure out how to use the generator? | 15:56 |
| *** janki has joined #tripleo | 15:56 | |
| bnemec | trown: You could. It's been fixed in the tool so it would still match. | 15:57 |
| bnemec | Regenerating is super easy though (quoth the Doctor). Open the tool, load the existing template directory, click Generate. | 15:57 |
| *** itlinux has joined #tripleo | 15:59 | |
| sshnaidm | pabelanger, do you have any objections to https://review.openstack.org/#/c/517928/ ? | 15:59 |
| *** brault has quit IRC | 16:00 | |
| *** spectr has quit IRC | 16:00 | |
| *** artom has joined #tripleo | 16:00 | |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-heat-templates master: Start using nic-configs from THT repo instead of tripleo-ci https://review.openstack.org/476708 | 16:01 |
| trown | sshnaidm: ok I think ^ should pass pep now (if I read the CI logs correctly) | 16:02 |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-common master: Do not update the container registry if is None https://review.openstack.org/518364 | 16:02 |
| *** oidgar has quit IRC | 16:04 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/puppet-tripleo master: Drop the old skip-metering-database flag https://review.openstack.org/515796 | 16:04 |
| *** adac has quit IRC | 16:05 | |
| *** ebarrera has quit IRC | 16:06 | |
| *** etingof has quit IRC | 16:08 | |
| *** marios has quit IRC | 16:09 | |
| openstackgerrit | Jose Luis Franco proposed openstack-infra/tripleo-ci master: Run overcloud-upgrade tag only in upgrades jobs. https://review.openstack.org/518367 | 16:09 |
| *** dtrainor has quit IRC | 16:09 | |
| openstackgerrit | Jose Luis Franco proposed openstack-infra/tripleo-ci master: Run overcloud-upgrade role only in upgrades jobs. https://review.openstack.org/518367 | 16:10 |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 16:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730671 | 16:10 |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 16:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 16:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,In progress] - Assigned to Attila Darazs (adarazs) | 16:10 |
| openstack | Launchpad bug 1730671 in tripleo "overcloud installation times out without useful errors in the logs" [Critical,Triaged] | 16:10 |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Do not failed update if no container registry is set https://review.openstack.org/518361 | 16:10 |
| *** dtrainor has joined #tripleo | 16:10 | |
| *** trown is now known as trown|brb | 16:11 | |
| *** fragatina has joined #tripleo | 16:11 | |
| udesale | bnemec, yes it was path related issue, it went ahead and created extra stack but with error | 16:13 |
| udesale | bnemec, Create_Failed: Resource CREATE failed: resources[0]: resources.openstack_baremetal_servers.Property error: resources.baremetal_ports.properties: Property private_net not assigned | 16:13 |
| *** trown|brb is now known as trown | 16:14 | |
| *** iranzo has quit IRC | 16:14 | |
| *** yprokule has joined #tripleo | 16:14 | |
| udesale | bnemec, after updating full path, I got above error^^, I am using just OVB, not with quickstart | 16:15 |
| *** aufi_ has quit IRC | 16:15 | |
| bnemec | udesale: Okay, but your primary env file is still the same? | 16:16 |
| udesale | bnemec, I copied it from my quickstart workspace ;), I think quickstart doesn't support extra undercloud like node on OVB | 16:18 |
| bnemec | udesale: Right, so if you're going to do this with standalone OVB I might suggest you start over with https://github.com/cybertron/openstack-virtual-baremetal/blob/master/environments/base.yaml as your base env file. | 16:19 |
| openstackgerrit | Andreas Scheuring proposed openstack/diskimage-builder master: Add zipl element as s390x architecture bootloader https://review.openstack.org/443548 | 16:20 |
| udesale | bnemec, ack | 16:20 |
| bnemec | udesale: Also note that OVB has a --id option on the deploy command that you can use to append the identifiers that the quickstart configuration has, so that will simplify your env file. | 16:20 |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Fix static-inventory option for minor update https://review.openstack.org/514975 | 16:21 |
| *** etingof has joined #tripleo | 16:22 | |
| chem | gfidente: by the way, stable/jewel puppet-ceph check is brocken | 16:22 |
| larsks | My overcloud (containerized) deploy runs all the way through to "CREATE_COMPLETE Stack CREATE completed successfully", but it doesn't actually deploy any services. | 16:22 |
| larsks | There are a bunch of images on the overcloud controller, but no containers actually running. | 16:22 |
| chem | gfidente: trying to see if it need some backport https://review.openstack.org/#/c/518338/ and https://review.openstack.org/#/c/518339/ for instance | 16:22 |
| larsks | It seems like there are two problems here. I'm willing to bet that the root cause is just me mis-configuring something, but shouldn't the installer recognize that nothing actually got started? | 16:23 |
| gfidente | chem curious how it got broken in the first place | 16:24 |
| larsks | It looks like both the docker-puppet-keystone_init_tasks and the docker-puppet-mysql_init_tasks containers failed. | 16:24 |
| udesale | bnemec, do I need add anything extra in base-extra-node.yaml if I start with base.yaml as base env file? | 16:24 |
| chem | gfidente: missing backport of the ruby/puppet env | 16:24 |
| chem | gfidente: (wild guess) | 16:24 |
| matbu | can someone A+ again this one : https://review.openstack.org/#/c/509718/ (looks like its stuck) | 16:25 |
| larsks | jaosorior: ^^^ | 16:25 |
| *** gvrangan has joined #tripleo | 16:25 | |
| bnemec | matbu: It's in the gate, but the queue is extremely long. See the channel topic. | 16:26 |
| *** fragatina has quit IRC | 16:26 | |
| matbu | bnemec: ha k sorry | 16:26 |
| *** thrash is now known as thrash|biab | 16:27 | |
| bnemec | udesale: Set all the parameters in base.yaml, override anything you want in base-extra-node, and then use a deploy command like this: | 16:27 |
| bnemec | bin/deploy.py --quintupleo --name ovb17 --id ovb17 -e env-base.yaml -e environments/all-networks-port-security.yaml -e environments/boot-from-volume.yaml --role environments/base-extra-node.yaml | 16:27 |
| bnemec | That should get you something very close to what the quickstart configuration was. | 16:28 |
| *** psachin has quit IRC | 16:28 | |
| udesale | bnemec, excellent, that was helpful, will try this. I was confused about how it gets other resources even if they are not specified in base.yaml | 16:29 |
| udesale | bnemec, now got it | 16:29 |
| udesale | bnemec, thanks | 16:29 |
| bnemec | udesale: Yeah, the preferred way to configure things is to use the standalone environment files instead of putting everything in one. | 16:30 |
| bnemec | Quickstart's configuration predates that feature though. | 16:30 |
| bnemec | udesale: Also, role files inherit everything from the base environment, which is why the role files are so much shorter. You only need to set the things you want to change. | 16:31 |
| udesale | bnemec, this now looks easier to me. | 16:32 |
| bnemec | udesale: There is documentation of all this too, if you get stuck: http://openstack-virtual-baremetal.readthedocs.io/en/latest/deploy/quintupleo.html | 16:35 |
| bnemec | If anything's missing let me know. | 16:35 |
| *** ramishra has quit IRC | 16:36 | |
| *** pradk has quit IRC | 16:36 | |
| udesale | bnemec, sure following the same, will let you know the my findings | 16:36 |
| bnemec | udesale: Sounds good | 16:37 |
| *** udesale has quit IRC | 16:38 | |
| *** anshul has quit IRC | 16:44 | |
| *** dparkes has quit IRC | 16:45 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-docs master: Upgrades developer documentation layout. https://review.openstack.org/517916 | 16:46 |
| *** abregman has quit IRC | 16:47 | |
| *** anilvenkata has quit IRC | 16:50 | |
| *** moshele has joined #tripleo | 16:51 | |
| *** agurenko has quit IRC | 16:52 | |
| *** pradk has joined #tripleo | 16:52 | |
| *** aputtur_ has joined #tripleo | 16:52 | |
| weshay | ocata to pike upgrades | 16:53 |
| weshay | http://logs.openstack.org/60/518260/1/check/legacy-tripleo-ci-centos-7-containers-multinode-upgrades/cceb41a/logs/subnode-2/var/log/messages.txt.gz#_Nov__7_09_25_18 | 16:53 |
| *** hamzy has quit IRC | 16:55 | |
| *** moshele has quit IRC | 16:56 | |
| *** newmember has quit IRC | 16:58 | |
| *** newmember has joined #tripleo | 16:58 | |
| *** anilvenkata has joined #tripleo | 16:59 | |
| *** gvrangan has quit IRC | 17:04 | |
| *** bobh has quit IRC | 17:05 | |
| *** jfrancoa has quit IRC | 17:05 | |
| mwhahaha | oh that's new, mistral failure on the undercloud install http://logs.openstack.org/20/517920/1/gate/legacy-tripleo-ci-centos-7-nonha-multinode-oooq/1b37ad0/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2017-11-07_16_40_58 | 17:06 |
| * d0ugal looks | 17:06 | |
| mwhahaha | ugh swift | 17:07 |
| mwhahaha | http://logs.openstack.org/20/517920/1/gate/legacy-tripleo-ci-centos-7-nonha-multinode-oooq/1b37ad0/logs/undercloud/var/log/mistral/executor.log.txt.gz#_2017-11-07_16_30_51_712 | 17:07 |
| mwhahaha | d0ugal: -^ | 17:07 |
| d0ugal | mwhahaha: that swift error is not likely to be the real error | 17:07 |
| mwhahaha | oh right that's teh ci log stuff | 17:07 |
| d0ugal | (that specific error always happens and is "expected") | 17:07 |
| d0ugal | The traceback I mean | 17:08 |
| mwhahaha | we shouldn't do that if the overcloud never deploys | 17:08 |
| * mwhahaha makes note to fix quickstart later | 17:08 | |
| *** jpich has quit IRC | 17:08 | |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 17:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730671 | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| *** dsariel has quit IRC | 17:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 17:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 17:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 17:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,In progress] - Assigned to Attila Darazs (adarazs) | 17:10 |
| openstack | Launchpad bug 1730671 in tripleo "overcloud installation times out without useful errors in the logs" [Critical,Triaged] | 17:10 |
| *** oidgar has joined #tripleo | 17:11 | |
| d0ugal | mwhahaha: it looks like the workflow finished, but for some reason the message never made it back. | 17:11 |
| d0ugal | http://logs.openstack.org/20/517920/1/gate/legacy-tripleo-ci-centos-7-nonha-multinode-oooq/1b37ad0/logs/undercloud/var/log/mistral/executor.log.txt.gz#_2017-11-07_16_41_24_308 | 17:11 |
| d0ugal | That is the last action call, a Zaqar queue post with SUCCESS as the status. | 17:12 |
| mwhahaha | :( | 17:12 |
| d0ugal | I never know how to debug these lost messaes | 17:12 |
| *** ffiore has quit IRC | 17:12 | |
| mwhahaha | d0ugal: i guess the question is did zaqar eat the message or did it make it to the clinet | 17:13 |
| d0ugal | mwhahaha: yeah, I can't see any clues in the zaqar logs (they are very short) | 17:14 |
| therve | Hi | 17:15 |
| d0ugal | mwhahaha: but this seems to be when zaqar got it http://logs.openstack.org/20/517920/1/gate/legacy-tripleo-ci-centos-7-nonha-multinode-oooq/1b37ad0/logs/undercloud/var/log/zaqar/zaqar.log.txt.gz#_2017-11-07_16_41_24_337 | 17:15 |
| therve | mwhahaha, Instack doesn't subscribe to Zaqar messages AFAIK | 17:15 |
| d0ugal | the timestamp and queue name match up | 17:15 |
| therve | It polls mistral status | 17:15 |
| d0ugal | therve: oh yeah, you are correct. | 17:15 |
| *** dtantsur is now known as dtantsur|afk | 17:16 | |
| *** mcornea has quit IRC | 17:16 | |
| therve | d0ugal, Look at the timestamp though | 17:16 |
| d0ugal | mwhahaha: okay, then the workflow just took longer than instack was setup to wait - no idea why it would take 360 seconds tho'! | 17:16 |
| therve | It timed out after 10 mins | 17:16 |
| mwhahaha | hrm | 17:16 |
| mwhahaha | do we exclude mistral from the promotion process? maybe a change merged that broken it or something | 17:17 |
| mwhahaha | let me see | 17:17 |
| therve | Looks like it took 10 minutes to upload a tarball to swift | 17:17 |
| bnemec | mwhahaha: There was talk of doing that a while back, but I'm not aware of it actually happening. | 17:17 |
| therve | http://logs.openstack.org/20/517920/1/gate/legacy-tripleo-ci-centos-7-nonha-multinode-oooq/1b37ad0/logs/undercloud/var/log/mistral/executor.log.txt.gz#_2017-11-07_16_30_52_547 | 17:17 |
| *** lucasagomes is now known as lucas-afk | 17:18 | |
| mwhahaha | oh this was for pike | 17:18 |
| *** fragatina has joined #tripleo | 17:18 | |
| mwhahaha | so not that | 17:18 |
| mwhahaha | swift errored | 17:19 |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart-extras master: Move instackenv.json copy to ovb-create-stack task https://review.openstack.org/517026 | 17:19 |
| mwhahaha | http://logs.openstack.org/20/517920/1/gate/legacy-tripleo-ci-centos-7-nonha-multinode-oooq/1b37ad0/logs/undercloud/var/log/mistral/executor.log.txt.gz#_2017-11-07_16_40_56_501 | 17:19 |
| d0ugal | d'oh | 17:19 |
| d0ugal | so we log the swift exception we want and hide the legit error | 17:19 |
| therve | There are kernel errors in messages | 17:20 |
| therve | d0ugal, So there is nothing hidden, it times out correctly | 17:21 |
| d0ugal | therve: right, but the workflow finishes without error - it thinks it was successful afaict | 17:24 |
| *** gvrangan has joined #tripleo | 17:28 | |
| *** gkadam has quit IRC | 17:29 | |
| *** tesseract has quit IRC | 17:30 | |
| mwhahaha | the execitions all say success http://logs.openstack.org/20/517920/1/gate/legacy-tripleo-ci-centos-7-nonha-multinode-oooq/1b37ad0/logs/postci.txt.gz | 17:30 |
| *** janki has quit IRC | 17:30 | |
| mwhahaha | d0ugal, filed a bug for this https://bugs.launchpad.net/tripleo/+bug/1730714 | 17:31 |
| openstack | Launchpad bug 1730714 in tripleo "CI: Undercloud install failed due to timeout while setting up mistral" [Critical,Triaged] | 17:31 |
| *** pchavva has joined #tripleo | 17:31 | |
| *** janki has joined #tripleo | 17:31 | |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-repos master: WIP Implement Proxy Support https://review.openstack.org/518127 | 17:31 |
| openstackgerrit | Merged openstack/diskimage-builder master: Move to a common lock-file directory https://review.openstack.org/514079 | 17:32 |
| janki | HI. Can I get some reviews for https://review.openstack.org/#/c/518260/. its a cherry-pick | 17:32 |
| *** janki has quit IRC | 17:32 | |
| *** salmankhan has quit IRC | 17:33 | |
| *** jmelvin has quit IRC | 17:36 | |
| bnemec | Wow, review begging and then quitting the IRC channel in the same second. | 17:37 |
| bnemec | Not good review karma. | 17:37 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates master: Enable in-cluster loadbalancing of kubernetes API https://review.openstack.org/518375 | 17:38 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates master: Clone kubespray to location accessible by the user https://review.openstack.org/518376 | 17:38 |
| mwhahaha | panda|ruck, adarazs|rover: where is the rss with the failures? Also what's the magical link for just the gate stuff in sova | 17:40 |
| *** tesseract has joined #tripleo | 17:40 | |
| d0ugal | mwhahaha: thanks. | 17:41 |
| mwhahaha | d0ugal: just spotted a 2nd | 17:41 |
| d0ugal | mwhahaha: I feel like the issue is that swift doesn't raise an exception. https://github.com/openstack/tripleo-common/blob/master/tripleo_common/actions/templates.py#L137 | 17:41 |
| mwhahaha | d0ugal: so it's definately a problem with stable/pike | 17:41 |
| d0ugal | oterwise we should see that I think | 17:41 |
| mwhahaha | http://logs.openstack.org/82/517082/2/gate/legacy-tripleo-ci-centos-7-nonha-multinode-oooq/b087058/job-output.txt.gz | 17:41 |
| d0ugal | mwhahaha: k, I can take a look tomorrow | 17:41 |
| mwhahaha | oh wait that one is a different problem | 17:42 |
| d0ugal | uh oh :) | 17:42 |
| mwhahaha | repo problems | 17:42 |
| * mwhahaha sighs | 17:42 | |
| d0ugal | I gotta run for now | 17:42 |
| mwhahaha | k | 17:42 |
| mwhahaha | http://logs.openstack.org/82/517082/2/gate/legacy-tripleo-ci-centos-7-nonha-multinode-oooq/b087058/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2017-11-07_17_22_51 was that one :/ | 17:43 |
| *** shreshtha has quit IRC | 17:44 | |
| *** radeks has joined #tripleo | 17:44 | |
| *** shreshtha has joined #tripleo | 17:44 | |
| *** pradk has quit IRC | 17:44 | |
| *** thrash|biab is now known as thrash | 17:46 | |
| *** salmankhan has joined #tripleo | 17:49 | |
| *** tesseract has quit IRC | 17:49 | |
| *** yprokule has quit IRC | 17:50 | |
| *** pradk has joined #tripleo | 17:51 | |
| *** akrivoka has quit IRC | 17:52 | |
| *** trown is now known as trown|lunch | 17:54 | |
| openstackgerrit | Merged openstack/tripleo-ui stable/pike: Fetch stacks only when current plan changes https://review.openstack.org/518346 | 17:54 |
| lvdombrkr | helo folks, what can mean this error when deploying overcloud? http://paste.openstack.org/raw/625758/ | 17:58 |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart-extras master: Fix tempest html generation file https://review.openstack.org/517906 | 18:00 |
| *** pchavva has quit IRC | 18:05 | |
| *** bobh has joined #tripleo | 18:05 | |
| *** jpena is now known as jpena|off | 18:09 | |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 18:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730671 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730714 | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 18:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 18:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 18:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,In progress] - Assigned to Attila Darazs (adarazs) | 18:10 |
| openstack | Launchpad bug 1730671 in tripleo "overcloud installation times out without useful errors in the logs" [Critical,Triaged] | 18:10 |
| openstack | Launchpad bug 1730714 in tripleo "CI: Undercloud install failed due to timeout while setting up mistral" [Critical,Triaged] | 18:10 |
| *** bobh has quit IRC | 18:10 | |
| *** amoralej is now known as amoralej|off | 18:11 | |
| jaosorior | larsks: how did it fail? | 18:12 |
| *** aputtur_ has quit IRC | 18:18 | |
| *** mary_grace has joined #tripleo | 18:19 | |
| openstackgerrit | Ian Main proposed openstack/tripleo-quickstart-extras master: DNM, update the undercloud install for containers https://review.openstack.org/517444 | 18:26 |
| *** oidgar has quit IRC | 18:28 | |
| *** akrivoka has joined #tripleo | 18:28 | |
| *** achadha has joined #tripleo | 18:28 | |
| *** achadha_ has joined #tripleo | 18:29 | |
| *** agopi has quit IRC | 18:30 | |
| *** panda|ruck is now known as panda|ruck|bbl | 18:30 | |
| *** achadha has quit IRC | 18:33 | |
| openstackgerrit | wes hayutin proposed openstack/instack-undercloud master: DNM, TESTING ONLY https://review.openstack.org/518118 | 18:33 |
| openstackgerrit | Ian Main proposed openstack/tripleo-quickstart master: DNM, update undercloud install options for containers https://review.openstack.org/517445 | 18:36 |
| *** agopi has joined #tripleo | 18:37 | |
| larsks | jaosorior: I'm seeing the same errors from yesterday; "Error getting v2 registry: Get https://192.168.24.1:8787/v2/: http: server gave HTTP response to HTTPS client" | 18:37 |
| larsks | I'm trying to figure out where the registry address is set. | 18:37 |
| mwhahaha | larsks: that's not an actual error | 18:38 |
| mwhahaha | larsks: it falls back, i went looking but that error exists in CI with everything working | 18:38 |
| larsks | mwhahaha: okay. the symptoms are that the deploy completes but doesnt' actually result in any containers running at all. | 18:38 |
| larsks | I'll keep looking to see if there is a different error. | 18:38 |
| mwhahaha | larsks: i ran through that with tonyb the other day and the acutal problem was bad container names | 18:39 |
| mwhahaha | larsks: check that the containers actually get pulled down and that they match what is on the undercloud | 18:40 |
| mwhahaha | larsks: usually it's a container prepare yaml discrepency (also this is a terrible UX) | 18:40 |
| larsks | mwhahaha: agree re: the ux; surely the deployment should realize that nothing was actually deployed :) | 18:40 |
| mwhahaha | well it should fatal on container failing to fetch | 18:40 |
| larsks | Is there a specific error I should be looking for in the logs that would highlite a container name mismatch? | 18:40 |
| mwhahaha | larsks: i don't have the specific error, it'll be around that part with the https/http thing where it says it cannot fetch the container | 18:41 |
| mwhahaha | you might be able to grep 'docker pull' | 18:42 |
| larsks | The only other errors I see in the journal from docker are DELETE errors: "Handler for DELETE /v1.24/containers/docker-puppet-mysql returned error: No such container: docker-puppet-mysql" | 18:43 |
| mwhahaha | larsks: yea not sure, if you put your log somewhere I can take a look | 18:43 |
| larsks | mwhahaha: which logs? Right now I'm looking through the journal on the overcloud controller. | 18:44 |
| mwhahaha | larsks: that one | 18:44 |
| *** aputtur_ has joined #tripleo | 18:44 | |
| larsks | mwhahaha: http://chunk.io/larsks/138dd0fac18343cea1ea74493b4c6d92 | 18:45 |
| *** hamzy has joined #tripleo | 18:45 | |
| mwhahaha | larsks: "AttributeError: 'list' object has no attribute 'get'", | 18:48 |
| larsks | mwhahaha: that's pretty subtle. | 18:49 |
| mwhahaha | i think that was a known issue | 18:49 |
| mwhahaha | thought it was fixed | 18:49 |
| *** derekh has quit IRC | 18:50 | |
| *** marrusl has quit IRC | 18:50 | |
| larsks | Hmm. This was a quickstart deployment yesterday using -R master-tripleo-ci. | 18:50 |
| larsks | Oh! But my tripleo-heat-templates may be a little older. | 18:50 |
| larsks | Is that where the fix was? | 18:50 |
| mwhahaha | probably | 18:51 |
| mwhahaha | volumes = config.get('volumes', [])", | 18:51 |
| larsks | That is what it should look like? Or that is the problem? | 18:52 |
| mwhahaha | that's where it's failing | 18:52 |
| openstackgerrit | Ian Main proposed openstack/tripleo-quickstart-extras master: DNM, update the undercloud install for containers https://review.openstack.org/517444 | 18:53 |
| larsks | That lines seems to still exist in master. | 18:53 |
| mwhahaha | larsks: http://paste.openstack.org/show/625762/ | 18:53 |
| larsks | In docker.py:match_config_volumes | 18:53 |
| mwhahaha | it points to a config problem | 18:53 |
| mwhahaha | i don't think that the fix was for that script | 18:53 |
| mwhahaha | so try updating your THT to currentish | 18:53 |
| larsks | Yup. I want a "Rebase & Deploy" t-shirt. | 18:54 |
| mwhahaha | rebase & deploy & cross fingers | 18:54 |
| openstackgerrit | Ian Main proposed openstack/tripleo-quickstart-extras master: DNM, update the undercloud install for containers https://review.openstack.org/517444 | 18:54 |
| *** itlinux has quit IRC | 18:58 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Fix wrong permission on database during mysql_init tasks. https://review.openstack.org/517909 | 18:59 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Fix cinder-{backup,volume} container with pacemaker https://review.openstack.org/517222 | 18:59 |
| openstackgerrit | Arx Cruz proposed openstack/tripleo-quickstart-extras master: Adding the possibility to pass the log url to tempestmail https://review.openstack.org/518327 | 19:00 |
| *** d0ugal has quit IRC | 19:01 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-common master: Add work_dir for some actions https://review.openstack.org/514784 | 19:01 |
| *** marrusl has joined #tripleo | 19:02 | |
| *** radeks has quit IRC | 19:04 | |
| slagle | dmsimard: are there any plans to package ara? | 19:05 |
| dmsimard | slagle: yes | 19:05 |
| *** bobh has joined #tripleo | 19:06 | |
| dmsimard | slagle: it's already packaged/approved for fedora but I haven't pushed it to stable yet (I pulled it to improve some things and then never got around to finishing it) | 19:06 |
| dmsimard | slagle: it wouldn't work for EPEL due to outdated dependencies but it could be packaged, for example, in the cloud SIG (RDO) | 19:07 |
| dmsimard | https://koji.fedoraproject.org/koji/packageinfo?packageID=24394 | 19:07 |
| *** trown|lunch is now known as trown | 19:07 | |
| slagle | dmsimard: ok, i'd be interested in consuming it from rdo i guess | 19:07 |
| dmsimard | slagle: in what context ? with tripleo-quickstart ? | 19:08 |
| slagle | dmsimard: no. i'd like to integrate it with our ansible actions that are triggered by mistral, so it needs to be packaged | 19:08 |
| dmsimard | slagle: ah, I see | 19:08 |
| dmsimard | It should be easier to do in RDO, especially now that ansible is in extras | 19:09 |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 19:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730671 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730714 | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 19:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 19:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 19:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,In progress] - Assigned to Attila Darazs (adarazs) | 19:10 |
| openstack | Launchpad bug 1730671 in tripleo "overcloud installation times out without useful errors in the logs" [Critical,Triaged] | 19:10 |
| openstack | Launchpad bug 1730714 in tripleo "CI: Undercloud install failed due to timeout while setting up mistral" [Critical,Triaged] | 19:10 |
| dmsimard | slagle: can you add a comment in this bz with the context: https://bugzilla.redhat.com/show_bug.cgi?id=1426193 I'll formally put it on my todo | 19:10 |
| openstack | bugzilla.redhat.com bug 1426193 in Package Review "Review Request: ara - Ansible Run Analysis, Record and visualize Ansible Playbook runs" [Unspecified,On_qa] - Assigned to misc | 19:10 |
| *** gvrangan has quit IRC | 19:10 | |
| *** bobh has quit IRC | 19:11 | |
| slagle | dmsimard: sure | 19:11 |
| dmsimard | thanks | 19:11 |
| *** achadha has joined #tripleo | 19:13 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates master: Add PlaybookVerbosity parameter to overcloud-ceph-ansible https://review.openstack.org/518383 | 19:14 |
| *** achadha_ has quit IRC | 19:18 | |
| *** achadha has quit IRC | 19:18 | |
| *** achadha has joined #tripleo | 19:19 | |
| openstackgerrit | Radomir Dopieralski proposed openstack/tripleo-heat-templates master: Fix rights to local_settings.d for dockerized Horizon https://review.openstack.org/518384 | 19:23 |
| *** achadha has quit IRC | 19:24 | |
| *** amoralej|off is now known as amoralej | 19:28 | |
| *** itlinux has joined #tripleo | 19:34 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: fix tripleo-ui non-secure setup https://review.openstack.org/518043 | 19:40 |
| *** adac has joined #tripleo | 19:43 | |
| *** derekh has joined #tripleo | 19:44 | |
| *** amoralej is now known as amoralej|off | 19:45 | |
| *** agopi has quit IRC | 19:46 | |
| *** salmankhan has quit IRC | 19:47 | |
| openstackgerrit | Ian Main proposed openstack/tripleo-quickstart-extras master: DNM, update the undercloud install for containers https://review.openstack.org/517444 | 19:49 |
| *** derekh has quit IRC | 19:49 | |
| openstackgerrit | Ian Main proposed openstack/tripleo-quickstart master: DNM, update undercloud install options for containers https://review.openstack.org/517445 | 19:49 |
| *** moshele has joined #tripleo | 19:50 | |
| *** achadha has joined #tripleo | 19:51 | |
| *** achadha has quit IRC | 19:52 | |
| *** achadha has joined #tripleo | 19:53 | |
| *** eck` is now known as eck`gone | 19:54 | |
| *** achadha has quit IRC | 19:54 | |
| *** moshele has quit IRC | 19:54 | |
| *** eck`gone is now known as eck` | 19:54 | |
| *** achadha has joined #tripleo | 19:55 | |
| openstackgerrit | Merged openstack/tripleo-docs master: Unify undercloud upgrade steps https://review.openstack.org/508475 | 19:58 |
| *** bobh has joined #tripleo | 19:58 | |
| *** achadha has quit IRC | 19:59 | |
| openstackgerrit | Alan Bishop proposed openstack/tripleo-heat-templates master: Add support for multiple Cinder RBD backends https://review.openstack.org/506715 | 20:00 |
| *** mary_grace has quit IRC | 20:02 | |
| weshay | Slower, sorry!!! | 20:03 |
| weshay | Slower, I'll go fix it | 20:03 |
| *** mary_grace has joined #tripleo | 20:04 | |
| openstackgerrit | Ian Main proposed openstack/tripleo-quickstart-extras master: DNM, update the undercloud install for containers https://review.openstack.org/517444 | 20:08 |
| *** itlinux has quit IRC | 20:09 | |
| Slower | weshay: think I got it there | 20:10 |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 20:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 20:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 20:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 20:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,In progress] - Assigned to Attila Darazs (adarazs) | 20:10 |
| openstackgerrit | Ian Main proposed openstack/tripleo-quickstart-extras master: DNM, update the undercloud install for containers https://review.openstack.org/517444 | 20:10 |
| *** itlinux has joined #tripleo | 20:12 | |
| *** catintheroof has quit IRC | 20:15 | |
| *** etingof has quit IRC | 20:19 | |
| *** jprovazn has quit IRC | 20:26 | |
| *** pkovar has joined #tripleo | 20:27 | |
| *** moshele has joined #tripleo | 20:29 | |
| *** dhill_ has quit IRC | 20:31 | |
| pabelanger | sshnaidm: lets bring it up next week at infra weekly meeting | 20:32 |
| *** pkovar has quit IRC | 20:32 | |
| *** moshele has quit IRC | 20:34 | |
| *** itlinux has quit IRC | 20:37 | |
| *** jmelvin has joined #tripleo | 20:38 | |
| *** dhill_ has joined #tripleo | 20:38 | |
| openstackgerrit | Ian Main proposed openstack/tripleo-quickstart master: DNM, update undercloud install options for containers https://review.openstack.org/517445 | 20:39 |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart-extras master: Add role to setup any centos vm to look like nodepool vm https://review.openstack.org/517411 | 20:40 |
| *** artom has quit IRC | 20:41 | |
| *** akrivoka has quit IRC | 20:42 | |
| *** etingof has joined #tripleo | 20:44 | |
| *** catintheroof has joined #tripleo | 20:46 | |
| *** aputtur_ has quit IRC | 20:50 | |
| *** itlinux has joined #tripleo | 20:50 | |
| *** catintheroof has quit IRC | 20:51 | |
| weshay | pabelanger, FYI.. https://bugs.launchpad.net/tripleo/+bug/1729686 | 20:52 |
| openstack | Launchpad bug 1729686 in tripleo "undercloud oooq subunit file creation creates default failures and should be skipped" [Medium,Triaged] - Assigned to wes hayutin (weshayutin) | 20:52 |
| *** ansmith_ has joined #tripleo | 20:52 | |
| *** ansmith_ has quit IRC | 20:52 | |
| jlabarre | looking at a way of adding a baremetal compute server to a tripleo-quickstart install. Alternatively, a way to use an undercloud & controller in VMs (presumably through VirtualBMC) with a manual install. The install steps presume all virtual or all baremetal | 20:54 |
| *** jkilpatr has quit IRC | 20:54 | |
| arxcruz | mwhahaha: https://review.openstack.org/#/c/517906/ fix the stackviz and tempest.html | 20:56 |
| *** catintheroof has joined #tripleo | 20:58 | |
| weshay | pabelanger, the context being the health reports for tripleo should improve quite a bit | 20:58 |
| dmsimard | slagle: apevec replied in the BZ, it seems there's a bit of red tape | 21:01 |
| pabelanger | weshay: great! we infact talked about that at openstack-health forum session yesterday, it would be great to get them non 100% | 21:02 |
| *** catintheroof has quit IRC | 21:02 | |
| *** chlong has quit IRC | 21:04 | |
| lvdombrkr | folks when tryng update overcloud get error: No valid host was found. No valid host found for resize | 21:05 |
| lvdombrkr | what does it means? | 21:05 |
| slagle | dmsimard: we've had mistral->ansible for a while already | 21:06 |
| lvdombrkr | im not trying scale overcloude, just add tls | 21:06 |
| dmsimard | slagle: I suppose it'd be an ARA or AWX/Tower kind of deal, I don't know, never been through that process | 21:07 |
| *** agopi has joined #tripleo | 21:08 | |
| *** panda|ruck|bbl is now known as panda|ruck | 21:08 | |
| slagle | dmsimard: possibly | 21:09 |
| slagle | dmsimard: it's not something I need to do (ara integration). just thought it might be nice. | 21:09 |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 21:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 21:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 21:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 21:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,In progress] - Assigned to Attila Darazs (adarazs) | 21:10 |
| *** raildo has quit IRC | 21:10 | |
| *** catintheroof has joined #tripleo | 21:11 | |
| dmsimard | slagle: you don't need to convince me :) | 21:11 |
| dmsimard | slagle: I had done the legwork a while back to have the heat ansible agent use ara | 21:12 |
| dmsimard | but I never got around to finishing it, mostly because I know nothing about heat :) | 21:12 |
| dmsimard | https://github.com/openstack/heat-agents/commit/0d52f5b04737ca66418451c1983c02f0ab58f638 | 21:13 |
| *** jkilpatr has joined #tripleo | 21:13 | |
| *** catintheroof has quit IRC | 21:15 | |
| *** ludovic has joined #tripleo | 21:15 | |
| ludovic | Hi everybody, maybe someone can help to resolve a update deployment of my undercloud | 21:17 |
| ludovic | in ocata stable release | 21:18 |
| *** moshele has joined #tripleo | 21:18 | |
| openstackgerrit | Andy Smith proposed openstack/tripleo-heat-templates master: WIP Separate rpc and notify messaging backends https://review.openstack.org/507963 | 21:21 |
| weshay | dmsimard, did you get +2 on elastic-recheck? | 21:21 |
| dmsimard | nope, not yet | 21:22 |
| *** moshele has quit IRC | 21:22 | |
| *** agopi has quit IRC | 21:24 | |
| *** moshele has joined #tripleo | 21:26 | |
| ludovic | Is someone already had this error with undercloud upgrade, INFO: Error: Failed to apply catalog: Execution of '/bin/openstack role list --quiet --format csv' returned 1: Missing value auth-url required for auth plugin password (tried 35, for a total of 170 seconds) ? | 21:28 |
| mwhahaha | ludovic: did you source stackrc? | 21:32 |
| mwhahaha | seems to be auth related not sure what the exact problem is | 21:32 |
| ludovic | yes of course stackrc was sourced | 21:33 |
| *** toure is now known as toure_biab | 21:33 | |
| *** moshele has quit IRC | 21:33 | |
| mwhahaha | ludovic: what are you upgrading to? or was it just minor update | 21:34 |
| ludovic | it's just a minor update ( i just download latest /etc/yum.repos.d/delorean-*-ocata.repo) | 21:35 |
| ludovic | so ocata -> latest ocata repo | 21:36 |
| *** ansmith has quit IRC | 21:37 | |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-common master: Do not update the container registry if is None https://review.openstack.org/518364 | 21:37 |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Fix static-inventory option for minor update https://review.openstack.org/514975 | 21:37 |
| *** salmankhan has joined #tripleo | 21:42 | |
| *** salmankhan has quit IRC | 21:46 | |
| *** dsneddon has quit IRC | 21:51 | |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common master: Validate roles data and network data https://review.openstack.org/508567 | 21:52 |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common master: Add a Get Networks workflow https://review.openstack.org/509419 | 21:52 |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common master: Add UpdateNetworks workflow https://review.openstack.org/513463 | 21:52 |
| *** trown is now known as trown|outtypewww | 21:54 | |
| dmsimard | slagle: if you'd like I can try to do some of the legwork | 21:57 |
| dmsimard | they know me and they know ara :) | 21:57 |
| openstackgerrit | Flavio Percoco proposed openstack/tripleo-common master: Add ansible and pyton-shade to the base image https://review.openstack.org/518402 | 21:58 |
| *** thrash is now known as thrash|g0ne | 22:01 | |
| *** abishop has quit IRC | 22:08 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 22:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 22:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 22:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 22:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,In progress] - Assigned to Attila Darazs (adarazs) | 22:10 |
| *** moshele has joined #tripleo | 22:12 | |
| *** adac has quit IRC | 22:12 | |
| *** eck` is now known as eck`gone | 22:13 | |
| *** ecerquei has quit IRC | 22:17 | |
| *** itlinux has quit IRC | 22:30 | |
| *** ansmith has joined #tripleo | 22:31 | |
| *** achadha has joined #tripleo | 22:37 | |
| *** itlinux has joined #tripleo | 22:41 | |
| *** achadha has quit IRC | 22:42 | |
| *** newmember has quit IRC | 22:42 | |
| *** dsneddon has joined #tripleo | 22:43 | |
| *** cdearborn_ has quit IRC | 22:43 | |
| *** ramishra has joined #tripleo | 22:45 | |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-repos master: Implement Proxy Support https://review.openstack.org/518127 | 22:51 |
| *** jmelvin has quit IRC | 22:57 | |
| *** itlinux has quit IRC | 22:59 | |
| rnoriega | o/ | 23:02 |
| rnoriega | 5 seconds review is for sale! :-) | 23:02 |
| rnoriega | https://review.openstack.org/#/c/511277/ | 23:02 |
| *** moshele has quit IRC | 23:04 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/instack-undercloud master: Fix the expirer cron task to run correctly https://review.openstack.org/517385 | 23:06 |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1727406 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729253 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1729586 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730111 | 23:10 |
| openstack | Launchpad bug 1727406 in tripleo "Zaqar subscriptions failed to report deployment error" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1730477 | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| *** prad has joined #tripleo | 23:10 | |
| openstack | Launchpad bug 1729253 in tripleo "CI issue: Failed to run cinder task ScheduleCreateVolumeTask, No valid backend was found" [Critical,In progress] - Assigned to Martin André (mandre) | 23:10 |
| openstack | Launchpad bug 1729586 in tripleo "CI: rdocloud node randomly going offline during jobs" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 23:10 |
| openstack | Launchpad bug 1730111 in tripleo "Volume service hostgroup@tripleo_iscsi failed to start.: CappedVersionUnknown: Unrecoverable Error" [Critical,Triaged] | 23:10 |
| openstack | Launchpad bug 1730477 in tripleo "legacy-puppet-syntax-3 job missing on instack-undercloud stable/newton branch" [Critical,In progress] - Assigned to Attila Darazs (adarazs) | 23:10 |
| *** ludovic has quit IRC | 23:11 | |
| *** prad has quit IRC | 23:21 | |
| *** bobh has quit IRC | 23:32 | |
| *** bobh has joined #tripleo | 23:32 | |
| *** bobh has quit IRC | 23:37 | |
| *** achadha has joined #tripleo | 23:38 | |
| *** d0ugal has joined #tripleo | 23:40 | |
| *** achadha_ has joined #tripleo | 23:40 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Fixes InternalApi Heat network resource https://review.openstack.org/512397 | 23:42 |
| *** achadha has quit IRC | 23:43 | |
| *** achadha_ has quit IRC | 23:44 | |
| openstackgerrit | Merged openstack/puppet-tripleo master: Make keystone notification topics configurable https://review.openstack.org/516969 | 23:49 |
| *** rodrigods has quit IRC | 23:51 | |
| *** rodrigods has joined #tripleo | 23:51 | |
| *** salmankhan has joined #tripleo | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!