| honza | jrist: you said latest master, right? I just nuked my node_modules and reinstalled, no issues; what node and npm versions? | 00:01 |
|---|---|---|
| honza | jrist: dumb question: do you have any local changes? | 00:02 |
| *** achadha has joined #tripleo | 00:04 | |
| *** achadha has quit IRC | 00:04 | |
| openstackgerrit | Dan Sneddon proposed openstack/tripleo-heat-templates master: Remove External network from Networker role NIC configs https://review.openstack.org/508360 | 00:05 |
| *** achadha has joined #tripleo | 00:05 | |
| *** tzumainn has quit IRC | 00:05 | |
| *** catintheroof has quit IRC | 00:06 | |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 00:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 00:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 00:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 00:10 |
| *** gbarros has joined #tripleo | 00:15 | |
| *** pmannidi has joined #tripleo | 00:17 | |
| *** gbarros has quit IRC | 00:17 | |
| *** mrunge_ has joined #tripleo | 00:25 | |
| *** mrunge has quit IRC | 00:26 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] gate testing https://review.openstack.org/508367 | 00:26 |
| *** gbarros has joined #tripleo | 00:28 | |
| *** rcernin has quit IRC | 00:29 | |
| *** ecerquei has joined #tripleo | 00:31 | |
| openstackgerrit | Aditya Vaja proposed openstack/tripleo-heat-templates master: added level of indirection causes incorrect hiera config https://review.openstack.org/508369 | 00:36 |
| *** itlinux has joined #tripleo | 00:38 | |
| *** aputtur has joined #tripleo | 00:41 | |
| *** threestrands has joined #tripleo | 00:52 | |
| *** threestrands has joined #tripleo | 00:52 | |
| *** brian83 has quit IRC | 00:54 | |
| *** limao has joined #tripleo | 00:59 | |
| *** dpawar has joined #tripleo | 01:00 | |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates master: [WIP] logging: merge fluentd-client and fluentd-base https://review.openstack.org/501457 | 01:05 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates master: [WIP] logging: use service_config_settings for fluentd https://review.openstack.org/501458 | 01:05 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates master: [WIP] logging: remove fluentd integration from common/services.yaml https://review.openstack.org/502180 | 01:05 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates master: [WIP] logging: update default log format to handle gnocchi https://review.openstack.org/502181 | 01:05 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates master: [WIP] logging: add fluentd configuration to additional services https://review.openstack.org/502182 | 01:05 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/puppet-tripleo master: [WIP] logging: support service_config_settings configuration mechanism https://review.openstack.org/500935 | 01:08 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/puppet-tripleo master: [WIP] logging: teach fluentd.pp about fluentd_service_user https://review.openstack.org/502330 | 01:08 |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 01:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 01:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 01:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 01:10 |
| *** ggillies has quit IRC | 01:21 | |
| *** gbarros has quit IRC | 01:22 | |
| *** aputtur has quit IRC | 01:22 | |
| *** ggillies has joined #tripleo | 01:23 | |
| *** lblanchard has joined #tripleo | 01:24 | |
| *** lblanchard has quit IRC | 01:36 | |
| *** dprince has quit IRC | 01:37 | |
| *** itlinux has quit IRC | 01:39 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] gate testing https://review.openstack.org/508367 | 01:42 |
| *** fzdarsky_ has joined #tripleo | 01:53 | |
| *** fzdarsky has quit IRC | 01:56 | |
| *** dpawar has quit IRC | 02:06 | |
| *** dpawar has joined #tripleo | 02:07 | |
| *** pmannidi has quit IRC | 02:08 | |
| *** pmannidi has joined #tripleo | 02:08 | |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 02:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 02:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 02:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 02:10 |
| *** pmannidi has quit IRC | 02:13 | |
| *** achadha_ has joined #tripleo | 02:18 | |
| *** achadha_ has quit IRC | 02:19 | |
| *** achadha has quit IRC | 02:20 | |
| *** rcernin has joined #tripleo | 02:21 | |
| *** achadha has joined #tripleo | 02:22 | |
| *** itlinux has joined #tripleo | 02:24 | |
| *** achadha has quit IRC | 02:26 | |
| *** achadha has joined #tripleo | 02:27 | |
| *** ramishra has joined #tripleo | 02:33 | |
| *** achadha has quit IRC | 02:34 | |
| *** itlinux has quit IRC | 02:50 | |
| *** dpawar has quit IRC | 02:51 | |
| *** dpawar has joined #tripleo | 02:52 | |
| *** limao has quit IRC | 02:55 | |
| *** limao has joined #tripleo | 02:55 | |
| *** gvrangan has joined #tripleo | 02:57 | |
| *** gvrangan has quit IRC | 03:01 | |
| *** dpawar has quit IRC | 03:04 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] gate testing https://review.openstack.org/508367 | 03:09 |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 03:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 03:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 03:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 03:10 |
| *** dpawar has joined #tripleo | 03:16 | |
| *** aditya_r has joined #tripleo | 03:20 | |
| *** gkadam has joined #tripleo | 03:26 | |
| *** rlandy has quit IRC | 03:37 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-common master: WIP Action for the UI to prepare container image parameters https://review.openstack.org/503296 | 03:45 |
| *** udesale has joined #tripleo | 03:48 | |
| *** ykarel has joined #tripleo | 03:50 | |
| *** mdnadeem has joined #tripleo | 03:50 | |
| jrist | honza: no local changes. might be the node or npm version: npm 5.4.2, node v6.0.0 | 03:53 |
| jrist | honza: no local changes | 03:53 |
| *** dbecker has quit IRC | 03:53 | |
| *** links has joined #tripleo | 04:02 | |
| *** dbecker has joined #tripleo | 04:08 | |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 04:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 04:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 04:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 04:10 |
| *** achadha has joined #tripleo | 04:21 | |
| *** etingof has quit IRC | 04:22 | |
| *** daidv has quit IRC | 04:27 | |
| *** daidv has joined #tripleo | 04:28 | |
| *** etingof has joined #tripleo | 04:29 | |
| *** pmannidi has joined #tripleo | 04:39 | |
| *** achadha has quit IRC | 04:46 | |
| *** dparkes has joined #tripleo | 04:52 | |
| *** marios has joined #tripleo | 04:54 | |
| *** jaganathan has joined #tripleo | 04:55 | |
| *** psachin has joined #tripleo | 04:56 | |
| *** etingof has quit IRC | 04:58 | |
| *** limao has quit IRC | 05:03 | |
| *** limao has joined #tripleo | 05:03 | |
| *** iranzo has joined #tripleo | 05:05 | |
| *** iranzo has joined #tripleo | 05:05 | |
| *** janki has joined #tripleo | 05:06 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 05:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 05:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 05:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 05:10 |
| *** pdeore has joined #tripleo | 05:12 | |
| *** ratailor has joined #tripleo | 05:14 | |
| *** jfrancoa has joined #tripleo | 05:15 | |
| *** rcernin has quit IRC | 05:15 | |
| Tengu | hello there! Small question: is there a way to configure logging in order to send all logs to a centralized rsyslog or ELK? I heard about "oslo", but git grep doesn't show much in pike heat templates. I also found openstack-tripleo-heat-templates/environments/logging-environment.yaml - dunno what it exactly does… | 05:15 |
| *** mrunge_ is now known as mrunge | 05:18 | |
| *** pcaruana has joined #tripleo | 05:24 | |
| *** atoth has quit IRC | 05:26 | |
| *** atoth has joined #tripleo | 05:26 | |
| *** pgadiya has joined #tripleo | 05:27 | |
| *** pcaruana has quit IRC | 05:29 | |
| Tengu | ah, I think I have it, reading the following blueprint: https://specs.openstack.org/openstack/tripleo-specs/specs/newton/tripleo-opstools-centralized-logging.html I wasn't far with the heat environment file I found. | 05:36 |
| *** psahoo has joined #tripleo | 05:38 | |
| *** jprovazn has joined #tripleo | 05:44 | |
| *** achadha has joined #tripleo | 05:45 | |
| *** artom has joined #tripleo | 05:52 | |
| *** rcernin has joined #tripleo | 05:52 | |
| *** artom has quit IRC | 05:52 | |
| *** artom has joined #tripleo | 05:53 | |
| *** artom has quit IRC | 05:53 | |
| *** artom has joined #tripleo | 05:54 | |
| *** achadha has quit IRC | 05:54 | |
| *** artom has quit IRC | 05:54 | |
| *** artom has joined #tripleo | 05:55 | |
| *** limao has quit IRC | 05:55 | |
| *** artom has quit IRC | 05:55 | |
| *** artom has joined #tripleo | 05:56 | |
| *** artom has quit IRC | 05:56 | |
| *** artom has joined #tripleo | 05:57 | |
| *** limao has joined #tripleo | 05:58 | |
| *** paramite has joined #tripleo | 06:05 | |
| *** ratailor is now known as rtailor | 06:06 | |
| *** etingof has joined #tripleo | 06:09 | |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 06:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 06:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 06:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 06:10 |
| *** dpawar has quit IRC | 06:26 | |
| *** dpawar has joined #tripleo | 06:26 | |
| *** mdnadeem has quit IRC | 06:30 | |
| *** artom has quit IRC | 06:42 | |
| *** artom has joined #tripleo | 06:43 | |
| *** artom has quit IRC | 06:44 | |
| *** artom has joined #tripleo | 06:45 | |
| *** artom has quit IRC | 06:45 | |
| *** mdnadeem has joined #tripleo | 06:45 | |
| *** artom has joined #tripleo | 06:46 | |
| *** artom has quit IRC | 06:46 | |
| *** artom has joined #tripleo | 06:47 | |
| *** artom has quit IRC | 06:47 | |
| *** artom has joined #tripleo | 06:48 | |
| *** artom has quit IRC | 06:48 | |
| *** mdnadeem_ has joined #tripleo | 06:48 | |
| *** artom has joined #tripleo | 06:49 | |
| *** artom has quit IRC | 06:49 | |
| *** artom has joined #tripleo | 06:50 | |
| *** artom has quit IRC | 06:50 | |
| *** ebarrera has joined #tripleo | 06:51 | |
| *** ebarrera has joined #tripleo | 06:51 | |
| *** artom has joined #tripleo | 06:51 | |
| *** mdnadeem has quit IRC | 06:51 | |
| *** alee has quit IRC | 06:51 | |
| *** artom has quit IRC | 06:51 | |
| *** artom has joined #tripleo | 06:52 | |
| *** artom has quit IRC | 06:52 | |
| *** jtomasek has joined #tripleo | 06:53 | |
| *** artom has joined #tripleo | 06:53 | |
| *** artom has quit IRC | 06:54 | |
| *** artom has joined #tripleo | 06:55 | |
| *** artom has quit IRC | 06:55 | |
| *** artom has joined #tripleo | 06:56 | |
| *** jlinkes has joined #tripleo | 06:56 | |
| *** artom has quit IRC | 06:56 | |
| *** artom has joined #tripleo | 06:57 | |
| bandini | so do we need to recheck or just be patient with jobs? (e.g. https://review.openstack.org/#/c/506914/) | 06:57 |
| *** artom has quit IRC | 06:57 | |
| *** artom has joined #tripleo | 06:58 | |
| *** artom has quit IRC | 06:58 | |
| *** artom has joined #tripleo | 06:59 | |
| *** shardy_afk is now known as shardy | 07:00 | |
| shardy | bandini: doesn't look like any jobs have run since yesterday according to http://tripleo.org/cistatus.html, and the channel status doesn't look encouraging.. | 07:01 |
| shardy | lets see what alert bugs we have | 07:01 |
| bandini | shardy: yeah from the zuulv3 mails I am kind of getting that this should all just work? | 07:03 |
| *** artom has quit IRC | 07:03 | |
| shardy | Hmm https://bugs.launchpad.net/tripleo/+bug/1720220 looks related, and according to weshay both reviews have merged | 07:03 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 07:03 |
| shardy | bandini: heh, yeah I'm still waking up here :) | 07:03 |
| * shardy checks ml | 07:03 | |
| bandini | shardy: ehehe. good morning! ;) | 07:03 |
| *** pgadiya has quit IRC | 07:04 | |
| *** pcaruana has joined #tripleo | 07:04 | |
| *** dparkes has quit IRC | 07:06 | |
| *** pmannidi has quit IRC | 07:07 | |
| shardy | bandini: https://review.openstack.org/#/c/474533/ is the latest gate run I can find and it still looks pretty broken, but I guess recheck your patch and we can follow the progress & see what happens :) | 07:07 |
| bandini | shardy: I let me try that. thanks! :) | 07:07 |
| *** aditya_r has quit IRC | 07:08 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 07:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719525 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 07:10 |
| openstack | Launchpad bug 1719525 in tripleo "CI: Zaqar error "404 not found" in undercloud" [Critical,Triaged] - Assigned to Thomas Herve (therve) | 07:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 07:10 |
| *** jpich has joined #tripleo | 07:12 | |
| *** mrch has joined #tripleo | 07:12 | |
| *** florianf has joined #tripleo | 07:16 | |
| *** ffiore has joined #tripleo | 07:17 | |
| shardy | mandre: Hey when you get a moment can you pls check the mail from janki on the ml re /var/lib/config-data vs /var/lib/config-data/puppet-generated - it seems we've got some inconsistent config volume references which breaks the generation of TRIPLEO_CONFIG_HASH in docker-puppet.py | 07:18 |
| shardy | we can probably add a workaround in docker-puppet, but I recall we talked about removing the duplicate dirs here at some point, so can we just do that? | 07:18 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates stable/pike: Containerized Fluentd client https://review.openstack.org/507506 | 07:19 |
| shardy | janki: lets wait for feedback from mandre, but my initial reaction is that the services should only reference /var/lib/config-data/<service> | 07:20 |
| shardy | Or we switch all services to the puppet-generated dir, and adjust the config volume logic in docker-puppet to match | 07:21 |
| shardy | but they should all be consistent | 07:21 |
| bandini | oh so http://zuulv3.openstack.org/ is the url now to check for stuff | 07:24 |
| janki | shardy, mandre I like second option. I tested option 1 locally and its working fine | 07:28 |
| Tengu | hello! small question: if I read correctly the heat stuff concerning fluentd, the fluentd repository isn't activated, right? Hence, I'm wondering how fluentd might be installed, especially since "yum search fluentd" doesn't return any package… ? | 07:29 |
| janki | shardy, consecutive observation - after changing hash manually and running paunch, the container is in restarting state and never fully starts | 07:29 |
| janki | could this be because of manual intervention? | 07:30 |
| shardy | janki: Well it probably means the service in the container isn't starting, so the next step will be to figure out if the manual config changes caused it | 07:30 |
| shardy | just changing the hash shouldn't make that happen | 07:30 |
| shardy | brb | 07:30 |
| *** shardy is now known as shardy_afk | 07:30 | |
| *** ccamacho has joined #tripleo | 07:37 | |
| *** jpena|off is now known as jpena | 07:39 | |
| *** aditya_r has joined #tripleo | 07:41 | |
| *** egonzalez has joined #tripleo | 07:44 | |
| *** abregman has joined #tripleo | 07:48 | |
| *** threestrands has quit IRC | 07:53 | |
| *** verdurin has quit IRC | 07:54 | |
| *** abregman is now known as abregman|afk | 07:58 | |
| *** shardy_afk is now known as shardy | 07:58 | |
| *** mcornea has joined #tripleo | 07:58 | |
| *** mdnadeem_ has quit IRC | 08:00 | |
| *** daidv has quit IRC | 08:03 | |
| *** daidv has joined #tripleo | 08:04 | |
| *** jprovazn has quit IRC | 08:07 | |
| *** abregman|afk is now known as abregman | 08:09 | |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 08:10 |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 08:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 08:10 |
| *** limao has quit IRC | 08:12 | |
| *** limao has joined #tripleo | 08:12 | |
| lvdombrkr | shardy: can you give me some advice how to trobolshoot my yesterday issue? | 08:13 |
| *** ykarel is now known as ykarel|lunch | 08:14 | |
| shardy | lvdombrkr: you need to check your environment, as it sounds like you aren't mapping OS::TripleO::Compute::Net::SoftwareConfig to a template that configures your nics | 08:15 |
| shardy | lvdombrkr: you can check this by running openstack stack environment show overcloud | tee overcloud_env.txt | 08:15 |
| shardy | then look in overcloud_env.txt for OS::TripleO::Compute::Net::SoftwareConfig | 08:15 |
| shardy | lvdombrkr: I suggest you debug the networking before attempting to debug the puppet getting stuck, as it may be due to the broken networking | 08:16 |
| *** anshul has joined #tripleo | 08:17 | |
| *** limao has quit IRC | 08:17 | |
| lvdombrkr | shardy:OS::TripleO::Compute::Net::SoftwareConfig is mapped i checked, ok will double check network config | 08:22 |
| shardy | lvdombrkr: Ok, but yesterday you said /etc/os-net-config/config.json was empty, which should not be the case if the Net::SoftwareConfig resource is correctly configured | 08:22 |
| openstackgerrit | Merged openstack/puppet-pacemaker master: Add meta attribute support to stonith manifests https://review.openstack.org/506919 | 08:23 |
| *** abregman is now known as abregman|afk | 08:26 | |
| mandre | shardy, janki: ack, I'll read the email thread and share my thoughts | 08:30 |
| *** limao has joined #tripleo | 08:30 | |
| shardy | mandre: thanks :) | 08:31 |
| shardy | based on the outcome we should probably add a new validation to CI to ensure all services conform to a pattern that will work with updates | 08:32 |
| *** suuuper has joined #tripleo | 08:33 | |
| mandre | janki: IIRC /var/lib/config-data/<config_volume> and /var/lib/config-data/puppet-generated/<config_volume> are always updated together, so we always have a new hash when the content of var/lib/config-data/puppet-generated/<config_volume> is updated | 08:34 |
| mandre | let me double check | 08:34 |
| shardy | mandre: yeah but the problem is docker-puppet.py has logic to detect if a service references /var/lib/config-data/<foo> | 08:36 |
| shardy | that doesn't consider /var/lib/config-data/puppet-generated/ - so either service templates should never reference /var/lib/config-data/puppet-generated/ as their config volume, or we need to add /var/lib/config-data/puppet-generated/ to docker-puppet? | 08:37 |
| *** jaosorior has joined #tripleo | 08:37 | |
| shardy | Was /var/lib/config-data/puppet-generated/ only for debugging, e.g can we just mandate that service templates reference /var/lib/config-data/thing instead? | 08:37 |
| mandre | shardy: the puppet-generated dir serves two purposes, it allows us to collect all meaningful config files in CI log collection without killing it, and gives us a way to copy to the containers just the files its needs without having to worry we missed a bind-mount | 08:40 |
| shardy | mandre: Ok, so we should perhaps just make docker-puppet look at both trees? | 08:41 |
| *** etingof has quit IRC | 08:41 | |
| mandre | shardy: I'm not sure I understand the issue, let me re-read the email once again | 08:41 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/docker/docker-puppet.py#L86 | 08:42 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/docker/docker-puppet.py#L362 | 08:42 |
| shardy | mandre: the issue is the config hash isn't generated if the service template only references /var/lib/config-data/puppet-generated/ | 08:42 |
| shardy | because we only consider /var/lib/config-data as the config_prefix for match_config_volume | 08:43 |
| shardy | that's because /var/lib/config-data/puppet-generated/ didn't exist when I wrote that code :) | 08:43 |
| mandre | shardy: if the content of the puppet-generated dir changed, that means that the content of the /var/lib/config-data/<service> also changed and we have a new hash, no? | 08:43 |
| shardy | mandre: well, no - but if it's valid for services to only reference /var/lib/config-data/puppet-generated/ and not /var/lib/config-data/<service> then we'll have to fix docker-puppet.py so it generates the hash in both cases | 08:44 |
| shardy | the issue is we don't generate any hash at all if the service only references /var/lib/config-data/puppet-generated/ | 08:44 |
| *** tosky has joined #tripleo | 08:44 | |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/docker/docker-puppet.py#L372 | 08:45 |
| shardy | we can probably just add a conditional there and try to match with puppet_generated_config_prefix or something | 08:45 |
| mandre | shardy: how is it even possible that we have a /var/lib/config-data/puppet-generated/<service> dir without the matching /var/lib/config-data/<service> one? | 08:45 |
| shardy | mandre: the directories exist, we just don't calculate the hash | 08:46 |
| shardy | if the service template only references /var/lib/config-data/puppet-generated/ | 08:46 |
| shardy | check the code, the match_config_volume logic doesn't currently handle that case | 08:46 |
| mandre | shardy: but we always generate the config file on update, don't we? so we should have a /var/lib/config-data dir | 08:47 |
| mandre | the content of the puppet-generated dir is just a subset of the /var/lib/config-data directory | 08:47 |
| shardy | mandre: yes, but match_config_volume looks at the mounted volumes in the container json definition | 08:47 |
| shardy | so if it doesn't have a volume referencing /var/lib/config-data/$service, we don't think the config exists for the service | 08:48 |
| shardy | or rather they aren't associated | 08:48 |
| * mandre is reading the cod3e | 08:48 | |
| shardy | this was added because some containers don't have a 1-1 mapping between the container name and the config volume they mount | 08:49 |
| shardy | so I added a check on the actual volumes that are mounted | 08:49 |
| shardy | probably we can just do match_config_volume twice, to catch this case janki hit | 08:49 |
| shardy | I initially thought we could just always bind mount /var/lib/config/data/$service, but forgot about the issue of bind mounting files & your kolla merge stuff | 08:50 |
| * shardy hacks on a patch | 08:50 | |
| mandre | shardy: oh I see now... I thought we generated the hash each time, I didn't know we did it only for the volumes that are referenced | 08:50 |
| *** gkadam has quit IRC | 08:51 | |
| shardy | mandre: yeah - well we generate the md5sum file, but we only add the hash to the container environment if it mounts a config volume | 08:51 |
| shardy | otherwise it doesn't care about the config that changed | 08:51 |
| shardy | let me push a patch and see if you think it makes sense | 08:51 |
| mandre | shardy: do we need that check, though? don't we generate the same hash if the config didn't change? | 08:52 |
| *** gfidente has joined #tripleo | 08:52 | |
| shardy | mandre: yes we do, the issue is we don't put any hash at all in the container environment, unless it mounts something under /var/lib/config-data/$service | 08:53 |
| shardy | if you only mount /var/lib/config-data/puppet-generated/thing no hash gets added | 08:53 |
| shardy | like if you mount /var/lib/config-data/neutron there's a hash, but if it's /var/lib/config-data/puppet-generated/neutron then it's not | 08:54 |
| shardy | added to the container environment | 08:54 |
| *** gkadam has joined #tripleo | 08:55 | |
| mandre | shardy: gotcha | 08:55 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Also match config volumes for /var/lib/config-data/puppet-generated/ https://review.openstack.org/508450 | 08:57 |
| shardy | mandre, janki: ^^ see what you think (untested) | 08:57 |
| *** ykarel|lunch is now known as ykarel | 08:59 | |
| mandre | shardy: what about just modifying match_config_volume() to look for prefix or prefix/puppet_generated ? | 08:59 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Also match config volumes for /var/lib/config-data/puppet-generated/ https://review.openstack.org/508450 | 08:59 |
| * janki reading backlog | 09:00 | |
| shardy | mandre: yeah that would work too | 09:00 |
| *** aditya_ra has joined #tripleo | 09:00 | |
| *** gkadam has quit IRC | 09:00 | |
| *** gkadam has joined #tripleo | 09:01 | |
| shardy | mandre: actually yeah this needs more testing - it'll do a false match and return a volume of "puppet-generated" I think | 09:01 |
| *** sri_ has joined #tripleo | 09:03 | |
| *** aditya_r has quit IRC | 09:03 | |
| janki | shardy, mandre this should work. we can also just check for ../config-data/puppet-generated/ as all services need to mount this for the service to start properly | 09:04 |
| *** ffiore has quit IRC | 09:09 | |
| *** abregman|afk is now known as abregman | 09:10 | |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 09:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 09:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 09:10 |
| *** ffiore has joined #tripleo | 09:10 | |
| *** bogdando has joined #tripleo | 09:11 | |
| *** aditya_ra has quit IRC | 09:12 | |
| *** achadha has joined #tripleo | 09:13 | |
| *** abregman is now known as abregman|afk | 09:15 | |
| *** achadha has quit IRC | 09:18 | |
| janki | shardy, config_volume would be "puppet-generated" and not "NOne" so the logic won't work | 09:18 |
| shardy | janki: yeah, I just mentioned that above and marked it WIP | 09:19 |
| Tengu | anyone "fluent" with the logging configuration with tripleO ? :) | 09:19 |
| Tengu | I'm having some questions. | 09:19 |
| shardy | janki: I'm testing locally and will update, thanks for the feedback | 09:20 |
| janki | shardy, ack | 09:20 |
| *** ffiore_ has joined #tripleo | 09:20 | |
| janki | shardy, sorry. missed your comment above. | 09:20 |
| *** gkadam_ has joined #tripleo | 09:21 | |
| *** ffiore has quit IRC | 09:21 | |
| *** achadha has joined #tripleo | 09:23 | |
| *** gkadam has quit IRC | 09:24 | |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Fix cold/live migration network config https://review.openstack.org/499387 | 09:24 |
| paramite | Hi guys, I'm getting "-1 Zuul" on pike backports. Is this legit or CI is broken? | 09:28 |
| shardy | paramite: see $topic - there are some ongoing issues due to the transition to zuul v3 | 09:29 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: App config and I18n selectors refactor https://review.openstack.org/494141 | 09:30 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Refactor services/utils into selectors https://review.openstack.org/506661 | 09:30 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Refactor Ironic and Ironic Inspector services https://review.openstack.org/506662 | 09:30 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Refactor ZaqarWebSocketService functions into thunks https://review.openstack.org/506667 | 09:30 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Refactor KeystoneApiService into thunks https://review.openstack.org/506663 | 09:30 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Refactor MistralApiService to thunks https://review.openstack.org/506664 | 09:30 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Refactor HeatApiService to thunks https://review.openstack.org/506665 | 09:30 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Refactor SwiftApiService into thunks https://review.openstack.org/506666 | 09:30 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Add redux-mock-store dependency https://review.openstack.org/508465 | 09:30 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Delete services/utils https://review.openstack.org/508466 | 09:30 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Remove unnecessary store imports from tests https://review.openstack.org/508467 | 09:30 |
| paramite | shardy, ah, thanks | 09:31 |
| *** panda|off is now known as panda | 09:31 | |
| shardy | Tengu: larsks is probably best person to ask about the logging things but he likely won't be around for a few hours yet, so feel free to ask the questions and we'll try to help | 09:32 |
| *** achadha has quit IRC | 09:37 | |
| *** salmankhan has joined #tripleo | 09:41 | |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Fix passing props for ToolbarFiltersForm https://review.openstack.org/508472 | 09:41 |
| *** udesale has quit IRC | 09:42 | |
| *** akrivoka has joined #tripleo | 09:45 | |
| *** achadha has joined #tripleo | 09:48 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-docs master: Update documentation for O to P upgrade https://review.openstack.org/496223 | 09:51 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-docs master: Unify undercloud upgrade steps https://review.openstack.org/508475 | 09:51 |
| Tengu | shardy: ah, thanks for the pointer :). | 09:51 |
| Tengu | shardy: (and larsks ): basically, the fluentd configuration done in tripleo will only configure an agent forwarding all syslog content to some remote fluentd server, right? | 09:53 |
| openstackgerrit | Janki Chhatbar proposed openstack/tripleo-heat-templates master: Make config_volumne consistent with ODL container name https://review.openstack.org/508478 | 09:54 |
| Tengu | shardy: (and larsks ): if so, is there a way to configure syslog to directly forward all its log to a remote syslog server? We probably won't use fluentd as we already have an rsyslog "collector". If fluentd doesn't do anything more than forwarding rsyslog stuff, we can bypass it. | 09:55 |
| *** achadha has quit IRC | 09:55 | |
| *** achadha has joined #tripleo | 09:57 | |
| *** bogdando has quit IRC | 10:02 | |
| *** egonzalez has quit IRC | 10:05 | |
| *** achadha has quit IRC | 10:06 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-docs master: Update documentation for O to P upgrade and update https://review.openstack.org/496223 | 10:07 |
| marios | matbu_: jfrancoa updated ^^ | 10:08 |
| *** numans has quit IRC | 10:08 | |
| *** trozet has quit IRC | 10:08 | |
| *** myoung has quit IRC | 10:08 | |
| *** jschlueter has quit IRC | 10:08 | |
| marios | jfrancoa: can you check for your changes from v6 ? | 10:08 |
| *** anshul has quit IRC | 10:09 | |
| shardy | Tengu: probably best to confirm with larsks later but AFAIK the centralized logging is pretty specific to fluentd at this point - we set up groups and monitoring sources for each service so it's not just forwarding the syslog for each node | 10:09 |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 10:10 |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 10:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 10:10 |
| Tengu | shardy: hmmm ok. I think, if sources are set in fluentd, I should be able to override a bit its configuration using hiera in order to inject another output module and related configuration. | 10:10 |
| *** numans has joined #tripleo | 10:10 | |
| shardy | Tengu: yeah that may be possible, not tried it myself though | 10:11 |
| Tengu | shardy: I'm currently deploying the logging-environment.yaml in order to see what it does exactly. I think that would be a good start in order to check that. | 10:11 |
| jfrancoa | marios: sure, I'll have a look. thanks | 10:11 |
| *** jaganathan has quit IRC | 10:16 | |
| *** achadha has joined #tripleo | 10:19 | |
| *** egonzalez has joined #tripleo | 10:20 | |
| *** mcornea has quit IRC | 10:20 | |
| *** trozet has joined #tripleo | 10:21 | |
| *** mcornea has joined #tripleo | 10:22 | |
| *** derekh has joined #tripleo | 10:22 | |
| jaosorior | Tengu: let me know how that goes. I have to start looking into logging in the near future. | 10:23 |
| *** achadha has quit IRC | 10:24 | |
| derekh | anybody have a minute for a pike backport here looking for a review https://review.openstack.org/#/c/507981/ | 10:25 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates master: Also match config volumes for /var/lib/config-data/puppet-generated/ https://review.openstack.org/508450 | 10:25 |
| shardy | mandre, janki: ^^ tested that locally and I think it'll resolve the problem | 10:25 |
| shardy | derekh: lgtm but I'm not sure the gate is working yet due to zuul v3 breakages | 10:26 |
| *** jschlueter|znc has joined #tripleo | 10:26 | |
| derekh | shardy: ahh ok, thanks | 10:27 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-docs master: Unify undercloud upgrade steps https://review.openstack.org/508475 | 10:27 |
| Tengu | jaosorior: ah :). well, for now : I have a configuration snippet telling syslog to forward all its log to fluentd, but in fact, fluentd isn't installed. Like, not at all. I suspect the fact fluentd repository isn't installed to be the culprit. | 10:28 |
| Tengu | there are no fluentd package available, at least on centos7. | 10:28 |
| *** myoung has joined #tripleo | 10:29 | |
| shardy | Tengu: http://paste.openstack.org/show/622301/ looks like we enable an extra repo to get it | 10:30 |
| shardy | baseurl=http://mirror.centos.org/centos/7/opstools/x86_64/ | 10:30 |
| Tengu | hmm. | 10:30 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Introduce patternfly-react dependency https://review.openstack.org/505609 | 10:30 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Add missing less file added by latest patternfly version https://review.openstack.org/505610 | 10:30 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Use ListView from patternfly-react for NodesListView https://review.openstack.org/505611 | 10:31 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Use ListView from patternfly-react for NodeDrives list https://review.openstack.org/505612 | 10:31 |
| jaosorior | gfidente: you around? | 10:31 |
| Tengu | shardy: apparently, it's not installed on my nodes. weird. | 10:31 |
| Tengu | shardy: that mirror should be set with tripleO I think… ? | 10:31 |
| shardy | Tengu: strange, how did you build your overcloud-full image? | 10:31 |
| Tengu | shardy: I took it from a mirror. wait, have to find the link back | 10:32 |
| shardy | Tengu: well the name looks like quickstart configured it, possibly we're missing something from the manual non-quickstart docs | 10:32 |
| Tengu | shardy: I take them from https://images.rdoproject.org/ | 10:32 |
| Tengu | shardy: ah, that's possible indeed. I should add it then. Is there a way to do so in heat, or should I do some side-provisionning? | 10:33 |
| shardy | Tengu: Ok we have a bug in the CI pipeline I suspect if those packages are missing | 10:33 |
| Tengu | shardy: :Þ | 10:33 |
| *** shardy has quit IRC | 10:33 | |
| *** achadha has joined #tripleo | 10:34 | |
| *** achadha has quit IRC | 10:38 | |
| *** salmankhan has quit IRC | 10:39 | |
| *** achadha has joined #tripleo | 10:40 | |
| *** salmankhan has joined #tripleo | 10:40 | |
| Tengu | so I guess I'll open an issue. | 10:40 |
| *** jprovazn has joined #tripleo | 10:43 | |
| *** marios has quit IRC | 10:44 | |
| *** daidv has quit IRC | 10:45 | |
| *** abishop has joined #tripleo | 10:47 | |
| *** ffiore_ has quit IRC | 10:49 | |
| *** ffiore_ has joined #tripleo | 10:52 | |
| *** dpawar has quit IRC | 10:52 | |
| *** achadha has quit IRC | 10:52 | |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Introduce patternfly-react dependency https://review.openstack.org/505609 | 10:52 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Add missing less file added by latest patternfly version https://review.openstack.org/505610 | 10:52 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Use ListView from patternfly-react for NodesListView https://review.openstack.org/505611 | 10:52 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Use ListView from patternfly-react for NodeDrives list https://review.openstack.org/505612 | 10:53 |
| *** akrivoka has quit IRC | 10:53 | |
| *** akrivoka has joined #tripleo | 10:53 | |
| Tengu | I think I can work around this fluentd issue in an easy and swift way. | 10:55 |
| jaosorior | gfidente: Do you know how ceph-ansible configures logging in containerized environments? | 10:57 |
| *** marios has joined #tripleo | 11:00 | |
| Tengu | aha, found in quickstart where the repository is set: https://github.com/openstack/tripleo-quickstart/blob/a49b0a2785f9770c8c9cd025bbaa9de8efa5f128/config/release/tripleo-ci/consistent-pike.yml | 11:01 |
| Tengu | should be set available in tripleo directly I think. | 11:01 |
| *** achadha has joined #tripleo | 11:02 | |
| *** marios has quit IRC | 11:05 | |
| *** rhallisey has joined #tripleo | 11:05 | |
| *** ffiore_ has quit IRC | 11:06 | |
| gfidente | jaosorior yeah uses journal | 11:06 |
| *** pkovar has joined #tripleo | 11:07 | |
| *** marios has joined #tripleo | 11:07 | |
| jaosorior | gfidente: does it? | 11:07 |
| jaosorior | gfidente: can you show me where that's configured? | 11:07 |
| jaosorior | larsks: ^^ | 11:07 |
| jaosorior | gfidente: yesterday we were told that ceph exposed a fifo where the logs would be forwarded to a sidecar container :/ | 11:08 |
| *** ffiore_ has joined #tripleo | 11:08 | |
| gfidente | jaosorior ok not sure who told you that | 11:08 |
| gfidente | but regarding ceph-ansible I am pretty sure it disables logging in /var/log so that it goes into journal | 11:09 |
| gfidente | sec | 11:09 |
| gfidente | looking up the commit | 11:09 |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 11:10 |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 11:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 11:10 |
| *** achadha has quit IRC | 11:11 | |
| gfidente | jaosorior this is one piece https://github.com/ceph/ceph-ansible/commit/aa69c2c007f5fd81d597aa5d1dd0eddac8fa35a3 | 11:12 |
| *** iranzo has quit IRC | 11:12 | |
| jaosorior | gfidente: I was looking at the source code and found that. Some docs would be useful :D | 11:13 |
| jaosorior | didn't know what that was for. | 11:13 |
| *** cdearborn has joined #tripleo | 11:13 | |
| openstackgerrit | Jan Provaznik proposed openstack/tripleo-common master: Allow nfs-ganesha setup in ceph-ansible workbook https://review.openstack.org/505126 | 11:14 |
| *** iranzo has joined #tripleo | 11:14 | |
| *** iranzo has joined #tripleo | 11:14 | |
| *** marios has quit IRC | 11:14 | |
| jaosorior | gfidente: do you know how do the journald logs get exposed then? does the journald socket get exposed via a volume or something like that? | 11:14 |
| *** achadha has joined #tripleo | 11:16 | |
| gfidente | it's docker doing that | 11:16 |
| gfidente | https://docs.docker.com/engine/admin/logging/journald/ | 11:16 |
| dciabrin | mandre: hey quick question: it seems that all our containers that inherit from openstack-base have 'less' installed, whereas those that inherit from base directly don't. do you know where I could look at the official build flags to confirm/disprove that? | 11:18 |
| *** salmankhan has quit IRC | 11:21 | |
| *** fzdarsky_ is now known as fzdarsky|lunch | 11:21 | |
| honza | jrist: npm 5? that seems very new to me; mine is 3.10 | 11:23 |
| honza | jtomasek: do you know anything about these strange legacy ci jobs in tripleo-ui? | 11:26 |
| *** achadha has quit IRC | 11:26 | |
| jtomasek | honza: which ones? | 11:27 |
| Tengu | jaosorior: oh, btw, I just saw custodia was integrated in freeIPA with its latest release. I'll extend certodia in order to let it use the new Vault service in freeIPA, authenticating with kerberos and so on. I've discussed with my boss, he's OK I take some time on that "small" project both at work and in private + full disclosure to openstack if you're interested. I'll open a github project for that | 11:28 |
| honza | jtomasek: all of our current patches have these legacy-tripleo-* jobs | 11:28 |
| Tengu | shortly. | 11:28 |
| honza | jtomasek: I guess I'll go dig in project-infra | 11:28 |
| jtomasek | honza: I have no idea, maybe it is new? | 11:29 |
| jtomasek | new legacy:) | 11:29 |
| honza | jtomasek: ok, cool, i thought maybe you heard something in your travels | 11:29 |
| honza | jtomasek: they seem to be broken all the time | 11:29 |
| jaosorior | Tengu: nice! keep me posted. and if you need to talk or have questions for custodia or freeipa folks, let me know; I know who could help out :) | 11:30 |
| Tengu | jaosorior: cool, thanks! I think it should be OK, using python official ipa client lib. | 11:31 |
| Tengu | I'll let the option to NOT use freeipa Vault service but some random custodia hosted at some endpoint. that should allow a wide range of people to use that script. | 11:31 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Add uuid dependency https://review.openstack.org/508492 | 11:33 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Use uuid instrad of deprecated node-uuid https://review.openstack.org/508493 | 11:33 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Remove node-uuid dependency https://review.openstack.org/508494 | 11:33 |
| *** salmankhan has joined #tripleo | 11:33 | |
| florianf | honza, jtomasek: those probably have to do with yesterday's switch from zuul v2 to v3? | 11:33 |
| honza | florianf: ah! | 11:34 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Use uuid instead of deprecated node-uuid https://review.openstack.org/508493 | 11:34 |
| honza | florianf: that's the pertinent information I was after :) | 11:34 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Remove node-uuid dependency https://review.openstack.org/508494 | 11:34 |
| florianf | honza: :-) | 11:35 |
| *** achadha has joined #tripleo | 11:35 | |
| jtomasek | honza, florianf: does it mean we need to do some infra work to use correct jobs? | 11:36 |
| honza | jtomasek: florianf: I'm investigating that. | 11:36 |
| jtomasek | honza: thanks! | 11:36 |
| florianf | jtomasek, honza: I don't think so. That should've been done by some conversion scripts or something | 11:36 |
| honza | welp, zuul.d/projects.yaml is 50k+ LOC | 11:36 |
| florianf | jtomasek, honza: http://lists.openstack.org/pipermail/openstack-dev/2017-September/122834.html | 11:37 |
| honza | thanks | 11:37 |
| *** morazi has joined #tripleo | 11:38 | |
| *** achadha has quit IRC | 11:39 | |
| jtomasek | florianf: can you please update https://review.openstack.org/#/c/494513/ so we can merge it? It is getting lost on 3rd page in gerrit now:) | 11:41 |
| florianf | jtomasek: oh! | 11:41 |
| florianf | :-) | 11:41 |
| florianf | jtomasek: yeah, I will. Thanks! | 11:41 |
| sri_ | hey folks: is there any solution for taking instance snapshot if instance created form volume ? | 11:42 |
| *** jpena is now known as jpena|lunch | 11:43 | |
| jaosorior | gfidente: As far as I had understood, the docker logging drivers pass the container's stdout and stderr to the driver. So what the journald driver would do is pass the containers' stdout/stderr to journald; however, it does not read the container's journald entries. | 11:44 |
| jaosorior | (I just tried) | 11:44 |
| *** brian83 has joined #tripleo | 11:45 | |
| *** jprovazn has quit IRC | 11:47 | |
| gfidente | jaosorior yeah which I think is what ceph does when log file is none | 11:48 |
| jaosorior | aaah | 11:48 |
| *** ffiore_ has quit IRC | 11:48 | |
| jaosorior | gfidente: ok ok, then that does make sense | 11:48 |
| jaosorior | gfidente: thanks | 11:49 |
| *** achadha has joined #tripleo | 11:50 | |
| *** ansmith has quit IRC | 11:54 | |
| *** marios has joined #tripleo | 11:55 | |
| *** raildo has joined #tripleo | 11:55 | |
| *** dprince has joined #tripleo | 11:57 | |
| *** liverpooler has joined #tripleo | 11:58 | |
| *** achadha has quit IRC | 11:58 | |
| fultonj | gfidente: i was thinking that https://review.openstack.org/#/c/501987 could depends-on https://review.openstack.org/#/c/508079 but i guess we'll wait for zuul | 11:59 |
| *** achadha has joined #tripleo | 12:00 | |
| *** thrash|g0ne is now known as thrash | 12:01 | |
| jistr | weshay, matbu_ nice re the RDO upgrade jobs :) https://review.rdoproject.org/r/#/c/9823 | 12:08 |
| weshay | jistr, ya.. I need to talk to dmsimard to get it across projects | 12:09 |
| *** salmankhan has quit IRC | 12:09 | |
| weshay | hopefully he'll have time today | 12:09 |
| *** salmankhan has joined #tripleo | 12:09 | |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 12:10 |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 12:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 12:10 |
| jistr | weshay: just FYI currently we test non-cont.->cont. on both Pike and Queens. It's good for Pike, but for Queens it's not what we'll intend to ship. We'll likely need new featuresets and new jobs for cont.->cont. upgrades. | 12:11 |
| *** rbowen has quit IRC | 12:11 | |
| jistr | (once they are testable) | 12:11 |
| *** rbowen has joined #tripleo | 12:11 | |
| weshay | oh right.. we need cont -> cont for queens | 12:12 |
| jistr | yea | 12:12 |
| *** tzumainn has joined #tripleo | 12:12 | |
| *** achadha has quit IRC | 12:13 | |
| weshay | jistr, EmilienM I'm going to review upgrade requirements for ci today w/ Mike and matbu_ starting to take some notes here: https://etherpad.openstack.org/p/tripleo-ci-squad-meeting | 12:13 |
| *** rtailor has quit IRC | 12:13 | |
| weshay | line 25 53 | 12:13 |
| weshay | if you guys want or need something aka stack updates let's add it | 12:14 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates master: Create mysql user for non-ha deployments https://review.openstack.org/508504 | 12:15 |
| Tengu | weshay: hmmmm, don't know if this is the time to point to a possible CI issue? https://bugs.launchpad.net/tripleo/+bug/1720338 | 12:15 |
| openstack | Launchpad bug 1720338 in tripleo "Fluentd not installed when enabling "logging-environment.yaml"" [Undecided,New] | 12:15 |
| *** rbrady has joined #tripleo | 12:15 | |
| *** achadha has joined #tripleo | 12:18 | |
| jistr | marios, matbu_, chem: Pike minor updates are to be done using the `config download` way, right? What about major upgrades? Will O->P major upgrades be fully done via `config download` or are we going the "old" way of `overcloud deploy -e *upgrade.yaml`? I'm asking mainly becuause it's important for what we just discussed with weshay ^^. Trying to see if we'll have significantly differing upgrade workflows | 12:19 |
| jistr | between O->P and P->Q... | 12:19 |
| weshay | Tengu, seems like we have a doc bug eh | 12:19 |
| Tengu | weshay: ah? care to explain? :) | 12:19 |
| *** trown|outtypewww is now known as trown | 12:20 | |
| *** MVenesio has joined #tripleo | 12:20 | |
| weshay | Tengu, we'll work through the bug | 12:20 |
| weshay | thanks | 12:20 |
| Tengu | weshay: cool, thanks! | 12:20 |
| chem | jistr: well it's a quickly moving target it seems and I'm stuck with osp10 upgrade currently. But to my knowledge we don't currently do the download stuff for upgrade and we may have a mistral workflow for the update. matbu_ should know more | 12:22 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates master: Create mysql user for non-ha deployments https://review.openstack.org/508504 | 12:25 |
| *** achadha has quit IRC | 12:27 | |
| marios | jistr: O->P is the old way... sec there is a docs review | 12:27 |
| marios | weshay: matbu added some notes there yesterday too | 12:28 |
| marios | jistr: see https://review.openstack.org/#/c/496223/ i added the minor update stuff there this morning | 12:28 |
| *** rlandy has joined #tripleo | 12:28 | |
| marios | jistr: so minor update is done with mistral download execute, and there is new cli for it too. major upgrade is basically same cli with the right env file for th einitial stack update, followed by upgrade-non-controller | 12:29 |
| jistr | marios: right, thanks, actually i think i saw it earlier, just got confused a bit | 12:29 |
| marios | jistr: ack | 12:29 |
| marios | jistr: and the confusion may be that, for the non controllers, we *are* doing download/execute ansible. but still using upgrade-non-controller to invoke it | 12:30 |
| jistr | marios: and for P->Q we're going for full download/execute even for controllers? (That's the aim for both deployment and upgrade?) | 12:31 |
| marios | jistr: right i try to capture that here https://review.openstack.org/507620 | 12:31 |
| *** fzdarsky|lunch is now known as fzdarsky | 12:32 | |
| *** achadha has joined #tripleo | 12:32 | |
| jistr | marios: perfect, thanks | 12:36 |
| jistr | marios, weshay: added some notes on lines 55-59 https://etherpad.openstack.org/p/tripleo-ci-squad-meeting | 12:36 |
| *** achadha has quit IRC | 12:36 | |
| weshay | thank you | 12:37 |
| marios | thanks jistr | 12:37 |
| jistr | marios: what i wrote there about having both non-cont.->cont. and cont.->cont. upgrades alongside and autodetecting -- not sure if we'll end up wanting to do that, but just in case -- we could probably use ansible blocks there http://docs.ansible.com/ansible/latest/playbooks_blocks.html | 12:38 |
| weshay | anybody notice that the zuul page isn't loading | 12:38 |
| weshay | http://status.openstack.org/zuul/ | 12:38 |
| jistr | marios: detect whether the service is already containerized and pick the matching block. Not sure if that will be necessary for some services (Cinder/Neutron are weird in this regard, semi-containerized), or if we'd end up dropping the non-cont.->cont. upgrades eventually. Just an idea... | 12:39 |
| *** ebarrera has quit IRC | 12:39 | |
| *** shardy has joined #tripleo | 12:39 | |
| jistr | s/eventually/right away/ :) | 12:40 |
| marios | jistr: yeah... like cinder-volume (if i recall) which is non containerized is the reason we had to add the post_upgrade_steps. But, even for those cases, we just run the upgrade_tasks in the puppet/services so even though non containerized, those are still part of the 'containerized' flow | 12:40 |
| *** ansmith has joined #tripleo | 12:41 | |
| *** ebarrera has joined #tripleo | 12:42 | |
| *** achadha has joined #tripleo | 12:42 | |
| jistr | marios: right but what i mean is that even though one includes the ../docker/services/*.yaml version of a service -- that tells you that you're upgrading to containerized, but doesn't tell you if you're upgrading *from* containerized... For upgrades to Pike we always assume that we're upgrading from non-containerized, but Queens introduces this new dimension to the upgrade matrix. | 12:43 |
| dmsimard | weshay: zuulv3.openstack.org | 12:43 |
| dmsimard | weshay: I'm not sure they have updated the old one | 12:44 |
| marios | jistr: ah ok yes, but the upgrade_tasks don't do anything special in that regard (they stop systemd things, if there). i guess if we have any tasks that do things to the containers we would have to choose (for example) | 12:44 |
| *** pchavva has joined #tripleo | 12:44 | |
| *** ansmith_ has joined #tripleo | 12:44 | |
| marios | jistr: so unless we have to explicitly stop containers if we are doing container->container ? ah ok i see now what you mean though | 12:44 |
| marios | jistr: but still, we will have different upgrade tasks for P->Q but then those will be in master, compared to stable/pike for the non->container scenario | 12:45 |
| *** jpena|lunch is now known as jpena | 12:47 | |
| jistr | marios: right, that's what i guess will happen for majority of services. Basically saying "we only support cont->cont upgrades in P->Q", and tailoring the tasks for that. Still wondering if we'll have services where it will not be that straightforward though. | 12:48 |
| *** jmelvin has joined #tripleo | 12:48 | |
| *** lblanchard has joined #tripleo | 12:49 | |
| marios | jistr: yeah i think its a safe assumption. for example today the O->P *non* containerized path is not really tested. its like an implicit assumption that P==containers (where possible :) ) | 12:49 |
| jistr | yea :) | 12:50 |
| *** achadha has quit IRC | 12:50 | |
| *** ecerquei has quit IRC | 12:50 | |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-common master: Implement upgrade workflow in mistral https://review.openstack.org/463765 | 12:50 |
| *** bnemec has joined #tripleo | 12:50 | |
| *** achadha has joined #tripleo | 12:52 | |
| *** ffiore_ has joined #tripleo | 12:54 | |
| *** rbowen has quit IRC | 12:55 | |
| *** MVenesio has quit IRC | 12:55 | |
| *** rbowen has joined #tripleo | 12:55 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/newton: Keep floating ip reachability during pacemaker migration. https://review.openstack.org/474967 | 12:56 |
| *** jprovazn has joined #tripleo | 13:02 | |
| *** catintheroof has joined #tripleo | 13:03 | |
| *** jcoufal has joined #tripleo | 13:04 | |
| *** achadha has quit IRC | 13:05 | |
| *** leitan has joined #tripleo | 13:06 | |
| *** Goneri has joined #tripleo | 13:09 | |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 13:10 |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 13:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 13:10 |
| *** abregman|afk is now known as abregman | 13:11 | |
| *** paramite has quit IRC | 13:15 | |
| *** achadha has joined #tripleo | 13:15 | |
| *** rbowen has quit IRC | 13:16 | |
| *** ansmith has quit IRC | 13:20 | |
| *** aputtur has joined #tripleo | 13:21 | |
| *** ansmith has joined #tripleo | 13:21 | |
| larsks | Tengu: just catching up on the earlier conversation. There is currently no facility for configuring syslog (one would need another tripleo composable service to support that). We're working on changes for queens that will make it easier to pass in arbitrary configuration to puppet-fluentd, which should make it easier to configure outputs and such. The configuration in pike is less flexible. | 13:24 |
| *** mrch has quit IRC | 13:24 | |
| *** noslzzp has joined #tripleo | 13:26 | |
| *** rbowen has joined #tripleo | 13:27 | |
| *** achadha has quit IRC | 13:28 | |
| *** achadha has joined #tripleo | 13:30 | |
| Tengu | larsks: hmmm ok. I think we can do a lot through ExtracConfig and hiera stuff in both syslog and fluentd. | 13:30 |
| Tengu | larsks: the main question I get: what's the use of fluentd, is it just a forwarder, or does it actively collect logs in /var/log/* and add them to the syslog input it also has? | 13:31 |
| larsks | Tengu: it collects logs (from files and from syslog) and forwards them to a remote fluentd instance. | 13:31 |
| larsks | (or directly to elastic, etc) | 13:31 |
| larsks | Although our service integration right now assumes the "forward to fluentd" configuration. | 13:32 |
| Tengu | larsks: hmm ok. as we're putting our own puppet in place in order to manage some specific configuration that can't be set through tripleO, is there any way to get fluentd configuration in order to "mimic" it with rsyslog? | 13:32 |
| Tengu | larsks: in fact, we don't really need fluentd in our setup, unless it does some magics with the log inputs before sending them away. | 13:33 |
| larsks | Tengu: it doesn't do any magics right now. | 13:33 |
| larsks | The assumption is that any metadata extraction/log transofmration will happen on the remote end. | 13:33 |
| Tengu | larsks: hmm ok. so until Quieens at least we're pretty safe configuring some stuff in syslog? | 13:33 |
| Tengu | perfect | 13:33 |
| Tengu | that's what we want anyway: move the extraction workload away | 13:34 |
| larsks | Well, even in queens you don't need to enable the fluentd service if you don't want it. | 13:34 |
| Tengu | and that's the way to do things ;) | 13:34 |
| Tengu | larsks: good :). | 13:34 |
| *** dansmith is now known as superdan | 13:34 | |
| openstackgerrit | Janki Chhatbar proposed openstack/tripleo-heat-templates master: Make config_volumne consistent with ODL container name https://review.openstack.org/508478 | 13:35 |
| Tengu | larsks: so, any way to get fluentd input sources ? I didn't find that in heat nor puppet - although for the latter I didn't spend hours searching | 13:35 |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-ui master: Make validation groups labels clickable https://review.openstack.org/494513 | 13:36 |
| *** psachin has quit IRC | 13:36 | |
| EmilienM | hi | 13:36 |
| EmilienM | weshay: ack | 13:37 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates master: Reintroduce containerized Cinder https://review.openstack.org/479001 | 13:38 |
| *** links has quit IRC | 13:39 | |
| larsks | Tengu: you could probably get at it from a heat template. | 13:40 |
| shardy | Tengu: try sudo hiera -c /etc/puppet/hiera.yaml tripleo::profile::base::logging::fluentd::fluentd_sources | 13:40 |
| jaosorior | larsks: do you see any value in the httpd access log file? Seems to me that it's repeated info with what ends up in the actual service lgos | 13:40 |
| larsks | You can see where the fluentd service builds a hiera configuration in https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/logging/fluentd-client.yaml#L53 | 13:40 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/role.role.j2.yaml#L594 | 13:41 |
| larsks | jaosorior: don't know, I would need to take a look at the httpd access log on a live system. I wouldn't expect it to have repeated info (I would expect that services would log only to service-specific files), but I guess I could be surprised. | 13:42 |
| *** achadha has quit IRC | 13:42 | |
| *** abregman is now known as abregman|afk | 13:43 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-common master: Fixup upgrade-non-controller to also accept IP address https://review.openstack.org/505624 | 13:46 |
| *** ykarel has quit IRC | 13:47 | |
| *** pdeore has quit IRC | 13:47 | |
| *** morazi has quit IRC | 13:47 | |
| *** gbarros has joined #tripleo | 13:47 | |
| *** toure_biab is now known as toure | 13:48 | |
| *** Dinesh_Bhor has quit IRC | 13:48 | |
| *** morazi has joined #tripleo | 13:48 | |
| *** achadha has joined #tripleo | 13:49 | |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart master: Update README to make add_repos use clearer https://review.openstack.org/501851 | 13:51 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart-extras master: Update validate-tempest role docs https://review.openstack.org/498836 | 13:51 |
| openstackgerrit | mathieu bultel proposed openstack/python-tripleoclient master: Implement Upgrade workflow via the tripleo cli https://review.openstack.org/463728 | 13:54 |
| *** achadha has quit IRC | 13:57 | |
| *** limao has quit IRC | 13:58 | |
| *** jcoufal_ has joined #tripleo | 13:58 | |
| *** limao has joined #tripleo | 13:59 | |
| Tengu | will try that, shardy :) | 14:00 |
| openstackgerrit | Ryan Brady proposed openstack/tripleo-common master: [WIP] Enhances Roles List https://review.openstack.org/508514 | 14:00 |
| *** jcoufal has quit IRC | 14:00 | |
| *** abregman|afk is now known as abregman | 14:01 | |
| Tengu | shardy: perfect! thanks :) | 14:01 |
| *** limao has quit IRC | 14:03 | |
| Tengu | larsks, shardy : thank to both of you :). I think I have now the requested information for a better logging integration in our infra. | 14:03 |
| *** achadha has joined #tripleo | 14:03 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates master: WIP: Add option for containers to log to stdout/stderr https://review.openstack.org/508517 | 14:05 |
| *** amoralej is now known as amoralej|off | 14:06 | |
| *** amoralej|off is now known as amoralej|lunch | 14:06 | |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 14:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 14:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: legacy-tripleo jobs failing on gate_hook: not found" [Critical,Triaged] | 14:10 |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-quickstart-extras master: Add check on download_overcloud_templates_rpm to trigger custom-tht https://review.openstack.org/479212 | 14:10 |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-quickstart master: Remove hardcoded overcloud_release value https://review.openstack.org/480461 | 14:10 |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-common master: Make validation inputs configurable via Mistral https://review.openstack.org/383763 | 14:13 |
| gchamoul | florianf: ^^^ | 14:13 |
| *** ramishra has quit IRC | 14:13 | |
| gchamoul | florianf: wait, I screwed up the sudoers file on the rebase! | 14:15 |
| *** aditya_r has joined #tripleo | 14:15 | |
| *** achadha has quit IRC | 14:16 | |
| *** jcoufal_ has quit IRC | 14:17 | |
| *** jcoufal has joined #tripleo | 14:18 | |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-quickstart-extras master: Use upgrade cli command instead https://review.openstack.org/483850 | 14:18 |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-quickstart-extras master: Use upgrade cli command instead https://review.openstack.org/483850 | 14:19 |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-quickstart-extras master: Use upgrade cli command instead https://review.openstack.org/483850 | 14:20 |
| *** aditya_ra has joined #tripleo | 14:20 | |
| *** achadha has joined #tripleo | 14:21 | |
| *** aditya_ra has quit IRC | 14:24 | |
| *** aditya_r has quit IRC | 14:24 | |
| *** achadha has quit IRC | 14:25 | |
| EmilienM | bnemec: it sounds like jobs don't appear anymore on http://tripleo.org/cistatus.html | 14:25 |
| EmilienM | bnemec: I meant new jobs from today, since zuul v3 migration | 14:25 |
| EmilienM | dprince: ^ | 14:26 |
| *** abregman is now known as abregman|afk | 14:26 | |
| bnemec | The format of the Jenkins messages probably changed. | 14:27 |
| bnemec | I'll look into it after my 1-1. | 14:28 |
| florianf | gchamoul: the patch with the sudoers fix hasn't been merged yet, so the rebase is probably correct. | 14:28 |
| bnemec | Yeah, different user and different job names. | 14:29 |
| dprince | EmilienM: looking | 14:30 |
| *** achadha has joined #tripleo | 14:30 | |
| weshay | matbu_, mtg in my bluejeans | 14:31 |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-common master: Make validation inputs configurable via Mistral https://review.openstack.org/383763 | 14:31 |
| EmilienM | bnemec: ouch | 14:32 |
| dprince | EmilienM: has to be related to job names or something | 14:33 |
| dprince | EmilienM: here http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/tripleo-jobs-gerrit.py#n13 | 14:33 |
| jaosorior | have a good weekend folks! | 14:33 |
| dprince | EmilienM: our script to generate the report only uses gerrit | 14:33 |
| EmilienM | yeah | 14:33 |
| *** jaosorior has quit IRC | 14:33 | |
| *** achadha has quit IRC | 14:34 | |
| EmilienM | bnemec: jobnames? they are the same afict | 14:35 |
| *** psahoo has quit IRC | 14:39 | |
| *** jcoufal_ has joined #tripleo | 14:40 | |
| *** jcoufal has quit IRC | 14:42 | |
| *** abregman|afk is now known as abregman | 14:44 | |
| *** achadha has joined #tripleo | 14:45 | |
| lvdombrkr | shardy: i double check network config and nothing find... can i paste compute.yaml and controller.yaml for you, can you look into? as i mentioned there is no problems when i deploy controller and compute in one node, but problem become when there is several nodes | 14:47 |
| *** shardy is now known as shardy_mtg | 14:49 | |
| lvdombrkr | shardy: maybe i can try deploy compute node with controller.yaml? than i will understood if problem cause compute.yaml | 14:49 |
| *** amoralej|lunch is now known as amoralej | 14:49 | |
| EmilienM | dprince: trying to run it manually | 14:50 |
| dmsimard | mwhahaha, EmilienM: made some progress on unblocking gate | 14:51 |
| dmsimard | new error is http://logs.openstack.org/89/507889/2/check/legacy-tripleo-ci-centos-7-scenario003-multinode-oooq-puppet/603ef1f/logs/devstack-gate-setup-workspace-new.txt | 14:52 |
| dmsimard | looking. | 14:52 |
| *** achadha has quit IRC | 14:54 | |
| *** ykarel has joined #tripleo | 14:57 | |
| *** abregman is now known as abregman|afk | 14:57 | |
| *** rcernin has quit IRC | 15:01 | |
| *** ykarel has quit IRC | 15:02 | |
| *** egonzalez has quit IRC | 15:02 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates master: Remove package if service stopped and disabled https://review.openstack.org/479886 | 15:05 |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-ui master: Prevent filter duplicates https://review.openstack.org/508536 | 15:06 |
| bnemec | EmilienM: The patch I looked at had them all named legacy-something, which I think would break the status page. | 15:07 |
| *** achadha has joined #tripleo | 15:08 | |
| *** ramishra has joined #tripleo | 15:10 | |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 15:10 |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 15:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: Most/all legacy-tripleo jobs failing on gate" [Critical,Triaged] | 15:10 |
| *** achadha has quit IRC | 15:12 | |
| *** myoung is now known as myoung|bbl | 15:15 | |
| *** ffiore_ has quit IRC | 15:15 | |
| *** ykarel has joined #tripleo | 15:16 | |
| *** ramishra has quit IRC | 15:17 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates master: Remove package if service stopped and disabled https://review.openstack.org/479886 | 15:17 |
| *** achadha has joined #tripleo | 15:18 | |
| *** v1k0d3n has quit IRC | 15:19 | |
| *** v1k0d3n has joined #tripleo | 15:20 | |
| *** iranzo has quit IRC | 15:20 | |
| *** trown is now known as trown|brb | 15:20 | |
| dmsimard | this sucks ... | 15:23 |
| dmsimard | EmilienM: is it safe to assume that all tripleo jobs require the same set of projects (i.e, for zuul cloning) | 15:23 |
| *** iranzo has joined #tripleo | 15:26 | |
| *** achadha has quit IRC | 15:26 | |
| *** trown|brb is now known as trown | 15:27 | |
| jrist | honza: turns out that cheerio isn't compatible with npm 5.4.2 or node 6.0.0 | 15:30 |
| jrist | cheerio.js | 15:30 |
| jrist | aka jquery for servers | 15:30 |
| jrist | which is a dep of something | 15:30 |
| jrist | haven't had a chance to figure out what yet | 15:31 |
| *** ffiore_ has joined #tripleo | 15:31 | |
| *** d0ugal has joined #tripleo | 15:32 | |
| *** chlong has quit IRC | 15:35 | |
| *** iranzo has quit IRC | 15:36 | |
| *** shardy_mtg is now known as shardy | 15:38 | |
| *** iranzo has joined #tripleo | 15:38 | |
| *** achadha has joined #tripleo | 15:42 | |
| *** iranzo has quit IRC | 15:42 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Switch cistatus page to zuul v3 https://review.openstack.org/508546 | 15:43 |
| *** janki has quit IRC | 15:43 | |
| bnemec | EmilienM: dprince: ^ | 15:43 |
| dmsimard | About halfway done sending a patch to fix the current gate failures | 15:43 |
| bnemec | That's working for me locally. | 15:43 |
| *** dtrainor has joined #tripleo | 15:44 | |
| dprince | bnemec: thanks +2 from me | 15:44 |
| *** jlinkes has quit IRC | 15:44 | |
| EmilienM | bnemec: thanks | 15:47 |
| bnemec | Heh, of course that probably can't merge while the jobs are all broken. | 15:48 |
| dmsimard | EmilienM, mwhahaha, bnemec: next fixes for broken gates are here: https://review.openstack.org/#/c/508548/ | 15:49 |
| dmsimard | that patch as well as the two parent patches | 15:49 |
| dmsimard | I'll do a recheck once the stack has landed | 15:49 |
| *** achadha has quit IRC | 15:51 | |
| *** derekh has quit IRC | 15:52 | |
| *** ebarrera has quit IRC | 15:57 | |
| *** zzzeek has quit IRC | 15:58 | |
| honza | jrist: whaaat, good for you for narrowing it down | 15:59 |
| *** zzzeek has joined #tripleo | 15:59 | |
| *** idlemind has joined #tripleo | 16:02 | |
| *** achadha has joined #tripleo | 16:02 | |
| *** achadha has quit IRC | 16:02 | |
| *** idlemind has quit IRC | 16:02 | |
| *** achadha has joined #tripleo | 16:02 | |
| *** idlemind has joined #tripleo | 16:02 | |
| *** achadha has quit IRC | 16:03 | |
| *** achadha has joined #tripleo | 16:03 | |
| hrybacki | trown: I reprovisioned my dev box to make sure there was nothing weird happening and was able to reproduce the same no valid hosts error again... I'm at a loss | 16:05 |
| lvdombrkr | shardy: if i map controller.yaml to compute node and controller node its works well | 16:06 |
| lvdombrkr | so issue caused by myself made compute.yaml | 16:06 |
| hrybacki | trown: attempting to do a vanilla deployment on that box to confirm that works | 16:07 |
| shardy | lvdombrkr: good to hear that you found the problem | 16:08 |
| *** owalsh_ has joined #tripleo | 16:09 | |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 16:10 |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 16:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: Most/all legacy-tripleo jobs failing on gate" [Critical,Triaged] | 16:10 |
| lvdombrkr | shardy: only diference between two templates is that controller.yaml have default route external network but comptute.yaml ControlPlane network | 16:10 |
| *** owalsh has quit IRC | 16:11 | |
| *** owalsh_ has quit IRC | 16:14 | |
| *** owalsh has joined #tripleo | 16:19 | |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-docs master: Remove source admonition https://review.openstack.org/508323 | 16:19 |
| openstackgerrit | Ian Main proposed openstack/python-tripleoclient master: WIP: Mount a tmpfs filesystem for heat tmpfiles https://review.openstack.org/508558 | 16:19 |
| *** mcornea has quit IRC | 16:20 | |
| *** itlinux has joined #tripleo | 16:23 | |
| *** achadha has quit IRC | 16:24 | |
| *** pcaruana has quit IRC | 16:26 | |
| *** gbarros has quit IRC | 16:29 | |
| *** trown is now known as trown|lunch | 16:30 | |
| radez | hrybacki: you have a sec, come look at this | 16:31 |
| radez | ? | 16:31 |
| jrist | honza: yeahhhhhhhhhhhh gonna have to downgrade npm/node for now though | 16:32 |
| honza | jrist: I can't figure out how you ended up with those two versions together. | 16:32 |
| *** panda is now known as panda|bbl | 16:36 | |
| *** jcoufal has joined #tripleo | 16:40 | |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common master: Validate roles data and network data https://review.openstack.org/508567 | 16:42 |
| jrist | honza: mac | 16:42 |
| jrist | :) | 16:42 |
| *** jcoufal_ has quit IRC | 16:42 | |
| thrash | jrist: jtomasek ^^^ ' | 16:43 |
| jrist | thrash: awwwwww yeah | 16:44 |
| jrist | thrash: progress++ | 16:44 |
| *** itlinux has quit IRC | 16:44 | |
| *** itlinux has joined #tripleo | 16:47 | |
| *** jcoufal_ has joined #tripleo | 16:50 | |
| *** marios has quit IRC | 16:51 | |
| *** suuuper has quit IRC | 16:52 | |
| *** jcoufal has quit IRC | 16:53 | |
| *** etingof has joined #tripleo | 16:53 | |
| openstackgerrit | Andy Smith proposed openstack/puppet-tripleo master: WIP: Updates to separate messaging backends https://review.openstack.org/508007 | 16:57 |
| dmsimard | EmilienM, mwhahaha, bnemec: still waiting on the 3 patch stack to land, zuul and nodepool are pretty backed up right now | 16:58 |
| *** salmankhan has quit IRC | 16:59 | |
| *** salmankhan has joined #tripleo | 17:00 | |
| *** ykarel has quit IRC | 17:02 | |
| *** tongl has joined #tripleo | 17:02 | |
| *** itlinux has quit IRC | 17:02 | |
| *** jfrancoa has quit IRC | 17:03 | |
| *** athomas has quit IRC | 17:06 | |
| *** achadha has joined #tripleo | 17:08 | |
| *** achadha_ has joined #tripleo | 17:10 | |
| *** achadha has quit IRC | 17:10 | |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 17:10 |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 17:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: Most/all legacy-tripleo jobs failing on gate" [Critical,Triaged] | 17:10 |
| *** ffiore_ has quit IRC | 17:10 | |
| *** salmankhan has quit IRC | 17:11 | |
| *** salmankhan has joined #tripleo | 17:12 | |
| *** achadha has joined #tripleo | 17:12 | |
| *** achadha_ has quit IRC | 17:12 | |
| *** achadha has quit IRC | 17:14 | |
| *** achadha has joined #tripleo | 17:14 | |
| *** achadha has quit IRC | 17:15 | |
| *** salmankhan has quit IRC | 17:16 | |
| *** achadha has joined #tripleo | 17:16 | |
| *** jpena is now known as jpena|off | 17:16 | |
| *** ykarel has joined #tripleo | 17:17 | |
| openstackgerrit | Andy Smith proposed openstack/tripleo-heat-templates master: WIP Separate rpc and notify messaging backends https://review.openstack.org/507963 | 17:19 |
| *** sri_ has quit IRC | 17:21 | |
| *** thrash is now known as thrash|biab | 17:26 | |
| *** tosky has quit IRC | 17:29 | |
| openstackgerrit | Lee Yarwood proposed openstack/tripleo-specs master: WIP - Introduce fast-forward upgrades https://review.openstack.org/497257 | 17:30 |
| *** florianf has quit IRC | 17:36 | |
| jtomasek | thrash|biab: thanks! | 17:37 |
| openstackgerrit | Merged openstack/tripleo-quickstart master: fix lp bug url and comments in contributing doc https://review.openstack.org/501854 | 17:41 |
| *** rdopiera has quit IRC | 17:41 | |
| *** rdopiera has joined #tripleo | 17:41 | |
| *** raildo has quit IRC | 17:45 | |
| *** trown|lunch is now known as trown | 17:49 | |
| *** raildo has joined #tripleo | 17:51 | |
| openstackgerrit | Dan Sneddon proposed openstack/tripleo-heat-templates master: Remove External network from Networker role NIC configs https://review.openstack.org/508360 | 17:51 |
| *** tosky has joined #tripleo | 17:51 | |
| *** ykarel has quit IRC | 17:52 | |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-pacemaker master: Fix constraint::order when constraint_params is undef https://review.openstack.org/508578 | 17:53 |
| *** thrash|biab is now known as thrash | 17:53 | |
| *** ecerquei has joined #tripleo | 17:55 | |
| *** gfidente is now known as gfidente|afk | 17:55 | |
| *** jpich has quit IRC | 17:58 | |
| honza | jtomasek: so, there was a bug in the new zuul configuration https://review.openstack.org/#/c/508548/2 --- just FYI | 17:58 |
| dmsimard | EmilienM, mwhahaha, bnemec: just fired a recheck on the patch we've been using for testing with https://review.openstack.org/#/c/507889/2 | 17:59 |
| dmsimard | honza, jtomasek yes, I've been working on this for a while now | 17:59 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-pacemaker master: Add autogenerated fence-amt manifest https://review.openstack.org/508580 | 17:59 |
| honza | dmsimard: thank you, sir | 17:59 |
| dmsimard | honza: it's more than just one bug :) | 17:59 |
| EmilienM | dmsimard: ack, thx | 18:00 |
| dmsimard | going to eat now, bbl | 18:00 |
| honza | dmsimard: i was just talking to mordred about this in openstack-infra | 18:00 |
| honza | i found out about it roundaboutly :) | 18:00 |
| dmsimard | honza: yeah please coordinate with me if you're interested in helping unbreaking the gate -- I've been keeping https://bugs.launchpad.net/tripleo/+bug/1720220 fairly up to date | 18:00 |
| openstack | Launchpad bug 1720220 in tripleo "CI: Most/all legacy-tripleo jobs failing on gate" [Critical,Triaged] | 18:00 |
| honza | dmsimard: now that the above is merged, i'm trying a few rechecks on ui patches | 18:01 |
| dmsimard | honza: no need to do a few rechecks | 18:01 |
| honza | no? | 18:01 |
| dmsimard | honza: it's wasteful until we know things are working for sure | 18:01 |
| dmsimard | the gate is already super busy | 18:01 |
| honza | dmsimard: ugh, sorry :( | 18:01 |
| dmsimard | honza: I already did a recheck on 507889 | 18:02 |
| dmsimard | afk for a while during the recheck and stuff... food | 18:02 |
| honza | dmsimard: I'll keep an eye on the bug then, and will try to fight off any temptations to recheck... | 18:02 |
| slagle | weshay: what's going on with rdo-build-kolla-integration? should we be ignoring it? | 18:04 |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 18:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 18:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: Most/all legacy-tripleo jobs failing on gate" [Critical,Triaged] | 18:10 |
| *** ansmith has quit IRC | 18:10 | |
| weshay | slagle, what are you looking at? | 18:25 |
| weshay | slagle, that's being used in https://review.rdoproject.org/zuul/ | 18:25 |
| slagle | weshay: https://review.openstack.org/#/c/505624/ is one example | 18:25 |
| weshay | check the 4hr pipeline | 18:25 |
| slagle | weshay: it's reporting failed on our patches | 18:26 |
| weshay | hrm.. not sure who set that up to check tripleo-common | 18:26 |
| weshay | dmsimard, would know what's up with that I suspect | 18:27 |
| * weshay looks that the project config for it | 18:27 | |
| slagle | mostly just wondering if it's supposed to be red | 18:27 |
| slagle | if so, i can be easily trained to ignore 3rd party CI :) | 18:27 |
| *** pkovar has quit IRC | 18:28 | |
| weshay | slagle, ya.. broken glass | 18:31 |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common master: Validate roles data and network data https://review.openstack.org/508567 | 18:38 |
| dmsimard | weshay: http://lists.openstack.org/pipermail/openstack-dev/2017-August/120433.html | 18:39 |
| * dmsimard back to food | 18:40 | |
| weshay | dmsimard, thanks | 18:40 |
| weshay | slagle, ^ | 18:40 |
| weshay | it should have a -nv at the end | 18:40 |
| *** tongl has quit IRC | 18:42 | |
| slagle | dmsimard: yes. i recall the email from 2 months ago. | 18:43 |
| slagle | dmsimard: doesnt really do much towards explaining why the job is red today | 18:43 |
| slagle | and if it's anything anyone should be caring about | 18:44 |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-ui master: Remove oslosphinx https://review.openstack.org/508588 | 18:47 |
| honza | dmsimard: wasteful indeed, both came back as failures :( | 19:00 |
| *** tongl has joined #tripleo | 19:05 | |
| openstackgerrit | Ben Nemec proposed openstack/instack-undercloud master: Generate a standalone stackrc https://review.openstack.org/508593 | 19:06 |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 19:10 |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 19:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: Most/all legacy-tripleo jobs failing on gate" [Critical,Triaged] | 19:10 |
| *** catintheroof has quit IRC | 19:20 | |
| *** catintheroof has joined #tripleo | 19:20 | |
| *** fzdarsky is now known as fzdarsky|afk | 19:24 | |
| *** catintheroof has quit IRC | 19:24 | |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-pacemaker master: Allow empty pcmk_host_list https://review.openstack.org/506914 | 19:26 |
| EmilienM | dprince: https://review.openstack.org/#/c/508306/2/roles/overcloud-deploy/tasks/deploy-overcloud.yml | 19:29 |
| *** aputtur has quit IRC | 19:34 | |
| *** jcoufal has joined #tripleo | 19:39 | |
| *** jcoufal_ has quit IRC | 19:41 | |
| *** jtomasek has quit IRC | 19:42 | |
| bugzy | Hello. I have a quick question related to using custom network templates. | 19:48 |
| bugzy | Do the nic naming convention and sequence in the template need to match the OS naming convention and sequence. | 19:48 |
| bugzy | For instance, I use the nic1, nic2, ...nicn in the template but the OS (centos7) has names like enp3s0f0, enp3s0f1, enp4s0f0, enp4s0f1. Should I be canging the template names to match what the OS sees? | 19:48 |
| bugzy | Note: I am deploying on baremetal if that makes any difference. | 19:49 |
| *** etingof has quit IRC | 19:49 | |
| bugzy | Note2: using bnemec's tool at https://github.com/cybertron/tripleo-scripts to generate the templates | 19:51 |
| bnemec | bugzy: You can do either. | 19:54 |
| bnemec | If you know which nics you want and they're the same across all nodes in a role you can use the actual OS names. | 19:54 |
| bnemec | If there's variation across nodes then you may have to use the nicX abstraction in os-net-config to handle that. | 19:55 |
| bnemec | The tricky thing about the abstraction is figuring out the ordering ahead of time. | 19:55 |
| dsneddon | bugzy, Only live interfaces will be counted. If you have cables plugged into em1 and em4, those will be nic1 and nic2 respectively. | 19:56 |
| dsneddon | bugzy, The ordering is: embedded interfaces first (em*, en*, eth*) alphanumerically, followed by non-embedded interfaces (p2p1, ensp3s21, etc.) alphanumerically. | 19:57 |
| bnemec | There's also a way for os-net-config to dump out the mapping for you if you pre-install one of the machines: https://github.com/openstack/os-net-config/blob/master/os_net_config/cli.py#L45 | 19:57 |
| dsneddon | bugzy, This works out the same between substantially similar hardware (like different series from the same manufacturer), but might go awry if you are mixing hardware from different vendors. | 19:57 |
| openstackgerrit | Andy Smith proposed openstack/tripleo-heat-templates master: WIP Separate rpc and notify messaging backends https://review.openstack.org/507963 | 19:57 |
| bnemec | (at least I think that's what it does) | 19:57 |
| dsneddon | bugzy, So there is a mapping feature (mapping MACs to NICs), but it's insanely hard to use. | 19:58 |
| bnemec | heh | 19:58 |
| bugzy | I see. This is very helpful. I will definitly have to redo my templates then. Thanks bnemec and dsneddon | 19:59 |
| dsneddon | bugzy, The other thing people do if they are stuck using different manufacturers between nodes is to disable the ill-named "predictable NIC naming" so everything is ethX | 20:00 |
| bugzy | dsneddon: Wouldn't that mean that I will need to modify the deployment images? | 20:02 |
| *** idlemind has quit IRC | 20:02 | |
| bugzy | since I acctually dont't have any realy access to the baremetal nodes till they have been deployed | 20:02 |
| bugzy | dont actually* | 20:03 |
| dsneddon | bugzy, Yes. Would you like to see the procedure? I can post it somewhere. | 20:03 |
| bugzy | real* | 20:03 |
| bugzy | yes that will be excellt | 20:03 |
| dsneddon | bugzy, I usually use virt-customize. | 20:03 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo master: WIP DNR Instance HA POC https://review.openstack.org/506922 | 20:08 |
| dsneddon | bugzy, https://paste.fedoraproject.org/paste/bdu3MhYF01WvUcYRA-~A1w | 20:09 |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 20:10 |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 20:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: Most/all legacy-tripleo jobs failing on gate" [Critical,Triaged] | 20:10 |
| bugzy | dsneddon: much thanks | 20:10 |
| dsneddon | bugzy, Actually, I think the use of sudo in those commands is superfluous, and it won't work if sudo isn't installed on the target image. | 20:11 |
| *** myoung|bbl is now known as myoung | 20:11 | |
| bugzy | dsneddon: got it. Thanks | 20:13 |
| dsneddon | bugzy, I just copied and pasted from three different sources, but they all do the same thing. | 20:13 |
| bugzy | yes, I was looking at that and the carefully placed "OR"s helped | 20:14 |
| * bugzy has to run to airport | 20:14 | |
| bugzy | thanks again | 20:14 |
| bugzy | I'll report back either way after I have the opportunity to work on it | 20:15 |
| *** Goneri has quit IRC | 20:17 | |
| *** jcoufal has quit IRC | 20:17 | |
| *** jprovazn has quit IRC | 20:21 | |
| *** jtomasek has joined #tripleo | 20:31 | |
| *** tongl has quit IRC | 20:31 | |
| *** ansmith_ has quit IRC | 20:32 | |
| *** jtomasek has quit IRC | 20:42 | |
| *** rhallisey has quit IRC | 20:42 | |
| *** rlandy has quit IRC | 20:44 | |
| *** trown is now known as trown|outtypewww | 20:47 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Config download support for standalone deployments https://review.openstack.org/505827 | 20:48 |
| openstackgerrit | Damien Ciabrini proposed openstack/tripleo-heat-templates master: mysql: expose IPv6 configuration to mysql puppet modules https://review.openstack.org/508614 | 20:49 |
| openstackgerrit | James Slagle proposed openstack/tripleo-common master: Config download support for all deployments https://review.openstack.org/508189 | 20:50 |
| *** bnemec has quit IRC | 20:51 | |
| *** jtomasek has joined #tripleo | 20:51 | |
| openstackgerrit | Damien Ciabrini proposed openstack/puppet-tripleo master: WIP Galera: add support for encrypted SST https://review.openstack.org/507852 | 20:53 |
| *** fultonj has quit IRC | 21:02 | |
| *** tongl has joined #tripleo | 21:07 | |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 21:10 |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 21:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: Most/all legacy-tripleo jobs failing on gate" [Critical,Triaged] | 21:10 |
| *** morazi has quit IRC | 21:11 | |
| *** jmelvin has quit IRC | 21:15 | |
| *** raildo has quit IRC | 21:17 | |
| openstackgerrit | Ian Main proposed openstack/python-tripleoclient master: Add undercloud hostname. https://review.openstack.org/508618 | 21:18 |
| *** jpena|off has quit IRC | 21:21 | |
| *** dprince has quit IRC | 21:21 | |
| *** amoralej has quit IRC | 21:22 | |
| *** eck` has quit IRC | 21:24 | |
| *** dalvarez has quit IRC | 21:25 | |
| *** radez has quit IRC | 21:25 | |
| *** radez has joined #tripleo | 21:28 | |
| *** dalvarez has joined #tripleo | 21:28 | |
| *** eck` has joined #tripleo | 21:28 | |
| *** jpena|off has joined #tripleo | 21:29 | |
| *** amoralej has joined #tripleo | 21:29 | |
| *** eck` is now known as eck`gone | 21:38 | |
| *** cdearborn has quit IRC | 21:39 | |
| *** ansmith_ has joined #tripleo | 21:41 | |
| *** etingof has joined #tripleo | 21:44 | |
| *** etingof has quit IRC | 21:49 | |
| *** pchavva has quit IRC | 21:54 | |
| *** dspano has joined #tripleo | 21:55 | |
| *** thrash is now known as thrash|g0ne | 22:00 | |
| *** morazi has joined #tripleo | 22:00 | |
| *** leitan has quit IRC | 22:08 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720458 | 22:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 22:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: Most/all legacy-tripleo jobs failing on gate" [Critical,Triaged] | 22:10 |
| openstack | Launchpad bug 1720458 in tripleo "Lastest delorean pip package causes error in autodoc" [Critical,Triaged] - Assigned to wes hayutin (weshayutin) | 22:10 |
| *** akrivoka has quit IRC | 22:10 | |
| *** etingof has joined #tripleo | 22:10 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: latest version of DLRN breaks CI https://review.openstack.org/508628 | 22:11 |
| *** etingof has quit IRC | 22:17 | |
| EmilienM | weshay: approving https://review.openstack.org/#/c/508628/1 | 22:17 |
| weshay | EmilienM, oy you are fast w/ the +2 :) | 22:17 |
| weshay | EmilienM, I was hoping to see if it's happening upstream, but I can see zuul or any of the logs :( | 22:18 |
| weshay | EmilienM, toure and I just found that error on his box | 22:18 |
| weshay | I guess it's safe to pin the version | 22:18 |
| EmilienM | weshay: yes, before the weekend its safe | 22:21 |
| EmilienM | weshay: we can always revert it. | 22:21 |
| EmilienM | weshay: in case, i'm online this week (bad weather here) | 22:21 |
| *** jtomasek has quit IRC | 22:22 | |
| *** jtomasek has joined #tripleo | 22:23 | |
| *** d0ugal has quit IRC | 22:23 | |
| *** lblanchard has quit IRC | 22:24 | |
| *** jtomasek has quit IRC | 22:27 | |
| *** achadha has quit IRC | 22:45 | |
| *** achadha has joined #tripleo | 22:46 | |
| *** achadha_ has joined #tripleo | 22:48 | |
| *** achadha_ has quit IRC | 22:50 | |
| *** achadha has quit IRC | 22:50 | |
| *** toure is now known as toure|gone | 22:52 | |
| *** etingof has joined #tripleo | 22:52 | |
| *** achadha has joined #tripleo | 22:52 | |
| *** achadha has quit IRC | 22:53 | |
| *** achadha has joined #tripleo | 22:55 | |
| *** etingof has quit IRC | 22:57 | |
| *** achadha has quit IRC | 22:59 | |
| *** dspano has quit IRC | 23:04 | |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1718387 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1719123 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720220 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1720458 | 23:10 |
| openstack | Launchpad bug 1718387 in tripleo "ping test is periodically failing for the gate-tripleo-ci-centos-7-nonha-multinode-oooq " [Critical,Triaged] - Assigned to Marios Andreou (marios-b) | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1719123 in tripleo "tempest fails on overcloud keystone admin tenant error" [Critical,Triaged] - Assigned to Arx Cruz (arxcruz) | 23:10 |
| openstack | Launchpad bug 1720220 in tripleo "CI: Most/all legacy-tripleo jobs failing on gate" [Critical,Triaged] | 23:10 |
| openstack | Launchpad bug 1720458 in tripleo "Lastest delorean pip package causes error in autodoc" [Critical,In progress] - Assigned to wes hayutin (weshayutin) | 23:10 |
| *** achadha has joined #tripleo | 23:18 | |
| *** achadha has quit IRC | 23:22 | |
| *** tongl has quit IRC | 23:25 | |
| *** abishop has quit IRC | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!