| openstackgerrit | melissaml proposed openstack/os-net-config master: fix a typo on bond_mapped.yaml https://review.openstack.org/555508 | 00:07 |
|---|---|---|
| *** aputtur has joined #tripleo | 00:07 | |
| *** ledo_ has quit IRC | 00:08 | |
| *** sai_p has quit IRC | 00:09 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Add HostsEntry to undercloud's /etc/hosts https://review.openstack.org/555041 | 00:10 |
| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757111 | 00:10 |
| openstack | Launchpad bug 1757111 in tripleo " fs020(both queens/master) tempest tests failing while booting an instance" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757174 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 00:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1757174 in tripleo "tripleo-buildimage-overcloud-full-centos-7 failing with diskimage_builder.element_dependencies.MissingElementException: Element 'size=4096'' not found" [Critical,Triaged] | 00:10 |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Arx Cruz (arxcruz) | 00:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 00:10 |
| *** dmacpher has quit IRC | 00:11 | |
| openstackgerrit | melissaml proposed openstack/puppet-pacemaker master: fix a typo in doc https://review.openstack.org/555515 | 00:11 |
| *** radeks__ has quit IRC | 00:15 | |
| openstackgerrit | melissaml proposed openstack/tripleo-docs master: fix typos in documentation https://review.openstack.org/555532 | 00:20 |
| weshay | fak.. if everything that is lined up to merge does merge.. we have a shot at a promotion .. | 00:24 |
| *** weshay is now known as weshay_PTO | 00:25 | |
| *** hamzy has joined #tripleo | 00:30 | |
| openstackgerrit | melissaml proposed openstack/os-net-config master: fix a typo on bond_mapped.yaml https://review.openstack.org/555508 | 00:32 |
| EmilienM | weshay_PTO: can you list the patches here? | 00:33 |
| EmilienM | I guess that's the 4 alerts? | 00:33 |
| *** yamahata has quit IRC | 00:38 | |
| EmilienM | tell me if there is more, but AFIK: https://review.openstack.org/#/c/555077/ https://review.openstack.org/#/c/555056/ https://review.openstack.org/#/c/555468/ https://review.openstack.org/#/c/555305/ https://review.openstack.org/#/c/554705/ and https://review.openstack.org/#/c/555232/ | 00:40 |
| EmilienM | I'm clearing up the gate so we can land the blockers first | 00:42 |
| *** agopi|out has joined #tripleo | 00:43 | |
| *** EmilienM changes topic to "DO NOT RECHECK - Welcome to Rocky. CI status - Promotions: RED; check/gate: RED; RDO CI jobs: Questionable | http://tripleo.org/ | https://docs.openstack.org/tripleo-docs/latest/" | 00:43 | |
| EmilienM | @all: please do not recheck, please do not restore patches I've abandoned | 00:44 |
| EmilienM | I've cleared the gate to land urgent stuffs | 00:44 |
| EmilienM | I'll restore your patches, don't worry | 00:44 |
| *** aputtur has quit IRC | 00:56 | |
| *** ihrachys has quit IRC | 01:04 | |
| *** wolverineav has quit IRC | 01:04 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757111 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757174 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 01:10 |
| openstack | Launchpad bug 1757111 in tripleo " fs020(both queens/master) tempest tests failing while booting an instance" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstack | Launchpad bug 1757174 in tripleo "tripleo-buildimage-overcloud-full-centos-7 failing with diskimage_builder.element_dependencies.MissingElementException: Element 'size=4096'' not found" [Critical,Triaged] | 01:10 |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Arx Cruz (arxcruz) | 01:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 01:10 |
| *** yamahata has joined #tripleo | 01:13 | |
| openstackgerrit | Doug Hellmann proposed openstack/instack master: add lower-constraints job https://review.openstack.org/555565 | 01:17 |
| openstackgerrit | Doug Hellmann proposed openstack/instack-undercloud master: add lower-constraints job https://review.openstack.org/555566 | 01:17 |
| *** ebarrera_ has quit IRC | 01:18 | |
| *** dmacpher has joined #tripleo | 01:21 | |
| *** rlandy|bbl is now known as rlandy | 01:22 | |
| *** wolverineav has joined #tripleo | 01:36 | |
| *** ykarel has joined #tripleo | 02:05 | |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757111 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757174 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 02:10 |
| openstack | Launchpad bug 1757111 in tripleo " fs020(both queens/master) tempest tests failing while booting an instance" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1757174 in tripleo "tripleo-buildimage-overcloud-full-centos-7 failing with diskimage_builder.element_dependencies.MissingElementException: Element 'size=4096'' not found" [Critical,Triaged] | 02:10 |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Arx Cruz (arxcruz) | 02:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 02:10 |
| *** rlandy has quit IRC | 02:15 | |
| *** fzdarsky__ has joined #tripleo | 02:34 | |
| *** dougbtv__ has joined #tripleo | 02:34 | |
| *** dougbtv_ has quit IRC | 02:34 | |
| *** fzdarsky_ has quit IRC | 02:37 | |
| *** fzdarsky__ has quit IRC | 02:38 | |
| *** ramishra has joined #tripleo | 02:39 | |
| *** fzdarsky__ has joined #tripleo | 02:41 | |
| *** ykarel has quit IRC | 02:44 | |
| *** psachin has joined #tripleo | 02:52 | |
| *** wolverineav has quit IRC | 02:54 | |
| *** wolverineav has joined #tripleo | 02:55 | |
| *** ykarel has joined #tripleo | 02:57 | |
| *** wolverineav has quit IRC | 02:59 | |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757111 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757174 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 03:10 |
| openstack | Launchpad bug 1757111 in tripleo " fs020(both queens/master) tempest tests failing while booting an instance" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| openstack | Launchpad bug 1757174 in tripleo "tripleo-buildimage-overcloud-full-centos-7 failing with diskimage_builder.element_dependencies.MissingElementException: Element 'size=4096'' not found" [Critical,Triaged] | 03:10 |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Arx Cruz (arxcruz) | 03:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 03:10 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Temporarly mark 3node job as non-voting https://review.openstack.org/555305 | 03:14 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Install tripleo-common from source https://review.openstack.org/554705 | 03:14 |
| *** ykarel has quit IRC | 03:15 | |
| *** fragatin_ has joined #tripleo | 03:25 | |
| openstackgerrit | Merged openstack/tripleo-common master: Add openvswitch element back in https://review.openstack.org/555056 | 03:25 |
| *** fragatin_ has quit IRC | 03:27 | |
| *** fragatin_ has joined #tripleo | 03:28 | |
| *** fragatina has quit IRC | 03:29 | |
| *** fragatin_ has quit IRC | 03:33 | |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart-extras master: Refactored validate-tempest role for undercloud and containers https://review.openstack.org/551441 | 03:34 |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart master: Run tempest on fs027 using containerized tempest https://review.openstack.org/551442 | 03:42 |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart master: Run tempest on undercloud on fs003 https://review.openstack.org/554522 | 03:42 |
| *** fragatina has joined #tripleo | 03:45 | |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Checking if the overcloud group is empty https://review.openstack.org/555232 | 03:46 |
| *** fragatina has quit IRC | 03:49 | |
| *** fragatina has joined #tripleo | 03:54 | |
| *** itlinux has quit IRC | 03:59 | |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder master: [WIP] Move environment create/cleanup to python https://review.openstack.org/555463 | 04:01 |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757111 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757174 | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1757111 in tripleo " fs020(both queens/master) tempest tests failing while booting an instance" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 04:10 |
| *** shreshtha has joined #tripleo | 04:10 | |
| openstack | Launchpad bug 1757174 in tripleo "tripleo-buildimage-overcloud-full-centos-7 failing with diskimage_builder.element_dependencies.MissingElementException: Element 'size=4096'' not found" [Critical,Triaged] | 04:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 04:10 |
| *** jaganathan has quit IRC | 04:13 | |
| *** dobson has quit IRC | 04:16 | |
| *** fragatina has quit IRC | 04:19 | |
| *** fragatina has joined #tripleo | 04:20 | |
| *** dobson has joined #tripleo | 04:20 | |
| *** weshay_PTO is now known as weshay | 04:21 | |
| *** links has joined #tripleo | 04:26 | |
| *** ykarel has joined #tripleo | 04:28 | |
| *** pgadiya has joined #tripleo | 04:29 | |
| *** moshele has joined #tripleo | 04:29 | |
| *** alee_ has quit IRC | 04:34 | |
| *** kambiz has quit IRC | 04:36 | |
| *** Tyrantelf has quit IRC | 04:36 | |
| *** alee_ has joined #tripleo | 04:36 | |
| *** Tyrantelf has joined #tripleo | 04:37 | |
| *** kambiz has joined #tripleo | 04:38 | |
| *** udesale has joined #tripleo | 04:41 | |
| *** pgadiya has quit IRC | 04:43 | |
| *** pickle has quit IRC | 04:45 | |
| *** skramaja has joined #tripleo | 04:47 | |
| *** ykarel_ has joined #tripleo | 04:47 | |
| *** ykarel has quit IRC | 04:50 | |
| *** pgadiya has joined #tripleo | 04:58 | |
| *** ratailor has joined #tripleo | 04:59 | |
| *** dpawar has joined #tripleo | 05:02 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757111 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757174 | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1757111 in tripleo " fs020(both queens/master) tempest tests failing while booting an instance" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 05:10 |
| openstack | Launchpad bug 1757174 in tripleo "tripleo-buildimage-overcloud-full-centos-7 failing with diskimage_builder.element_dependencies.MissingElementException: Element 'size=4096'' not found" [Critical,Triaged] | 05:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 05:10 |
| *** pgadiya has quit IRC | 05:19 | |
| *** jaganathan has joined #tripleo | 05:20 | |
| *** moshele has quit IRC | 05:21 | |
| *** gyankum has joined #tripleo | 05:23 | |
| *** psahoo has joined #tripleo | 05:25 | |
| *** pgadiya has joined #tripleo | 05:33 | |
| *** agopi|out has quit IRC | 05:33 | |
| *** mdnadeem has joined #tripleo | 05:34 | |
| openstackgerrit | Merged openstack/diskimage-builder master: proliant-tools: add net-tools package to support hpsum utility https://review.openstack.org/548154 | 05:36 |
| *** yamahata has quit IRC | 05:38 | |
| *** holser__ has joined #tripleo | 05:39 | |
| Tengu | hello there | 05:43 |
| *** pdeore has joined #tripleo | 05:43 | |
| *** yprokule has joined #tripleo | 05:47 | |
| *** dmacpher has quit IRC | 05:47 | |
| *** anilvenkata has joined #tripleo | 05:51 | |
| *** holser__ has quit IRC | 05:53 | |
| *** dpawar has quit IRC | 06:02 | |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757111 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757174 | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstack | Launchpad bug 1757111 in tripleo " fs020(both queens/master) tempest tests failing while booting an instance" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 06:10 |
| openstack | Launchpad bug 1757174 in tripleo "tripleo-buildimage-overcloud-full-centos-7 failing with diskimage_builder.element_dependencies.MissingElementException: Element 'size=4096'' not found" [Critical,Triaged] | 06:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 06:10 |
| *** mmethot has joined #tripleo | 06:16 | |
| *** _mmethot_ has quit IRC | 06:18 | |
| *** fragatina has quit IRC | 06:23 | |
| *** mmethot has quit IRC | 06:27 | |
| *** mmethot has joined #tripleo | 06:27 | |
| *** masco has joined #tripleo | 06:28 | |
| *** radeks__ has joined #tripleo | 06:30 | |
| *** dbecker has quit IRC | 06:30 | |
| *** dpawar has joined #tripleo | 06:38 | |
| *** quiquell has joined #tripleo | 06:40 | |
| *** quiquell has quit IRC | 06:41 | |
| *** quiquell has joined #tripleo | 06:42 | |
| *** jfrancoa has joined #tripleo | 06:43 | |
| *** marios has joined #tripleo | 06:44 | |
| *** dbecker has joined #tripleo | 06:45 | |
| *** hjensas has quit IRC | 06:45 | |
| *** dpawar has quit IRC | 06:47 | |
| *** fragatina has joined #tripleo | 06:48 | |
| *** aufi has joined #tripleo | 06:55 | |
| *** ykarel_ is now known as ykarel|lunch | 06:58 | |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-heat-templates master: Apply the vhost group parameter for the vhost_sockets directory creation https://review.openstack.org/555661 | 07:01 |
| *** karthiks has quit IRC | 07:05 | |
| *** dpawar has joined #tripleo | 07:06 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757111 | 07:10 |
| openstack | Launchpad bug 1757111 in tripleo " fs020(both queens/master) tempest tests failing while booting an instance" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757174 | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1757174 in tripleo "tripleo-buildimage-overcloud-full-centos-7 failing with diskimage_builder.element_dependencies.MissingElementException: Element 'size=4096'' not found" [Critical,Triaged] | 07:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 07:10 |
| *** pgadiya has quit IRC | 07:16 | |
| *** jtomasek has joined #tripleo | 07:17 | |
| *** karthiks has joined #tripleo | 07:18 | |
| *** hanish has joined #tripleo | 07:19 | |
| *** dparkes has joined #tripleo | 07:20 | |
| *** pcaruana has joined #tripleo | 07:21 | |
| *** masco has quit IRC | 07:23 | |
| *** dparkes has quit IRC | 07:24 | |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Refactor PlansActions to use startWorkflow https://review.openstack.org/552638 | 07:26 |
| *** pgadiya has joined #tripleo | 07:30 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/instack-undercloud master: Updated from global requirements https://review.openstack.org/555672 | 07:30 |
| *** cylopez has joined #tripleo | 07:33 | |
| chandankumar | Hello | 07:36 |
| *** masco has joined #tripleo | 07:36 | |
| *** cylopez has left #tripleo | 07:39 | |
| hanish | hi all i am trying queens tripleo installation, but overcloud deployment fails with failed to pull image: docker.io/tripleomaster/centos-binary-neutron-server:current-tripleo | 07:40 |
| *** gkadam has joined #tripleo | 07:41 | |
| hanish | Can someone help me on this | 07:41 |
| Tengu | hanish: hello, you might want to take the tripleoqueens namespace instead of tripleomaster | 07:42 |
| hanish | @Tengu, how can i do that | 07:43 |
| Tengu | hanish: there's a command that will generate a docker registry configuration file. | 07:43 |
| *** florianf has joined #tripleo | 07:43 | |
| Tengu | 2s, searching | 07:44 |
| Tengu | hanish: https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/overcloud.html#preparing-the-environment | 07:44 |
| hanish | @Tengu, i do not want containerized installation, i would like general installation, i am using openstack overcloud deploy --templates | 07:44 |
| Tengu | hanish: general install is now containers. | 07:45 |
| Tengu | don't try the baremetal one, as its support will be removed. I'm currently strugling migrating my baremetal env to containers, I can assure you you don't want to do that :) | 07:46 |
| hanish | @tengu thanks for the info, i would try the container based installation, but is there a way to try baremetal one | 07:47 |
| *** dparkes has joined #tripleo | 07:48 | |
| Tengu | hanish: don't think so - I think all the defaults are so to containers now for queens. | 07:48 |
| *** dpawar has quit IRC | 07:48 | |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Refactor PlansActions to use startWorkflow https://review.openstack.org/552638 | 07:48 |
| hanish | @Tengu thank you very much, i would try it. | 07:50 |
| Tengu | :) | 07:50 |
| * Tengu goes back to head-banging on ceph for his migration | 07:50 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-tripleoclient master: Updated from global requirements https://review.openstack.org/555234 | 07:52 |
| *** dparkes has quit IRC | 07:53 | |
| openstackgerrit | Harald Jensås proposed openstack/instack-undercloud master: Fix next_hop for metadata service host route on local_subnet https://review.openstack.org/554908 | 07:54 |
| *** holser__ has joined #tripleo | 07:55 | |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/472607 | 07:55 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common master: Updated from global requirements https://review.openstack.org/555687 | 07:56 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-validations master: Updated from global requirements https://review.openstack.org/555688 | 07:56 |
| openstackgerrit | Harald Jensås proposed openstack/python-tripleoclient master: Fix pre-flight validation undercloud-passwords.conf https://review.openstack.org/554674 | 07:59 |
| *** ccamacho has joined #tripleo | 08:02 | |
| ccamacho | happy (red)ocky friday! | 08:03 |
| *** hjensas has joined #tripleo | 08:10 | |
| *** hjensas has quit IRC | 08:10 | |
| *** hjensas has joined #tripleo | 08:10 | |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757111 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757174 | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1757111 in tripleo " fs020(both queens/master) tempest tests failing while booting an instance" [Critical,In progress] - Assigned to Alex Schultz (alex-schultz) | 08:10 |
| openstack | Launchpad bug 1757174 in tripleo "tripleo-buildimage-overcloud-full-centos-7 failing with diskimage_builder.element_dependencies.MissingElementException: Element 'size=4096'' not found" [Critical,Triaged] | 08:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 08:10 |
| *** ebarrera has joined #tripleo | 08:12 | |
| *** kopecmartin has joined #tripleo | 08:13 | |
| *** ebarrera has quit IRC | 08:15 | |
| *** ebarrera has joined #tripleo | 08:15 | |
| *** paramite_ has joined #tripleo | 08:15 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Include pacemaker in Telemetry role https://review.openstack.org/554307 | 08:18 |
| *** tesseract has joined #tripleo | 08:20 | |
| *** bogdando has joined #tripleo | 08:26 | |
| d0ugal | shardy: Hey | 08:26 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui master: Don't pass history with message actions https://review.openstack.org/555696 | 08:28 |
| *** dparkes has joined #tripleo | 08:29 | |
| Tengu | anyone with some ceph knowledge in here? I'm having a hard time with it… | 08:32 |
| jaosorior | Tengu: gotta poke gfidente or fultonj | 08:34 |
| Tengu | jaosorior: ah, wasn't sure about fultonj :). gfidente isn't here for now | 08:34 |
| *** egallen has joined #tripleo | 08:34 | |
| Tengu | sooo… fultonj hello, are you here? would you have a moment for some ceph issues in a baremetal -> container migration process? :) | 08:35 |
| jaosorior | Tengu: fultonj is in the states | 08:35 |
| Tengu | oh. so asleep. | 08:35 |
| *** gfidente has joined #tripleo | 08:35 | |
| Tengu | can't remember the name of the other one that was with gfidenty at PTG - I think he's un Poland or, at least, Europe. | 08:36 |
| *** amoralej|off is now known as amoralej | 08:36 | |
| d0ugal | that narrows it down ;) | 08:36 |
| Tengu | d0ugal: hmm. not sure it was you :). but do you have some knowledge with ceph? :D | 08:37 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Delete rendered nic configs during --clean https://review.openstack.org/549865 | 08:37 |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Remove composable role config from testenv https://review.openstack.org/555468 | 08:37 |
| d0ugal | Tengu: Not really! | 08:37 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: Fixes ODL container failing to start due to missing etc config https://review.openstack.org/555035 | 08:37 |
| *** cshastri has joined #tripleo | 08:37 | |
| Tengu | although you took part in the move to ceph-ansible ;). well. I'll dig a bit more, hopefully gfidente will connect today :) | 08:38 |
| Tengu | (and have time) | 08:38 |
| d0ugal | Tengu: sort-of. I'm Mistral/Workflows person, so I helped with some of the plumbing | 08:38 |
| d0ugal | ... but I have never actually used Ceph. | 08:38 |
| *** egallen has quit IRC | 08:39 | |
| Tengu | ok. so yeah. no way you help me with the auth/encryption issue I have :'( | 08:39 |
| *** dpawar has joined #tripleo | 08:39 | |
| *** pgadiya has quit IRC | 08:41 | |
| *** pgadiya has joined #tripleo | 08:41 | |
| openstackgerrit | Merged openstack/puppet-tripleo stable/queens: Fixes incorrect ownership of ODL TLS cert/key https://review.openstack.org/554909 | 08:42 |
| *** tosky has joined #tripleo | 08:45 | |
| openstackgerrit | Merged openstack/os-net-config master: Adding ethtool command after binding dpdk drivers in Mellanox nics https://review.openstack.org/554048 | 08:45 |
| Tengu | oh, speaking of… | 08:46 |
| Tengu | hello gfidente :)) | 08:46 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Mount openvswitch dir rather than socket https://review.openstack.org/555077 | 08:47 |
| openstackgerrit | Merged openstack/tripleo-puppet-elements master: add puppet-vitrage and vitrage rpms into image https://review.openstack.org/553351 | 08:47 |
| Tengu | gfidente: sooo… I've checked the time on all my controllers, checked the content of the different keyrings, checked the content of the ceph config in the only container running, and apparently all is OK. even ceph versions are the same in the container and on the baremetal… I'm lost. The error is still here, stating it can't decrypt messages. | 08:49 |
| Tengu | gfidente: if you have any idea… | 08:49 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add the service_config_settings from nova-compute base https://review.openstack.org/554596 | 08:49 |
| *** jpena|off is now known as jpena | 08:49 | |
| *** akrivoka has joined #tripleo | 08:53 | |
| shardy | d0ugal: Hey, good morning, sorry missed your ping earlier | 08:54 |
| Tengu | o____O wait… restarting one of the controllers without containerized ceph seems to do things… what the f**k ?! | 08:57 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Fix checking if the overcloud group is empty https://review.openstack.org/555703 | 08:58 |
| *** cshastri has quit IRC | 08:58 | |
| Tengu | ah. nope. | 08:59 |
| Tengu | GOT IT | 09:01 |
| Tengu | wow | 09:01 |
| Tengu | gfidente: there's a "big" change when we pass from puppet-ceph to ceph-ansible: the mon keyring located in /var/lib/ceph/mon/ changes! with puppet-ceph, a directory named after the HOSTNAME only is created, with proper keys and so. Apparently, ceph-ansible creates a new directory based on the FQDN instead, with new keyring. | 09:02 |
| Tengu | I stumbled on this blogpost: https://medium.com/@george.shuklin/silly-mistakes-with-ceph-mon-9ef6c9eaab54 and saw that major difference. | 09:03 |
| jaosorior | bandini, dciabrin_: If I install a new version of a resource agent (install a new script), is it necessary to delete the resources or do they get automatically picked up? | 09:04 |
| dciabrin_ | jaosorior, they get picked up. pacemaker spawns a new shell script any time it schedules a monitor operation (every 20s or so) | 09:05 |
| dciabrin_ | jaosorior, that's for non containerized env, isn't it? | 09:06 |
| *** jpich has joined #tripleo | 09:06 | |
| Tengu | so in order to make it work, the best thing to do would be to copy the puppet-ceph directory, at least the keyring. | 09:06 |
| bandini | jaosorior: they get automatically installed and picked up *except* for galera/rabbit/redis inside containers, which will need a rebuild | 09:08 |
| bandini | oh dciabrin replied already :) | 09:08 |
| bandini | me-- | 09:08 |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757174 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1757174 in tripleo "tripleo-buildimage-overcloud-full-centos-7 failing with diskimage_builder.element_dependencies.MissingElementException: Element 'size=4096'' not found" [Critical,Triaged] | 09:10 |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 09:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 09:10 |
| *** anilvenkata has quit IRC | 09:12 | |
| *** anilvenkata has joined #tripleo | 09:12 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-upgrade master: Add roles-data into overcloud upgrade prepare. https://review.openstack.org/555370 | 09:13 |
| *** lucas-afk is now known as lucasagomes | 09:13 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-upgrade stable/queens: Add roles-data into overcloud upgrade prepare. https://review.openstack.org/555389 | 09:18 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-upgrade stable/queens: DNM: Testing P->Q from queens branch. https://review.openstack.org/552080 | 09:18 |
| *** sri_ has joined #tripleo | 09:20 | |
| openstackgerrit | Harald Jensås proposed openstack/puppet-tripleo master: Add support for Ironic Networking Baremetal https://review.openstack.org/545434 | 09:20 |
| jaosorior | bandini, dciabrin_: thanks! | 09:20 |
| *** rcernin has quit IRC | 09:20 | |
| matbu | marios: jistr if we change this : https://review.openstack.org/#/c/554484/1/workbooks/package_update.yaml | 09:21 |
| matbu | marios: jistr we need to provide the value for the heat-admin user in the cli | 09:21 |
| *** ktibi has joined #tripleo | 09:22 | |
| *** ratailor has quit IRC | 09:25 | |
| openstackgerrit | Damien Ciabrini proposed openstack/tripleo-heat-templates stable/queens: Fix update of pacemaker container images during major upgrade https://review.openstack.org/555713 | 09:26 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Set ExternalInterfaceDefaultRoute when setting ExternalNetCidr https://review.openstack.org/554225 | 09:27 |
| *** radeks_ has joined #tripleo | 09:27 | |
| *** ratailor has joined #tripleo | 09:27 | |
| *** radeks__ has quit IRC | 09:28 | |
| *** stendulker has joined #tripleo | 09:29 | |
| *** radeks__ has joined #tripleo | 09:29 | |
| *** radeks_ has quit IRC | 09:32 | |
| *** chem|eod is now known as chem | 09:33 | |
| *** panda|off is now known as panda | 09:33 | |
| *** quiquell has quit IRC | 09:35 | |
| *** suuuper has joined #tripleo | 09:36 | |
| *** gkadam is now known as gkadam-afk | 09:36 | |
| *** derekh has joined #tripleo | 09:37 | |
| *** matbu has quit IRC | 09:38 | |
| *** dbecker has quit IRC | 09:39 | |
| openstackgerrit | waleed mousa proposed openstack/os-net-config stable/queens: Adding ethtool command after binding dpdk drivers in Mellanox nics https://review.openstack.org/555718 | 09:40 |
| *** ykarel|lunch is now known as ykarel | 09:40 | |
| *** gkadam-afk is now known as gkadam | 09:43 | |
| *** dsariel has quit IRC | 09:44 | |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Use TLS by default with autogenerated certificate in the undercloud https://review.openstack.org/552781 | 09:44 |
| *** quiquell has joined #tripleo | 09:44 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add EnablePuppet (defaults to true) https://review.openstack.org/550192 | 09:45 |
| openstackgerrit | Merged openstack/tripleo-validations master: Updated from global requirements https://review.openstack.org/555688 | 09:45 |
| *** matbu has joined #tripleo | 09:45 | |
| *** quiquell has quit IRC | 09:46 | |
| *** ratailor has quit IRC | 09:52 | |
| *** bogdando has quit IRC | 09:52 | |
| *** bogdando has joined #tripleo | 09:53 | |
| *** ratailor has joined #tripleo | 09:53 | |
| *** quiquell has joined #tripleo | 09:56 | |
| *** quiquell has quit IRC | 09:56 | |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-docs master: WIP: Add FFU docs https://review.openstack.org/549892 | 09:56 |
| Tengu | in fact, this BM to container migration is a lot like super-mario, or final fantasy: you have to unlock each level in order to end the game. My current boss level: ceph. Once it's unlocked, I have no idea what will be behind the door :D | 09:57 |
| *** dsariel has joined #tripleo | 09:57 | |
| *** quiquell has joined #tripleo | 09:58 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-common master: TLS by default for the overcloud if no certificate provided https://review.openstack.org/554926 | 09:59 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-upgrade master: Add the ability to limit the hosts where to apply FFU https://review.openstack.org/554833 | 10:03 |
| *** dsariel has quit IRC | 10:03 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates stable/newton: Work around faulty package in iptables. https://review.openstack.org/554647 | 10:08 |
| holser__ | stevebaker - are you around | 10:08 |
| *** rcernin has joined #tripleo | 10:08 | |
| holser__ | concerning https://review.openstack.org/#/c/554676/3 I do really want to move that to RPC/REST and not to have dependency from tripleo-common | 10:09 |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757174 | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 10:10 |
| openstack | Launchpad bug 1757174 in tripleo "tripleo-buildimage-overcloud-full-centos-7 failing with diskimage_builder.element_dependencies.MissingElementException: Element 'size=4096'' not found" [Critical,Triaged] | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 10:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 10:10 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Add environment/services-baremetal/*. https://review.openstack.org/550081 | 10:11 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Default environment/services/* to docker https://review.openstack.org/550060 | 10:11 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-common stable/pike: Increase the image size for security hardened images to 23G https://review.openstack.org/555726 | 10:11 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-image-elements stable/pike: Increase the /var partition from 7G to 10G https://review.openstack.org/553768 | 10:12 |
| *** suuuper has quit IRC | 10:14 | |
| *** suuuper has joined #tripleo | 10:14 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-heat-templates master: Enable ironic rescue mode by default https://review.openstack.org/555336 | 10:14 |
| *** salmankhan has joined #tripleo | 10:15 | |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-common stable/queens: Increase the image size for security hardened images to 23G https://review.openstack.org/555729 | 10:20 |
| *** salmankhan1 has joined #tripleo | 10:20 | |
| *** liverpooler has quit IRC | 10:21 | |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-image-elements stable/queens: Increase the image size to 23G https://review.openstack.org/555730 | 10:21 |
| *** salmankhan has quit IRC | 10:22 | |
| *** salmankhan1 is now known as salmankhan | 10:22 | |
| shardy | holser__: stevebaker is based in New Zealand so probably won't be around now until Monday - you could drop him a mail perhaps | 10:24 |
| shardy | holser__: FWIW tripleoclient already depends on tripleo-common? | 10:24 |
| holser__ | nope | 10:24 |
| holser__ | but I'll talk to stevebaker | 10:25 |
| shardy | https://github.com/openstack/python-tripleoclient/blob/master/requirements.txt#L19 | 10:25 |
| shardy | holser__: ^^ | 10:25 |
| *** dbecker has joined #tripleo | 10:26 | |
| shardy | holser__: all of the workflows and a bunch of utils are imported from tripleo common | 10:26 |
| holser__ | hmmm | 10:26 |
| holser__ | ok | 10:26 |
| holser__ | that's why I want to put workflows to t-h-t | 10:26 |
| shardy | actually we just reference the wofkflow names, but grep shows the utils we depend on | 10:26 |
| holser__ | as common is bloated by other things | 10:26 |
| shardy | holser__: yeah the workflows could move somewhere, not sure tht is the right place though | 10:27 |
| shardy | as workflows aren't part of the plan managed by users | 10:27 |
| holser__ | we may create tripleo-workflows repo | 10:27 |
| shardy | we could create a new repo though if having them in -common is causing problems | 10:27 |
| holser__ | and move them to that repo | 10:27 |
| holser__ | at least it's logical | 10:27 |
| shardy | yeah that would probably be best - would we put the associated actions in there too? | 10:27 |
| holser__ | yeah | 10:28 |
| shardy | some things have effectively been incubated in -common I guess | 10:28 |
| holser__ | yeah - but if we talk about security - easier to have lightweight repos/packages to pass audit | 10:28 |
| holser__ | also CI and other staff | 10:28 |
| holser__ | stuff* | 10:28 |
| holser__ | and workflow team will be happy as it should improve speed | 10:29 |
| shardy | holser__: Ok I'm not sure I get that argument, but I agree it makes logical sense to move the workflows and actions somewhere, as they are not required by the client and other things | 10:29 |
| shardy | holser__: how will it improve speed? | 10:29 |
| holser__ | dev velocity | 10:29 |
| holser__ | not speed | 10:29 |
| holser__ | sorry | 10:29 |
| shardy | holser__: yeah, how - you mean having folks core only on that repo or something? | 10:30 |
| holser__ | yeah! | 10:30 |
| d0ugal | holser__, shardy: the workflows squad is planning on proposing a tripleo-workflows repo | 10:30 |
| d0ugal | but we just hadn't gotten around to it yet | 10:30 |
| holser__ | d0ugal that's what I am talking about | 10:30 |
| shardy | holser__: sure, we've kinda already done that informally several times - and also IMO folks should strive to review more than just a subset of the TripleO architecture, even if they start in one area | 10:31 |
| d0ugal | +1 | 10:31 |
| shardy | but regardless +1 on a new repo, it makes no sense for e.g tripleoclient to depend on all the workflows/actions etc | 10:31 |
| holser__ | +1 | 10:32 |
| d0ugal | shardy: btw I am on PTO next week. I had been meaning to mention this earlier | 10:34 |
| d0ugal | shardy: is there anything I can do with the workflow patches I put up today? | 10:34 |
| bandini | arxcruz|rover, panda: are there any chances dciabrin and I can get a live environment that is failing on for https://bugs.launchpad.net/tripleo/+bug/1757556 | 10:35 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 10:35 |
| shardy | d0ugal: ack, I'm about to pull and test them, but I think we're waiting on UI feedback from jtomasek | 10:35 |
| bandini | we spent some time on logs, but did not get many clues | 10:35 |
| arxcruz|rover | bandini: weshay has a env up i think, but he's not online yet | 10:35 |
| d0ugal | shardy: Yeah, I think so | 10:35 |
| shardy | d0ugal: if it's cool with you I'll push to them while you're away and coordinate with thrash/rbrady so we can continue to make progress | 10:35 |
| d0ugal | shardy: feel free to make any changes while I'm out - I guess the swift paths will likely need a tweak | 10:36 |
| shardy | I'm hoping we can get the UI deploying via config download next week | 10:36 |
| bandini | arxcruz|rover: ack | 10:36 |
| d0ugal | shardy: apetrich may be able to help too, if you need a european tz :) | 10:36 |
| shardy | d0ugal: ack thanks will remember that :) | 10:36 |
| panda | bandini: for you, no | 10:37 |
| apetrich | shardy, neat | 10:37 |
| bandini | panda: lol | 10:38 |
| panda | bandini: it's 3nodes multinode, so it's not in rdocloud. Not that easy to hold servers for debugging there | 10:39 |
| bandini | I see | 10:39 |
| panda | bandini: have you tried the reproducer script ? | 10:39 |
| openstackgerrit | Oliver Walsh proposed openstack/puppet-tripleo master: WIP: fail hard if we detect that the neutron/nova/ceilometer host has changed https://review.openstack.org/555732 | 10:40 |
| bandini | panda: no last time I tried reproducer script it exploded super badly. I'll try again though, maybe I am in luck | 10:40 |
| *** hanish has quit IRC | 10:40 | |
| *** dtantsur|afk is now known as dtantsur | 10:41 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: [FFU] Hook to allow user to pass a custom script for repo switching. https://review.openstack.org/554953 | 10:41 |
| panda | bandini: we are interested in the explosions too, also I don't think you have many choices. Go, and reproduce, we'll watch your back. | 10:41 |
| * bandini wears helmet and reproduces all the things | 10:42 | |
| *** zoli is now known as zoli|lunch | 10:42 | |
| dtantsur | I tried reproduce-quickstart yesterday, it just hung :) | 10:42 |
| *** pblaho has joined #tripleo | 10:43 | |
| owalsh | chem: FYI https://review.openstack.org/555732 is the best solution I could think of for the nova/neutron host changing... | 10:43 |
| panda | dtantsur: does that happen often ? | 10:43 |
| dtantsur | panda: 1 of 1 :) I can try a few more times for statistics | 10:44 |
| panda | 100% failure is not that bad | 10:44 |
| dtantsur | generally, the idea is awesome. just the instance (undercloud?) never got ssh-able | 10:44 |
| * dtantsur notices "DO NOT RECHECK - Welcome to Rocky" in the subject. Well, welcome indeed :D | 10:44 | |
| chem | owalsh: going to look thanks | 10:45 |
| jtomasek | d0ugal: I am trying to get the patches tested today with GUI code, to see how I am able to retrieve the status from the messages stored in swift | 10:46 |
| jtomasek | d0ugal: but I'll probably be able to report on that on monday | 10:46 |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: WIP: Explicitly set nova/neutron/ceilometer host to expected fqdn https://review.openstack.org/555278 | 10:47 |
| owalsh | chem: this should be safer now ^^^ | 10:48 |
| d0ugal | jtomasek: Great. If you missed it, I am on PTO next week. | 10:48 |
| panda | dtantsur: please retry, the reproducer is really important, we want to know if it's not working properly | 10:48 |
| jtomasek | d0ugal: ack | 10:48 |
| dtantsur | panda: I have quite a few patches open, some of them do fail the CI :) so will try | 10:48 |
| d0ugal | jtomasek: but anyone should feel free to update the patch if needed. | 10:49 |
| d0ugal | jtomasek: I suspect you might want us to change/tweak the swift paths if nothing else | 10:49 |
| jtomasek | d0ugal, shardy: I am on PTO from wednesday till the end of next week | 10:49 |
| d0ugal | ah :) | 10:49 |
| jtomasek | d0ugal: by end of tuesday I plan to have config-download GUI integration patch ready to for others to test | 10:50 |
| *** dsariel has joined #tripleo | 10:50 | |
| d0ugal | jtomasek: great, I look forward to a demo when I get back ;) | 10:50 |
| *** kmy has quit IRC | 10:52 | |
| *** kmy has joined #tripleo | 10:53 | |
| * Tengu battling Ceph Boss | 10:53 | |
| Tengu | hopefully I'll beat him this time, and be able to go forward :] | 10:53 |
| Tengu | tripleo migration, that super-mario-like game :] | 10:54 |
| Tengu | only issue: you can't save during the play, have to finish it or restart from 0. | 10:54 |
| quiquell | Hi, I am trying to run the openstack python client from my laptop towards my libvirt overcloud | 10:57 |
| quiquell | I am doing this: OS_CACERT=$(pwd)/overcloud-cacert.pem openstack server list | 10:58 |
| quiquell | But getting CERTIFICATE_VERIFY_FAILED, anyone has an idea of the problem ? | 10:58 |
| shardy | jtomasek: ack sounds good - let me know if I can help modify the workflow stuff etc, and perhaps you can hand over the UI work before you leave next week? | 10:59 |
| shardy | jtomasek: I was unsure if you wanted to get the status directly from swift, or if we need a wrapper "get plan status" workflow? | 10:59 |
| d0ugal | shardy: that is a pretty good idea. Then if we ever change the location etc. it can be hidden. | 11:00 |
| shardy | d0ugal: Yeah and if we needed to add logic, e.g check the status of the stack or whatever, it could be transparent | 11:00 |
| jtomasek | shardy: I'll know exactly if that's needed on monday | 11:00 |
| shardy | jtomasek: Ok sounds good, I'm around Monday so we can iterate on it if needed | 11:00 |
| jtomasek | shardy: ack | 11:01 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci master: Install the zuul-cloned dib and tripleo-common https://review.openstack.org/555737 | 11:01 |
| openstackgerrit | Derek Higgins proposed openstack/tripleo-common stable/queens: Build images with a 4k block size filesystem https://review.openstack.org/554493 | 11:02 |
| jaosorior | quiquell: a lot of things could be wrong there. First of all, what's the URL you're trying to access? | 11:02 |
| *** pkovar has joined #tripleo | 11:03 | |
| quiquell | jaosorior: https://[triple quickstart hostname]:13000/v2.0/tokens | 11:05 |
| dtantsur | panda: today it failed much faster: keystoneauth1.exceptions.http.BadRequest: Expecting to find domain in project | 11:06 |
| *** udesale has quit IRC | 11:06 | |
| jaosorior | quiquell: you can do openssl x509 -connect [tripleo quickstart hostname]:13000 | 11:06 |
| jaosorior | quiquell: that will tell you if the certificate matches the hostname | 11:07 |
| jaosorior | it will also tell you other errors, such as, if the cert is trusted or not | 11:07 |
| dtantsur | panda: I suspect OVB is not compatible with keystone v3 or what.. | 11:07 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates stable/queens: Include pacemaker in Telemetry role https://review.openstack.org/555739 | 11:08 |
| panda | dtantsur: try using v2 credentials | 11:08 |
| dtantsur | panda: yeah, I have to prepare them first.. | 11:09 |
| panda | dtantsur: you can download the credentials file from rdcloud directly | 11:09 |
| dtantsur | like, how long has keystone v2 been dead? several years? :) | 11:09 |
| dtantsur | panda: it's easier to create a copy of the normal credentials, I suspect | 11:10 |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757174 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1757174 in tripleo "tripleo-buildimage-overcloud-full-centos-7 failing with diskimage_builder.element_dependencies.MissingElementException: Element 'size=4096'' not found" [Critical,Triaged] | 11:10 |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 11:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 11:10 |
| *** athomas_ has joined #tripleo | 11:14 | |
| *** dprince has joined #tripleo | 11:14 | |
| dtantsur | panda: it hangs on "return stack state", and the stack is CREATE_FAILED | 11:15 |
| dtantsur | Error validating value 'key-52727': The Key (key-52727) could not be found | 11:15 |
| *** adarazs is now known as adarazs_lunch | 11:16 | |
| dtantsur | which is true, I have key-46831 | 11:16 |
| * dtantsur retries | 11:17 | |
| panda | dtantsur: hhmm should be automatically generated | 11:18 |
| dtantsur | panda: it was generated, but the names did not match somehow | 11:19 |
| ykarel | dtantsur, are you using same workdir, if yes i think it's happening due to ansible cache thing | 11:19 |
| dtantsur | ykarel: same workdir as what? I'm not passing any workdir argument | 11:19 |
| dtantsur | okay, now the same error, and it did not even create any additional keypairs | 11:20 |
| ykarel | dtantsur, how are you deploying, reproducer script, or directly quickstart.sh | 11:20 |
| dtantsur | ykarel: running https://logs.rdoproject.org/54/552554/5/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/Z61c4159b1bcd4ceab5aa56f8e8eac028/reproducer-quickstart.sh | 11:20 |
| dtantsur | with no arguments | 11:21 |
| *** moshele has joined #tripleo | 11:21 | |
| ykarel | it auto creates different workdir for each run, so it should be ok | 11:21 |
| dtantsur | yep, it seems to bootstrap everything from scratch | 11:22 |
| *** moshele has quit IRC | 11:26 | |
| ykarel | dtantsur, can you check env.yaml is created correctly | 11:27 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-quickstart-extras master: Parametrize overcloud-prep-containers script name. https://review.openstack.org/548472 | 11:27 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-quickstart-extras master: Add support to mixed upgrade for overcloud-prep-container role. https://review.openstack.org/540473 | 11:27 |
| *** pcaruana has quit IRC | 11:27 | |
| openstackgerrit | Merged openstack/tripleo-quickstart master: Support multiple environment files https://review.openstack.org/554473 | 11:28 |
| dtantsur | ykarel: nothing looks suspicious. it mentions key_name: 'key-94346', but this keypair was not created | 11:29 |
| *** stendulker has quit IRC | 11:30 | |
| chem | owalsh: oki, so I think that the issue is coming from something in rhel that changed what is considered fqdn, that would mean for ugprade that we would fail during converge ... so it may be something that we have to deal with during upgrade anyway one way or another (I'm depressed) ... so that fail hard review ... maybe we should put it on hold when we have a better grasp of what is happening here. | 11:30 |
| chem | owalsh: ouch long one | 11:30 |
| ykarel | dtantsur, and in ansible logs what is output of task: "Add keypair" | 11:32 |
| *** ansmith has quit IRC | 11:32 | |
| chem | owalsh: by the way when you're saying that facter is unpredictable what do you mean, that it can return different fqdn ? | 11:32 |
| *** bfournie has quit IRC | 11:32 | |
| dtantsur | ykarel: are there any detailed logs? I don't see such task in the output | 11:32 |
| chem | owalsh: because maybe your main patch is the solution :) | 11:32 |
| *** bfournie has joined #tripleo | 11:32 | |
| ykarel | dtantsur, only on the console i think | 11:33 |
| owalsh | chem: there are a few issues... once is that the fqdn fact changes at some point, another is where CloudDomain hasn't been set correctly at initial deployment | 11:33 |
| dtantsur | ykarel: ah, wait | 11:33 |
| dtantsur | fatal: [localhost]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result"} | 11:33 |
| dtantsur | ...ignoring | 11:33 |
| owalsh | chem: ultimately the result is the nova/neutron/ceilo host changing... and bad things happen | 11:34 |
| dtantsur | well, we need an uncensored version :) | 11:34 |
| ykarel | dtantsur, hm it failed due to some reason | 11:34 |
| chem | owalsh: oki, that's exactly our issue here | 11:34 |
| owalsh | chem: e.g live migration doesn't work as it's using the wrong IP | 11:34 |
| ykarel | dtantsur, yesterday you told that you are using keystone v3 credentials, can you try using v2 once | 11:34 |
| dtantsur | ykarel: why do we censor the log output? the public key is not secret... | 11:34 |
| chem | owalsh: "our" => complete lost of l3 connectivity after upgrade because neutron::host changed | 11:34 |
| ykarel | dtantsur, i don't know that | 11:34 |
| dtantsur | ykarel: I'm using v2, v3 proved not working with OVB indeed | 11:34 |
| ykarel | dtantsur, ack | 11:35 |
| chem | owalsh: during mitaka to newton upgrade | 11:35 |
| owalsh | chem: yea... so with the fail hard patch, puppet will never change it, but forces the operator to realise there is a problem and do *something* about it | 11:36 |
| *** salmankhan has quit IRC | 11:36 | |
| ykarel | dtantsur, for the time being you can try removing it: https://github.com/openstack/tripleo-quickstart-extras/blob/master/roles/ovb-manage-stack/tasks/ovb-create-stack.yml#L53 | 11:36 |
| dtantsur | ykarel: doing exactly it :) | 11:36 |
| ykarel | ack | 11:36 |
| *** bfournie has quit IRC | 11:36 | |
| *** suuuper has quit IRC | 11:36 | |
| chem | owalsh: yeah, for upgrade that will happen at the end of the converge step ... which is so so | 11:37 |
| chem | owalsh: but maybe https://review.openstack.org/#/c/555278 is completly solving the issue | 11:37 |
| dtantsur | ykarel: well, we do rm -rf on each call, so my changes get overridden.. | 11:37 |
| owalsh | chem: solves it if there isn't already an issue i.e fqdn_canonical == fqdn | 11:38 |
| chem | owalsh: which isn't the case in some rhel release | 11:39 |
| dtantsur | ykarel: aha, Cannot use v2 authentication with domain scope. I guess I have some env vars still defined in my shell, lemme start anew | 11:39 |
| chem | owalsh: so I guess we still have some stuff to do during upgrade to make it happens ... that's a nightmare | 11:39 |
| ykarel | dtantsur, yeah that's the reason | 11:39 |
| dtantsur | ykarel: but being incompatible with keystone v3 is a huge issue in year 2018 :) | 11:40 |
| chem | owalsh: oki, we (upgrade squad) have to find the root cause of that change in what is returned by puppet fqdn | 11:40 |
| Tengu | is there a "god mode" for tripleo migration? :D | 11:41 |
| ykarel | dtantsur, yes it should be fixed, please file a bug for it to be visible to the team | 11:41 |
| *** shreshtha has quit IRC | 11:42 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart master: Use tripleo-upgrade role for p->q job. https://review.openstack.org/540072 | 11:43 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-quickstart master: Use tripleo-upgrade role for p->q job. https://review.openstack.org/540072 | 11:43 |
| *** salmankhan has joined #tripleo | 11:45 | |
| owalsh | chem: NB facter does it's best to guess the fqdn, which could be significant - https://github.com/puppetlabs/facter/blob/32358935d38380b6b8e1a6e2ff29c29e37ead2f9/lib/src/facts/posix/networking_resolver.cc | 11:45 |
| chem | owalsh: hum ... not sure we use facter 3 in the tested setup ... have to look into when I get the an env running | 11:46 |
| chem | owalsh: thanks for the pointer | 11:46 |
| owalsh | chem: logic is similar in older versions IIRC /me was just too lazy to figure out the correct src | 11:47 |
| chem | owalsh: hehe, ack | 11:47 |
| owalsh | chem: just thinking re the fail hard patch. For upgrades we could set AllowIncorrectHostConfig: true, would just warn in that case... then on the next deploy it would fail hard | 11:48 |
| chem | owalsh: well ... so convergence would not fail, but all *:host configuration would have been changed and some stuff wouldn't work anymore (like all fip) ... \o/ | 11:49 |
| openstackgerrit | Jose Luis Franco proposed openstack-infra/tripleo-ci master: Run playbooks with custom args https://review.openstack.org/553474 | 11:50 |
| openstackgerrit | Jose Luis Franco proposed openstack-infra/tripleo-ci master: Use tripleo-upgrade role in undercloud upgrades job. https://review.openstack.org/548974 | 11:50 |
| *** arxcruz|rover is now known as arxcruz|rover|lu | 11:50 | |
| owalsh | chem: no, never changes. If current_host != correct_host we stick with current_host and warn/fail | 11:50 |
| chem | owalsh: oh .. I see | 11:51 |
| *** egallen has joined #tripleo | 11:51 | |
| chem | owalsh: oki, reviewing hard the fail hard review maybe that could solve it all | 11:53 |
| owalsh | chem: at least the operator knows something is wrong and we bail out instead of making things worse | 11:53 |
| *** pgadiya has quit IRC | 11:53 | |
| fultonj | Tengu: i am here now, but i need to jump on a call | 11:53 |
| fultonj | Tengu: not sure if you got your issue resolved (i only read some of the backlog) | 11:53 |
| owalsh | chem: not 100% sure what to do to resolve the issue after that though | 11:54 |
| *** raildo has joined #tripleo | 11:54 | |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Fix custom_nameserver must be a list https://review.openstack.org/555227 | 11:54 |
| fultonj | maybe we can chat in about ~1 hour | 11:54 |
| chem | owalsh: I'm thinking more along the line of "note change that parameter ever, even if it's in a bad state" and do some extra manual procedure to make the switch | 11:54 |
| chem | owalsh: s/note/not/ | 11:54 |
| Tengu | fultonj: hello! well, I'm trying to dig deeper in the ceph keyring stuff - I'm just launching a new test on a clean lab. | 11:54 |
| owalsh | chem: yea, it's the extra manual procedure that I'm not sure about yet | 11:55 |
| Tengu | fultonj: basically, I detected puppet-ceph pushed "mon" keys in a directory, and ceph-ansible pushes the "mon" keys in another one, and it creates a new mon key for that occasion. | 11:55 |
| chem | owalsh: I've seen ajo doing that for neutron through api and it's involving | 11:55 |
| Tengu | fultonj: ping me when you're available :). | 11:55 |
| fultonj | Tengu: ack | 11:55 |
| ajo | chem: doing what? :) | 11:55 |
| chem | ajo: hi :) we're discuting changing host value in neutron and other | 11:55 |
| Tengu | should have a result in about 1h30 with the current run I guess. | 11:55 |
| chem | ajo: with owalsh | 11:56 |
| ajo | ah, ok | 11:56 |
| *** jaosorior has quit IRC | 11:56 | |
| chem | ajo: owalsh has some good stuff there https://review.openstack.org/#/c/555732/1 and https://review.openstack.org/#/c/555732/1 | 11:56 |
| chem | oups | 11:56 |
| *** rfolco|off is now known as rfolco|ruck | 11:57 | |
| chem | ajo: the second one is https://review.openstack.org/#/c/555278/ (for this to never happen again) | 11:57 |
| *** dbecker has quit IRC | 11:57 | |
| ajo | chem: owalsh reading :) | 11:57 |
| chem | owalsh: man ...the fail hard review totally solve it | 11:58 |
| *** suuuper has joined #tripleo | 11:58 | |
| chem | owalsh++ | 11:58 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add access to /var/run/openvswitch from collectd https://review.openstack.org/554612 | 12:00 |
| ajo | owalsh: chem, looks like a good idea, we make the admin aware of it... | 12:00 |
| ajo | and responsible if they really want to change it, or ignore it | 12:00 |
| owalsh | chem, ajo: we also have ExtraHostFileEntries in queens, so when it fails hard, an operator could workaroud it temporarily be adding the necessary /etc/hosts entries to fix live-migration | 12:01 |
| owalsh | s/be/by/ | 12:01 |
| ajo | nice | 12:02 |
| owalsh | and then err.. call their friendly support guy, or come here and talk to us to figure out how to fix it for real | 12:02 |
| chem | owalsh: ajo we need to have the necessary steps described somewhere I think ... | 12:03 |
| *** jlabarre has joined #tripleo | 12:03 | |
| chem | owalsh: ajo because as I see it allow_incorrect_host_config will always be true during upgrade and update | 12:03 |
| ajo | chem: yes, we could config the automatic l3 agent failover | 12:04 |
| ajo | and then the only admin steps will be cleaning up the old agents | 12:04 |
| ajo | I guess | 12:04 |
| ajo | but we may need to try that | 12:04 |
| owalsh | chem: maybe we could catch the warning and make it very obvious | 12:05 |
| *** liverpooler has joined #tripleo | 12:07 | |
| *** gkadam_ has joined #tripleo | 12:07 | |
| owalsh | chem: do you know of any other issues with the fqdn changing? somebody mentioned galera not liking it IIRC | 12:09 |
| *** gkadam has quit IRC | 12:09 | |
| owalsh | bandini: ^^^ know anything about that? | 12:09 |
| *** bfournie has joined #tripleo | 12:10 | |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757174 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| openstack | Launchpad bug 1757174 in tripleo "tripleo-buildimage-overcloud-full-centos-7 failing with diskimage_builder.element_dependencies.MissingElementException: Element 'size=4096'' not found" [Critical,Triaged] | 12:10 |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 12:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 12:10 |
| *** abishop has joined #tripleo | 12:10 | |
| *** ccamacho is now known as ccamacho|lunch | 12:10 | |
| chem | owalsh: I worked on a case where it happened and we had to add entry in the hosts file in the end. | 12:11 |
| owalsh | chem: I'm probably confusing that with changing the CloudDomain param, I expect that would play havoc with HA, rabbitmq wouldn't like it either I expect | 12:12 |
| chem | owalsh: yeah that was a CloudDomain change | 12:13 |
| *** tosky has quit IRC | 12:15 | |
| *** tosky has joined #tripleo | 12:15 | |
| *** pdeore has quit IRC | 12:16 | |
| chem | ajo: about the agent failover it seems to be set to true by default ! https://github.com/openstack/tripleo-heat-templates/blob/stable/newton/puppet/services/neutron-api.yaml#L36-L39 | 12:16 |
| amoralej | i'me updated cryptography libraries in deps for queens and master | 12:16 |
| amoralej | pleas let me know if you note any abnormal behavior | 12:16 |
| bandini | owalsh: yeah the fact that we chose non-fqdn hostnames as defaults make it pretty much hardcoded now :( I.e. changing it in an existing deployment will break most HA services | 12:17 |
| *** adarazs_lunch is now known as adarazs | 12:17 | |
| chem | ajo: it's a missing backport \o/ | 12:17 |
| chem | ajo: that's consensual and simple enough to do then :) | 12:18 |
| chem | ajo: going to do it | 12:18 |
| *** suuuper has quit IRC | 12:19 | |
| owalsh | bandini: could be worse, at least you're not relying on the fqdn puppet fact that magically changed at some point and broke everything :-) | 12:19 |
| bandini | owalsh: holy guacamole. tell me more? when did it change/break? | 12:20 |
| bandini | we do have a bunch of hostname tests scattered in our manifests | 12:20 |
| owalsh | chem: ^^^ when did this happen? Quite a while ago IIRC, when the nova services suddenly changed from short hostnames to fqdn | 12:21 |
| *** zoli|lunch is now known as zoli | 12:22 | |
| owalsh | bandini: if it's short hostnames I don't think it's a problem, domain changing or not matching CloudDomain is a problem though | 12:24 |
| bandini | owalsh: yeah changing clouddomain or stuff just not matching is a big no no on a truckload of layers iirc | 12:25 |
| ajo | chem++ :D | 12:25 |
| *** jaganathan has quit IRC | 12:26 | |
| chem | owalsh: bandini don't know for sure, but it seems that a combinaison of facter and rhel version change what was returned by ::fqdn | 12:27 |
| chem | owalsh: bandini and maybe etc/hosts, so sometimes it's only the short name and some other time it's <short_name>.localdomain | 12:28 |
| *** sri_ has quit IRC | 12:28 | |
| owalsh | chem: and DHCP/DNS | 12:29 |
| chem | owalsh: oh yeah, that as well :) | 12:29 |
| chem | owalsh: bandini as ajo said, uuid would have been waaaayyy better | 12:30 |
| *** pcaruana has joined #tripleo | 12:30 | |
| *** rcernin has quit IRC | 12:30 | |
| *** egallen has quit IRC | 12:31 | |
| bandini | chem: yeah :/ that ship has ssailed a long time ago unfortunately | 12:31 |
| owalsh | chem: recall something similar with subscription manager and fqdns around that time too. hostname -f output changed too maybe | 12:31 |
| *** egallen has joined #tripleo | 12:31 | |
| owalsh | EmilienM,mwhahaha: know anything about the root cause of ^^^? | 12:31 |
| chem | bandini: well for next product we will know better :) | 12:32 |
| bandini | :) | 12:32 |
| chem | bandini: hostname changes all the time | 12:32 |
| bandini | chem: well I feel that one issue is that the hostname we set depends on way too many layers (dhcp, cloud-init, puppet, the weather, ...) | 12:33 |
| thrash | bandini: chicken sacrifices... | 12:34 |
| *** rlandy has joined #tripleo | 12:34 | |
| bandini | thrash: exactly :) | 12:34 |
| thrash | bandini: http://i1.wp.com/metsdaddy.com/wp-content/uploads/2016/03/IMG_3683.jpg | 12:34 |
| chem | ajo: oh boy ... the commit that changed the default value to true is not consensual ... https://github.com/openstack/tripleo-heat-templates/commit/08a201e23853455ffce6ff0d6c72163547ae7c06 | 12:35 |
| chem | ajo: discretly this change from false to true | 12:35 |
| chem | ajo: in that | 12:35 |
| owalsh | chem, bandini: do have an idea for the CloudDomain issue... the biggest problem AFAICT is the /etc/hosts has the wrong entires... | 12:35 |
| owalsh | so for example /etc/hosts has foo.localdomain when fqdn (form DHCP) is foo.example.com | 12:36 |
| chem | owalsh: yeah, the final workaround is to hardcode the missing entry in etc/hosts | 12:36 |
| owalsh | chem: so I'm thinking we add an AltCloudDomains param, list of other domains to include in /etc/hosts | 12:37 |
| chem | owalsh: bandini going make my daily dance around the fire where we have a goat ready to be eaten | 12:37 |
| *** egallen has quit IRC | 12:37 | |
| *** arxcruz|rover|lu is now known as arxcruz|rover | 12:37 | |
| *** chem is now known as chem|lunch | 12:37 | |
| * owalsh wonders if a huge /etc/hosts can cause issues | 12:37 | |
| *** lblanchard has joined #tripleo | 12:37 | |
| chem|lunch | owalsh: once I had too many charactere in one of the host line and mariadb couldn't cope | 12:38 |
| chem|lunch | owalsh: it was a long time ago | 12:38 |
| owalsh | chem|lunch: ack. I'll create an etherpad for all of this after lunch, try to keep track of all of the gotchas :-) | 12:40 |
| * owalsh biab | 12:40 | |
| *** pchavva has joined #tripleo | 12:42 | |
| *** lucasagomes is now known as lucas-hungry | 12:43 | |
| *** trown has joined #tripleo | 12:47 | |
| *** dparkes has quit IRC | 12:52 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-docs master: Refactor backup and restore docs https://review.openstack.org/544975 | 12:53 |
| openstackgerrit | Harald Jensås proposed openstack/instack-undercloud master: Use the new dnsmasq PXE filter in ironic-inspector https://review.openstack.org/523944 | 12:55 |
| *** dsariel has quit IRC | 12:57 | |
| openstackgerrit | Merged openstack/python-tripleoclient master: Fix the error message in _validate_no_ip_change https://review.openstack.org/553658 | 12:59 |
| *** dparkes has joined #tripleo | 12:59 | |
| *** ansmith has joined #tripleo | 13:03 | |
| *** morazi has joined #tripleo | 13:03 | |
| *** aputtur has joined #tripleo | 13:03 | |
| *** chem|lunch is now known as chem | 13:04 | |
| weshay | rfolco|ruck, arxcruz|rover so https://review.openstack.org/#/c/555056/ merged, /me wonders if the latest jobs are using a new image | 13:05 |
| *** Goneri has joined #tripleo | 13:06 | |
| rfolco|ruck | weshay, I am checking from https://images.rdoproject.org/master/rdo_trunk/tripleo-ci-testing/ | 13:07 |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757174 | 13:10 |
| openstack | Launchpad bug 1757174 in tripleo "tripleo-buildimage-overcloud-full-centos-7 failing with diskimage_builder.element_dependencies.MissingElementException: Element 'size=4096'' not found" [Critical,Triaged] | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 13:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 13:10 |
| *** suuuper has joined #tripleo | 13:10 | |
| weshay | rfolco|ruck, https://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-master-upload/5dce828/undercloud/home/jenkins/overcloud_image_build.log.txt.gz | 13:13 |
| *** jmelvin has joined #tripleo | 13:15 | |
| *** tzumainn has joined #tripleo | 13:15 | |
| rfolco|ruck | weshay, I guess this is a yes | 13:15 |
| *** psahoo has quit IRC | 13:16 | |
| *** tcw has joined #tripleo | 13:17 | |
| Tengu | duh… ceph-mon commited suicide -.- | 13:17 |
| Tengu | fultonj: ping? :) | 13:17 |
| rfolco|ruck | also, 1757174 should be fixed w/ https://review.openstack.org/#/c/554705/ | 13:17 |
| *** skramaja has quit IRC | 13:17 | |
| fultonj | hi Tengu | 13:18 |
| fultonj | it's a busy time | 13:18 |
| fultonj | Tengu: when last we spoke you had upgraded to Queens w/ puppet-ceph managing ceph | 13:18 |
| Tengu | fultonj: :) ok - just a confirmation: the hostname as configured in ceph is important for authentication, right? i.e. if puppet-ceph called a node "node01", and ansible call that same node "node01.domain", it will create some issues? | 13:19 |
| fultonj | you were going to move it to containers w/ ceph-ansible management using two playbooks | 13:19 |
| Tengu | fultonj: ah, no, I'm still on Pike, and trying to upgrade pike BM -> pike Containers, without updating the repositories. | 13:19 |
| fultonj | Tengu: i would get the host names consistent yes, i don't know exactly all the things that can go wrong | 13:19 |
| Tengu | fultonj: that was the "easiest" way to trigger the "bm to container" process in pike upgrade | 13:19 |
| *** dtantsur is now known as dtantsur|brb | 13:20 | |
| fultonj | Tengu: ok, so you have Pike/Jewel on baremetal | 13:20 |
| Tengu | fultonj: right | 13:20 |
| Tengu | fultonj: and trying to move to Pike in Containers | 13:20 |
| Tengu | with ceph-ansible instead of (deprecated) puppet-ceph | 13:20 |
| fultonj | Tengu: you have a dev system where you're testing what you'll to to your prod system and so far the results are not great | 13:20 |
| fultonj | true? ^ | 13:21 |
| Tengu | fultonj: right | 13:21 |
| Tengu | results are: ceph part is crashing, container can't join quorum | 13:21 |
| fultonj | are you on your undercloud, with an ansible inventory running a playbook directly? | 13:21 |
| fultonj | mon container can't join quorum? | 13:21 |
| fultonj | or are you doing stack updates with new tht to trigger ceph-ansible ? | 13:22 |
| Tengu | fultonj: I'm on the undercloud, running the "openstack overcloud deploy" command with the "major-upgrade-composable-steps" env added | 13:22 |
| fultonj | Tengu: ahhhh | 13:22 |
| Tengu | fultonj: specifically: mon.lab-controller-1.cloud.camptocamp.com@-1(probing) e2 not in monmap and have been in a quorum before; must have been removed (and it commits suicide right after) | 13:22 |
| openstackgerrit | zenghui.shi proposed openstack/puppet-tripleo master: Add fact to get array of nic alias name https://review.openstack.org/553119 | 13:23 |
| fultonj | Tengu: why are you doing major-upgrade-composable-steps? | 13:23 |
| fultonj | your overcloud is staying on pike for now? | 13:23 |
| Tengu | fultonj: indeed - but I have to trigger the "ocata -> pike" upgrade steps in order to ensure services are stopped/disabled and replaced by the containers. Or so did I understand | 13:23 |
| *** egallen has joined #tripleo | 13:24 | |
| fultonj | Tengu, i thought your overcloud was already in pike containers | 13:24 |
| fultonj | not for ceph, but for openstack containers | 13:24 |
| Tengu | fultonj: nope, it's not - that's the point of all the tests I'm doing :). | 13:25 |
| fultonj | the upgrade process should be | 13:25 |
| fultonj | 1. major upgrade composable steps + keep puppet-ceph | 13:25 |
| Tengu | fultonj: deployed pike on BM, because when we did it, there were many "warning" notes about "it should be OK with containers in pike, but" | 13:25 |
| fultonj | 2. upgrade nodes + keep puppet-ceph | 13:25 |
| fultonj | 3. converge add ceph-ansible | 13:25 |
| fultonj | i have never tested ceph upgrade in the middle of major upgrade composable steps | 13:26 |
| Tengu | hmm ok. so bad idea to do all in one run. | 13:26 |
| fultonj | possibly | 13:26 |
| fultonj | i don't know for a fact that it brfeaks anything | 13:26 |
| fultonj | Tengu: it does break HCI actually | 13:27 |
| fultonj | so you do it on step3 so HCI is ok | 13:27 |
| Tengu | good for me. | 13:27 |
| fultonj | else the docker service isn't yet running on other overcloud nodes | 13:27 |
| Tengu | I'm re-arming the lab and will do some changes in my env. | 13:27 |
| Tengu | right | 13:27 |
| Tengu | hmm wrong. docker is actually running | 13:27 |
| Tengu | but only one controller-node has the ceph container. | 13:28 |
| Tengu | and that one fails to join the cluster, firstly because of new keyring being generated, secondly probably due to name changes for identity (hostname -> fqdn) | 13:28 |
| openstackgerrit | Harald Jensås proposed openstack-infra/tripleo-ci master: Collect container service configuration /var/lib/config-data/ https://review.openstack.org/555771 | 13:29 |
| Tengu | fultonj: another question: is it a good idea to `cp -a' the content of /var/lib/ceph/mon/ceph-<hostname> to /var/lib/mon/ceph-<fqdn> priori doing the migration? That kind of should ensure mon keyring isn't re-created… ? | 13:30 |
| fultonj | Tengu: you shouldn't need to | 13:30 |
| fultonj | do the cp | 13:31 |
| fultonj | let ceph-ansible do it for you | 13:31 |
| fultonj | is it a good idea? no | 13:31 |
| Tengu | fultonj: ok. I'll do as you propose, and will come back if I have (new) issues :). | 13:31 |
| *** mcornea has joined #tripleo | 13:31 | |
| fultonj | i mean, we haev a process and if it fails, then that's a bug | 13:31 |
| fultonj | but better to follow the process | 13:31 |
| fultonj | Tengu: so... let me understand this... | 13:32 |
| fultonj | 3 mons | 13:32 |
| fultonj | 2 are as-is non-containers | 13:32 |
| fultonj | 1 went down (the first in the upgrade) | 13:32 |
| Tengu | fultonj: right | 13:32 |
| fultonj | it was containerized but it cannot join qurom w/ the cluster | 13:32 |
| *** lucas-hungry is now known as lucasagomes | 13:32 | |
| mcornea | shardy: gfidente hello o/ any thoughts on how I can move forward with https://review.openstack.org/#/c/555312/1/environments/major-upgrade-converge-docker.yaml ? I'm getting a RuntimeError: maximum recursion depth exceeded error when applying it | 13:32 |
| fultonj | Tengu: you could get on that thing and try to get it joined w/ manual steps to *know* why it couldn't join | 13:33 |
| Tengu | fultonj: exact. first of all due to wrong crypto - a new keyring was created by ceph-ansible, hosted in /var/lib/ceph/mon/ceph-<FQDN> instead of /var/lib/ceph/mon/ceph-<HOSTNAME> | 13:33 |
| fultonj | then destroy and redo and see, if the key and hostname matters, when handled correctly, fix it for you | 13:33 |
| gfidente | mcornea yeah looking into that in #heat | 13:33 |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Fix update of pacemaker container images during major upgrade https://review.openstack.org/547476 | 13:33 |
| fultonj | Tengu: let's remember what happens | 13:33 |
| fultonj | we could have a doc for people who might get in the same situation | 13:34 |
| Tengu | fultonj: seeing that, I tried to copy the existing keyring - that wasn't sufficient - I then tried to copy the whole directory; wasn't enough, probably due to hostname vs fqdn mismatch in ceph config/auth | 13:34 |
| fultonj | "So you didn't upgrade to containers when you moved to pike..." | 13:34 |
| fultonj | a pamphlet like you see in the doctors office | 13:34 |
| Tengu | fultonj: :] we installed Pike from scratch, no upgrade - as said: the many warning messages weren't really engaging for containers :). | 13:35 |
| Tengu | fultonj: more over, default upstream was still baremetal, not containers :D | 13:35 |
| Tengu | this changed only for Queens - but queens doesn't provide a way to migrate from BM to containers. | 13:35 |
| Tengu | so yep. that's a nasty situation. | 13:35 |
| fultonj | Tengu: don't get me wrong, i'm not like trying to say "you should have..." i'm more so trying to take this situation and document from it as i think others might hit it | 13:36 |
| *** panda is now known as panda|lunch | 13:36 | |
| *** thrash is now known as thrash|appt | 13:37 | |
| openstackgerrit | zenghui.shi proposed openstack/puppet-tripleo master: Add interface mapping support for sriov configuration https://review.openstack.org/499448 | 13:38 |
| Tengu | fultonj: don't worry. I also encountered a network failure at some point, but restarting the network service + pacemaker on the controllers helped going through. The "boss level" now is ceph :). | 13:38 |
| Tengu | fultonj: anyway. reverting the 'puppet-ceph to ansible-ceph' part in my git, and will give another try. | 13:38 |
| fultonj | Tengu: https://github.com/ceph/ceph-ansible/blob/master/infrastructure-playbooks/switch-from-non-containerized-to-containerized-ceph-daemons.yml | 13:39 |
| fultonj | ^ this should migrate each non-container ceph-daemon to a contianer | 13:39 |
| Tengu | fultonj: hmm, is it called in tripleo standard deploy? | 13:39 |
| fultonj | no | 13:39 |
| *** dansmith is now known as superdan | 13:39 | |
| fultonj | it's called on upgrade | 13:39 |
| *** dsariel has joined #tripleo | 13:40 | |
| fultonj | during step 3, converge, the included THT passes this playbook as an argument | 13:40 |
| Tengu | fultonj: hmm ok. so I should inject that in the 3rd point | 13:40 |
| fultonj | the default is a diff playbook | 13:40 |
| fultonj | Tengu: or... | 13:40 |
| fultonj | make an inventory and run it directly | 13:41 |
| fultonj | including.. converting your variables to the new format | 13:41 |
| *** yamahata has joined #tripleo | 13:41 | |
| *** egallen_ has joined #tripleo | 13:41 | |
| fultonj | tripleo does that for you when you move to pike | 13:41 |
| fultonj | the move from pike-bm to pike-containers with the same 1,2,3 process i listed above, _might_ do the same | 13:41 |
| Tengu | yep, already converted all the puppet-ceph vars to ceph-ansible. tripleo checks if we have something related to puppet-ceph and refuse to go forward. I also hit that issue :). | 13:42 |
| fultonj | that's a feature | 13:42 |
| Tengu | right. should have said feature, not issue. | 13:42 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-common master: Add healthcheck script for libvirtd https://review.openstack.org/550492 | 13:43 |
| gfidente | Tengu reality is it would be pretty bad to run ceph-ansible without having done that conversion first, destroy that and stuff | 13:43 |
| Tengu | although I didn't know how to drop it for good, it was still present in the exported env at that time. anyway. will do as you listed, I took nodes about that, and… crossing fingers :9. | 13:43 |
| *** egallen has quit IRC | 13:43 | |
| fultonj | Tengu: so when i practice ocata>pike upgrades i don't upgrade ceph until the converge step and keep my save puppet-ceph params during major upgrade composable | 13:43 |
| Tengu | of course. | 13:43 |
| gfidente | Tengu we were looking for ways to be safe than sorry | 13:43 |
| Tengu | always better like this ;) | 13:43 |
| chandankumar | weshay: EmilienM How stackrc is generated in tripleo deployment? os_auth_url on stackrc does not have api endpoint | 13:43 |
| gfidente | Tengu yeah at least it stopped without breaking things and forced you to take a look | 13:44 |
| gfidente | it's not great | 13:44 |
| chandankumar | it is blocking for running tempest on undercloud | 13:44 |
| chandankumar | mandre: Hello | 13:44 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-common master: Add healthcheck script for Horizon https://review.openstack.org/550118 | 13:44 |
| *** egallen_ is now known as egallen | 13:44 | |
| Tengu | gfidente: well, it's a pre-flight check mainly, so not a big issue - it's when it stop/fails later during the run, when the stack has been modified, it's a bigger issue. In my case, the stack becomes inconsistent and I can't do anything else than "re-arm" the lab :). | 13:45 |
| gfidente | exactly! | 13:45 |
| Tengu | it's like a really old SuperMario game when you can't save state during the game :D | 13:45 |
| gfidente | dont hate us then | 13:45 |
| chandankumar | mandre: EmilienM mwhahaha please have a look at this one http://logs.openstack.org/42/551442/4/check/tripleo-ci-centos-7-undercloud-containers/b78606f/logs/undercloud/home/zuul/tempest_output.log.txt.gz#_2018-03-23_08_20_26 | 13:46 |
| Tengu | gfidente: I of course don't hate you ;). ceph isn't an easy piece. | 13:46 |
| *** hjensas has quit IRC | 13:46 | |
| chandankumar | related to this review https://review.openstack.org/#/c/551442/ | 13:46 |
| EmilienM | chandankumar: it's generated in THT/extraconfig/post_deploy/undercloud_post.sh | 13:46 |
| mwhahaha | chandankumar: sounds like the command to launch the container is bad | 13:47 |
| openstackgerrit | Lukas Bezdicka proposed openstack/tripleo-heat-templates master: Upgrades: Refactor httpd_enabled variable https://review.openstack.org/555781 | 13:48 |
| *** guits_ has quit IRC | 13:49 | |
| chandankumar | EmilienM: thanks :-) | 13:52 |
| chandankumar | mwhahaha: please comment on the review what to fix? | 13:52 |
| mwhahaha | chandankumar: it's likely not in that review | 13:53 |
| chandankumar | mwhahaha: is there a way i can put all the stuff under docker run command in a sub script pass it with in the docker and when docker run called the script got called | 13:53 |
| chandankumar | mwhahaha: https://review.openstack.org/551441 | 13:54 |
| mwhahaha | chandankumar: it's likely in https://review.openstack.org/#/c/551441/7/roles/validate-tempest/templates/configure-tempest.sh.j2@80 | 13:54 |
| mwhahaha | but i am not sure exactly | 13:54 |
| openstackgerrit | Marios Andreou proposed openstack/python-tripleoclient stable/queens: Wire up a --skip-tags parameter for overcloud upgrade run cli https://review.openstack.org/553372 | 13:55 |
| *** dpawar has quit IRC | 13:55 | |
| *** d0ugal_ has joined #tripleo | 13:57 | |
| *** d0ugal has quit IRC | 13:57 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common stable/queens: Add openvswitch element back in https://review.openstack.org/555791 | 13:57 |
| *** links has quit IRC | 13:58 | |
| *** marios has quit IRC | 13:59 | |
| *** marios has joined #tripleo | 13:59 | |
| *** pdeore has joined #tripleo | 14:00 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates master: Honor {{role.name}}ExtraConfig with deprecated params https://review.openstack.org/555795 | 14:01 |
| *** anilvenkata has quit IRC | 14:02 | |
| *** kopecmartin has quit IRC | 14:04 | |
| mwhahaha | arxcruz|rover: rfolco|ruck: do we have a bug for gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-pike-branch because https://logs.rdoproject.org/89/554889/3/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-pike-branch/Zc6acfbc0c0ba443595b315d3416a0c9a/undercloud/home/jenkins/overcloud_prep_images.log.txt.gz#_2018-03-22_14_47_47 | 14:04 |
| rfolco|ruck | mwhahaha, checking | 14:05 |
| *** agopi has joined #tripleo | 14:05 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates stable/queens: Mount openvswitch dir rather than socket https://review.openstack.org/555801 | 14:06 |
| rfolco|ruck | mwhahaha, https://bugs.launchpad.net/tripleo/+bug/1758143 | 14:07 |
| openstack | Launchpad bug 1758143 in tripleo "ovb 3ctlr 1comp pike consistently failing to register node" [High,Triaged] | 14:07 |
| mwhahaha | k | 14:07 |
| *** gkadam has joined #tripleo | 14:07 | |
| *** gkadam has quit IRC | 14:08 | |
| *** ratailor has quit IRC | 14:08 | |
| mwhahaha | oh noes ssh timedout has bled into OVB | 14:09 |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757174 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1757174 in tripleo "tripleo-buildimage-overcloud-full-centos-7 failing with diskimage_builder.element_dependencies.MissingElementException: Element 'size=4096'' not found" [Critical,Triaged] | 14:10 |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 14:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 14:10 |
| openstackgerrit | Lukas Bezdicka proposed openstack/tripleo-heat-templates master: Upgrades: Refactor httpd_enabled variable https://review.openstack.org/555781 | 14:11 |
| arxcruz|rover | mwhahaha: can i fix release bug https://bugs.launchpad.net/tripleo/+bug/1758180 since https://review.openstack.org/#/c/555468/ was already merged ? | 14:11 |
| openstack | Launchpad bug 1758180 in tripleo "reproducer for 3node incorrectly overrides the role configuration" [Critical,Triaged] | 14:11 |
| mwhahaha | arxcruz|rover: yes | 14:12 |
| *** jpena is now known as jpena|lunch | 14:12 | |
| *** yamahata has quit IRC | 14:13 | |
| *** amoralej is now known as amoralej|lunch | 14:14 | |
| arxcruz|rover | mwhahaha: https://bugs.launchpad.net/tripleo/+bug/1756936 can also be closed right? since it was just a rebase ? | 14:15 |
| openstack | Launchpad bug 1756936 in tripleo "rdo jobs ignore depends-on tripleo-common" [Low,Triaged] | 14:15 |
| openstackgerrit | Marius Cornea proposed openstack/tripleo-upgrade master: Include connectivity check prepare scripts during FFU https://review.openstack.org/554914 | 14:15 |
| mwhahaha | arxcruz|rover: yea i think so | 14:15 |
| *** yamahata has joined #tripleo | 14:17 | |
| bogdando | EmilienM: hi. Today is a great day :D | 14:17 |
| bogdando | https://trello.com/c/xkyfUGST/34-identify-all-blockers-for-ovb-fs001#comment-5ab50ba184cf81857d0a7d1f | 14:17 |
| arxcruz|rover | mwhahaha: weshay can you guys take a look at https://review.openstack.org/555703 | 14:17 |
| bogdando | I created a specia featureset that mimics ovb, but w/o the host openstack cloud, and now we have the same blocker reproduced | 14:18 |
| bogdando | \o/ | 14:18 |
| *** panda|lunch is now known as panda | 14:18 | |
| mwhahaha | nice | 14:18 |
| bogdando | https://github.com/bogdando/oooq-warp/blob/master/tht/config/general_config/featureset062.yml | 14:18 |
| EmilienM | bogdando: wow nice | 14:20 |
| bogdando | mtu is 1500 on my host and on undercloud VM, so seems like now we can check if a subntet mtu update would help | 14:20 |
| EmilienM | bogdando: yeah you need to apply my patch on openstackclient | 14:20 |
| EmilienM | which isn't merged I think :/ | 14:20 |
| bogdando | yeah, that's the plan | 14:21 |
| EmilienM | oh it's in gate actually | 14:21 |
| bogdando | EmilienM: can't say what is still wrong to https://logs.rdoproject.org/56/542556/76/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/Z5d59a09c12e74d4993859ed4edcd3233/undercloud/home/jenkins/overcloud_prep_network.log.txt.gz tho | 14:22 |
| EmilienM | bogdando: I'm looking now | 14:22 |
| bogdando | I used absolutely the same iptables on my UC, but applied manually post deploy | 14:22 |
| bogdando | so quickstart drops them on the floor somewhere | 14:23 |
| bogdando | imo | 14:23 |
| EmilienM | yeah it's a bit weird | 14:24 |
| *** pkovar has quit IRC | 14:25 | |
| EmilienM | bogdando: maybe we can just ignore this chunk of code in overcloud_prep_network | 14:26 |
| bogdando | let's try | 14:26 |
| EmilienM | bogdando: since we have the workaround already, that manage the masquerading | 14:26 |
| EmilienM | bogdando: I'll try now, let me push a pathc | 14:26 |
| bogdando | k | 14:26 |
| *** pkovar has joined #tripleo | 14:27 | |
| *** holser__ has quit IRC | 14:30 | |
| EmilienM | bogdando: i'm trying to see what is purging the nat table first | 14:31 |
| bogdando | and I'm trying to figure out how to update ctlplane with new mtu | 14:32 |
| bogdando | w/o recreating it | 14:32 |
| EmilienM | bogdando: I wonder if it's puppet | 14:32 |
| bogdando | highly likely! | 14:32 |
| bogdando | oh, it's network set that I want | 14:33 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-upgrade master: Correct var name tripleo_upgrade_validations_non_fatal. https://review.openstack.org/555813 | 14:33 |
| bogdando | hm, only running "openstack network set --mtu=1350 ctlplane didn't help" | 14:34 |
| bogdando | anythin else to do to the ports/vms prolly? | 14:35 |
| EmilienM | bogdando: not sure you can do afterward | 14:35 |
| *** sai-mtg is now known as sai-intrw | 14:36 | |
| bogdando | ping -M dont -s 1500 | 14:36 |
| bogdando | passes, so I prolly need to reboot VMs | 14:37 |
| openstackgerrit | Rafael Folco proposed openstack/diskimage-builder master: Revert "Remove tripleo jobs" https://review.openstack.org/555814 | 14:37 |
| *** dtantsur|brb is now known as dtantsur | 14:37 | |
| bogdando | nvm, it should be do, not dont | 14:37 |
| openstackgerrit | Rafael Folco proposed openstack/diskimage-builder master: Revert "Remove tripleo jobs" https://review.openstack.org/555814 | 14:38 |
| *** pdeore has quit IRC | 14:38 | |
| bogdando | so EmilienM, mwhahaha that I have is: ping -M do -s 1472 still works over network mtu updated to 1350 and -s 1473 Message too long | 14:39 |
| *** florianf has quit IRC | 14:39 | |
| bogdando | it seems not yet fully applied | 14:39 |
| openstackgerrit | Carlos Goncalves proposed openstack/tripleo-heat-templates master: Containerize Neutron LBaaS service plugin https://review.openstack.org/555011 | 14:40 |
| *** dciabrin_ has quit IRC | 14:40 | |
| *** dciabrin has joined #tripleo | 14:40 | |
| bogdando | yeah, same results if reset to 1000 mtu | 14:40 |
| openstackgerrit | Carlos Camacho proposed openstack/puppet-tripleo master: Add missing cron jobs for Overcloud cleanup https://review.openstack.org/555362 | 14:42 |
| *** psachin has quit IRC | 14:42 | |
| bogdando | mwhahaha, EmilienM: after changing mtu on overcloud VMs' eth0 and UC's br-ctlplane, ping results changed to a sorter mtu, but ssh still fails. Not mtu problem it seems! | 14:46 |
| bogdando | but was a nice try | 14:46 |
| *** egallen has quit IRC | 14:46 | |
| mwhahaha | k | 14:47 |
| mwhahaha | bogdando: what does ssh -v get you? | 14:47 |
| mwhahaha | anything useful? | 14:47 |
| *** pickle has joined #tripleo | 14:47 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates master: Add purge to Nova cleanup cron. https://review.openstack.org/551966 | 14:49 |
| bogdando | mwhahaha: I think, for my env the issue is the mismatching /home/heat-admin/.ssg/authorized_keys and undercloud's id_rsa.pub | 14:50 |
| bogdando | shall they match, right? how come they don't ... sigh | 14:50 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: undercloud: workaround for masquerading network in CI/OVB https://review.openstack.org/553620 | 14:51 |
| EmilienM | bogdando: trying this ^ | 14:51 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: undercloud: workaround for masquerading network in CI/OVB https://review.openstack.org/553620 | 14:51 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: roles: rename overcloud-prep-containers to prep-containers https://review.openstack.org/543014 | 14:52 |
| bogdando | EmilienM: good idea! | 14:52 |
| EmilienM | bogdando: https://logs.rdoproject.org/56/542556/76/openstack-check/gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/Z5d59a09c12e74d4993859ed4edcd3233/undercloud/var/log/containers/neutron/server.log.txt.gz#_2018-03-23_09_31_15_304 | 14:52 |
| EmilienM | look ^ you can see the MTU set in the neutron network | 14:52 |
| EmilienM | neutron.api.v2.base [req-8e20ac09-4193-44af-b773-87908f0f6d20 78088386b9624770bf6dc9010dc06359 72e0869aae164e5eb8044eee53f2d542 - default default] Request body: {u'network': {u'admin_state_up': True, u'provider:network_type': u'flat', u'name': u'ctlplane', u'provider:physical_network': u'ctlplane', u'mtu': u'1350'}} | 14:53 |
| EmilienM | bogdando: so my patch works for the mtu, on the undercloud | 14:53 |
| -openstackstatus- NOTICE: zuul.o.o has been restarted to pick up latest code base and clear memory usage. Both check / gate queues were saved, be sure to check your patches and recheck when needed. | 14:53 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: prep-containers: include containerized undercloud bits https://review.openstack.org/543024 | 14:53 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: undercloud: add missing TLS environments when preparing containers https://review.openstack.org/545444 | 14:53 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: Remove adjust-interface-mtus script https://review.openstack.org/546216 | 14:53 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart-extras master: DO NOT REVIEW - Workarounds for containerized undercloud https://review.openstack.org/545628 | 14:53 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: Run ovb-ha (fs001) with a containerized undercloud https://review.openstack.org/542556 | 14:54 |
| *** florianf has joined #tripleo | 14:54 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient master: Don't delete the tripleo Zaqar queue https://review.openstack.org/555193 | 14:56 |
| *** colonwq has quit IRC | 14:57 | |
| *** vpickard has quit IRC | 14:57 | |
| *** ykarel has quit IRC | 14:57 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: DNM - test containerized undercloud upgrades https://review.openstack.org/553633 | 14:58 |
| *** jpena|lunch is now known as jpena | 14:58 | |
| *** lblanchard has quit IRC | 14:59 | |
| *** ramishra has quit IRC | 15:00 | |
| bogdando | EmilienM: that's really cool so we can that simple inject openstack patches into envs under test | 15:00 |
| EmilienM | yes it is | 15:01 |
| *** d0ugal_ has quit IRC | 15:01 | |
| bogdando | mwhahaha, EmilienM: so at least for my env, the problem is mismatching keys | 15:01 |
| bogdando | interesting why the keys diverged (ironic injects them, right?) for classic underloud vs containerized | 15:01 |
| *** d0ugal__ has joined #tripleo | 15:02 | |
| mwhahaha | yea it should get it from the default nova key | 15:02 |
| *** mdnadeem has quit IRC | 15:02 | |
| *** d0ugal__ has quit IRC | 15:02 | |
| *** d0ugal has joined #tripleo | 15:02 | |
| *** vpickard has joined #tripleo | 15:03 | |
| *** ihrachys has joined #tripleo | 15:03 | |
| *** vpickard has quit IRC | 15:03 | |
| *** pradk has joined #tripleo | 15:05 | |
| bogdando | mwhahaha, EmilienM: we're missing something like openstack keypair show --public-key default for containerized UC it seems | 15:06 |
| mwhahaha | are we missing the keypair create? | 15:06 |
| bogdando | there is the key | 15:06 |
| bogdando | and it is matching | 15:06 |
| EmilienM | mwhahaha: the create is in extraconfig/post_deploy/undercloud_post.sh | 15:07 |
| bogdando | tho, undercloud does not attempt to use it when executing create admin ssh scipt | 15:07 |
| bogdando | something we missed from instack prolly | 15:07 |
| * mwhahaha doesn't like undercloud_post.sh | 15:07 | |
| bogdando | /usr/share/openstack-tripleo-heat-templates/deployed-server/scripts/enable-ssh-admin.sh this one | 15:08 |
| EmilienM | the only thing that instack does it to create the 'default' keypair, which is what is doing extraconfig/post_deploy/undercloud_post.sh | 15:08 |
| *** colonwq has joined #tripleo | 15:09 | |
| *** amoralej|lunch is now known as amoralej | 15:09 | |
| *** hjensas has joined #tripleo | 15:09 | |
| bogdando | it uses bad OVERCLOUD_SSH_KEY | 15:09 |
| bogdando | falls back to .ssh/id_rsa | 15:10 |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757174 | 15:10 |
| openstack | Launchpad bug 1757174 in tripleo "tripleo-buildimage-overcloud-full-centos-7 failing with diskimage_builder.element_dependencies.MissingElementException: Element 'size=4096'' not found" [Critical,Triaged] | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| bogdando | and that's an undercloud key | 15:10 |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 15:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 15:10 |
| bogdando | not the nova generated one | 15:10 |
| mwhahaha | nova doesn't generate one | 15:10 |
| mwhahaha | we upload the undercloud done | 15:10 |
| mwhahaha | s/done/one | 15:10 |
| bogdando | hm | 15:10 |
| bogdando | so extraconfig/post_deploy/undercloud_post.sh does not create but uploads? | 15:10 |
| mwhahaha | keypair create takes the a key file | 15:10 |
| mwhahaha | yea | 15:11 |
| mwhahaha | it uses the undercloud user's key | 15:11 |
| bogdando | ok, but for my env id_rsa was definitely not uploaded | 15:11 |
| *** rbrady is now known as rbrady-afk | 15:12 | |
| *** aufi has quit IRC | 15:12 | |
| mwhahaha | might be the problem then | 15:12 |
| bogdando | yupp | 15:12 |
| mwhahaha | let me look at that script | 15:13 |
| mwhahaha | https://github.com/openstack/tripleo-heat-templates/blob/master/extraconfig/post_deploy/undercloud_post.sh#L146-L156 | 15:13 |
| mwhahaha | so | 15:13 |
| mwhahaha | what's the output of that for your deploy | 15:13 |
| mwhahaha | did it reuse one or did it like use the root key | 15:13 |
| bogdando | I found mismatch in the deploy log | 15:14 |
| bogdando | let me share it | 15:14 |
| mwhahaha | i like that we generate the keys twice | 15:15 |
| mwhahaha | https://github.com/openstack/tripleo-heat-templates/blob/master/extraconfig/post_deploy/undercloud_post.sh#L51-L65 | 15:15 |
| * mwhahaha refers back to his comment about no likeing undercloud_post.sh | 15:15 | |
| bogdando | mwhahaha, EmilienM: https://pastebin.com/4VzC972D race? | 15:19 |
| bogdando | the code created the key in both places | 15:19 |
| bogdando | within a second | 15:19 |
| hjensas | mwhahaha: is L155 there correct? it passes the key as stdin. Should it not be --public-key ? | 15:19 |
| *** chlong has quit IRC | 15:19 | |
| jfrancoa | EmilienM: I just saw you rechecked undercloud upgrades job, it's failing due to this: https://review.openstack.org/#/c/555813/ it's an easy review, if you have some time. | 15:20 |
| jfrancoa | EmilienM: otherwise I can add a Depends-On | 15:20 |
| mwhahaha | hjensas: yea possibly, not sure why we're using stdin | 15:20 |
| mwhahaha | bogdando: hjensas is right we probably should change L155 to be openstack keypair create 'default' --public-key $HOMEDIR/.ssh/id_rsa.pub | 15:21 |
| *** holser__ has joined #tripleo | 15:21 | |
| *** itlinux has joined #tripleo | 15:21 | |
| mwhahaha | it might not actually be a problem but it's a better thing | 15:21 |
| bogdando | \o/ | 15:21 |
| EmilienM | jfrancoa: ok, I'll review | 15:21 |
| jfrancoa | EmilienM: thanks a lot | 15:21 |
| mwhahaha | bogdando: the help says if it's not used it creates a private key | 15:22 |
| mwhahaha | so maybe that was it | 15:22 |
| * mwhahaha shrugs | 15:22 | |
| mwhahaha | *magic* | 15:22 |
| bogdando | indeed | 15:22 |
| * mwhahaha goes back to digging in sosreport logs | 15:22 | |
| *** thrash|appt is now known as thrash | 15:23 | |
| *** trown has quit IRC | 15:23 | |
| bogdando | ok, I'll play with that post script | 15:23 |
| *** trown has joined #tripleo | 15:24 | |
| bogdando | one more typo found http://git.openstack.org/cgit/openstack/tripleo-heat-templates/tree/extraconfig/post_deploy/undercloud_post.sh#n191 :) | 15:25 |
| bogdando | it should be casted via hiera | 15:25 |
| *** jtomasek has quit IRC | 15:25 | |
| *** melwitt is now known as jgwentworth | 15:26 | |
| hjensas | mwhahaha: yeah, looks like stdin is not actually working. http://paste.openstack.org/show/710165/ | 15:28 |
| mwhahaha | yea | 15:29 |
| mwhahaha | bogdando: try fixing the keypair commands | 15:29 |
| hjensas | mwhahaha: passing in undercloud.conf works just as good as passing in a key, and it outputs the private key for what it generated. | 15:29 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Fix ssh keys validation workflow trigger UC post https://review.openstack.org/555831 | 15:31 |
| bogdando | hjensas: good idea | 15:33 |
| openstackgerrit | Miguel Angel Ajo proposed openstack/tripleo-quickstart master: add default overcloud-prep tags https://review.openstack.org/553895 | 15:33 |
| bogdando | hjensas: openstack keypair create test < undercloud.conf | 15:34 |
| bogdando | works ? :-O | 15:34 |
| hjensas | bogdando: it doesnt use anything from stdin. | 15:35 |
| bogdando | ah | 15:35 |
| bogdando | :D | 15:35 |
| hjensas | bogdando: it would be equivalent to run "openstack keypair create test" | 15:35 |
| bogdando | got it | 15:35 |
| *** shardy has quit IRC | 15:35 | |
| *** links has joined #tripleo | 15:40 | |
| *** yprokule has quit IRC | 15:41 | |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-docs master: WIP: Add FFU docs https://review.openstack.org/549892 | 15:45 |
| *** links has quit IRC | 15:46 | |
| *** paramite_ has quit IRC | 15:47 | |
| -openstackstatus- NOTICE: Gerrit will be temporarily unreachable as we restart it to complete the rename of some projects. | 15:51 | |
| *** dparkes has quit IRC | 15:51 | |
| *** suuuper has quit IRC | 15:54 | |
| *** suuuper has joined #tripleo | 15:54 | |
| bogdando | hjensas, mwhahaha, EmilienM: I will submit the fix after gerrit maintenance | 15:55 |
| *** suuuper has quit IRC | 15:56 | |
| EmilienM | bogdando: good | 16:00 |
| *** gyankum has quit IRC | 16:00 | |
| *** lblanchard has joined #tripleo | 16:01 | |
| *** EmilienM changes topic to "Welcome to Rocky. CI status - Promotions: RED; check/gate: GREEN; RDO CI jobs: Questionable | http://tripleo.org/ | https://docs.openstack.org/tripleo-docs/latest/" | 16:01 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-heat-templates master: Fix undercloud key upload to nova keypairs UC post https://review.openstack.org/555841 | 16:01 |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 16:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 16:10 |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-docs master: Update information on configuring networks for ironic in the overcloud https://review.openstack.org/555347 | 16:16 |
| chem | yolanda: I'm just wondering now if ovs hack shouldn't go in the update_tasks as well .. | 16:16 |
| *** agopi is now known as agopi|lunch | 16:17 | |
| dtrainor | It's been a while, I forgot - should ControlVirtualIP come from ExternalAllocationPools? | 16:17 |
| *** fzdarsky__ is now known as fzdarsky|afk | 16:18 | |
| openstackgerrit | Damien Ciabrini proposed openstack/tripleo-heat-templates master: Upgrade data on disk on mariadb major upgrade https://review.openstack.org/546666 | 16:20 |
| *** wojdec has joined #tripleo | 16:20 | |
| mandre | d0ugal: hey Dougal, I'm exploring a little bit the mistral workflows we have and noticed we're still polling in the tripleo.stack.v1 workflows... I'm wondering if it wouldn't be possible to have mistral react to events sent on a zaqar queue instead? | 16:21 |
| bnemec | dtrainor: No, those are two separate networks. | 16:21 |
| mandre | d0ugal: apparently heat is capable of sending events to zaqar but I'm not aware of us using it in tripleo | 16:22 |
| mandre | https://docs.openstack.org/heat/latest/template_guide/environment.html#retrieving-events | 16:22 |
| wojdec | Hi! I'm converting a few heat service templates that have been working ok with puppet3 modules to run with Puppet 4, and am wondering if there is a systemic way to allow "default" values (eg params.pp) of the puppet modules to be used? The previously working way of passing an empty string doesn't work, since Puppet 4 interprets this now as the actual setting. So am getting lots of "" settings instead of the defaults | 16:23 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart-extras master: Allow custom overcloud ssh user for deployments https://review.openstack.org/555846 | 16:25 |
| yolanda | chem yes, you are right. This will come now as part of minor update, along with 7.5 kernel | 16:25 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart master: Pass in overcloud-ssh-key via its dedicated args https://review.openstack.org/555847 | 16:25 |
| owalsh | EmilienM: ok to recheck now the gate is green or are we waiting for the ssh fixes to land? | 16:25 |
| *** etingof is now known as etingof|afk | 16:26 | |
| EmilienM | owalsh: I'm not sure what we are waiting, but to me we can recheck | 16:26 |
| * owalsh hits go | 16:27 | |
| *** ykarel has joined #tripleo | 16:27 | |
| *** radeks__ has quit IRC | 16:27 | |
| yolanda | chem, so we should apply both in update and upgrade (for people that forget about updates) ? | 16:27 |
| *** radeks__ has joined #tripleo | 16:27 | |
| chem | yolanda: yeah I guess both won't hurt | 16:29 |
| yolanda | ok, and i added another comment | 16:29 |
| chem | yolanda: I've added one as well | 16:30 |
| openstackgerrit | Damien Ciabrini proposed openstack/tripleo-heat-templates master: Make HA containers log to /var/log/containers after upgrade https://review.openstack.org/553424 | 16:30 |
| openstackgerrit | Bogdan Dobrelya proposed openstack/tripleo-quickstart master: Run ovb-ha (fs001) with a containerized undercloud https://review.openstack.org/542556 | 16:32 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo stable/newton: Release 5.6.9 (newton) https://review.openstack.org/555848 | 16:32 |
| d0ugal | mandre: I believe it is possible, just that nobody has tried wiring it all together | 16:33 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo stable/ocata: Release 6.5.11 (ocata) https://review.openstack.org/555849 | 16:33 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo stable/pike: Release 7.4.11 (pike) https://review.openstack.org/555850 | 16:34 |
| mandre | therve: hi! do you know if this spec to notify mistral from zaqar is implemented https://review.openstack.org/#/c/318202? | 16:34 |
| d0ugal | mandre: https://www.youtube.com/watch?v=b36XbF0Q_vU | 16:34 |
| d0ugal | mandre: I believe they may be demoing it in that ^ | 16:35 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo stable/queens: Release 8.3.1 (queens) https://review.openstack.org/555852 | 16:35 |
| mandre | d0ugal: ack, sounds like a fun little project | 16:35 |
| d0ugal | mandre: yup, something I have wanted to play with but never had time :) | 16:36 |
| therve | mandre, Yes | 16:36 |
| dtrainor | thanks, bnemec. what network does ControlVirtualIP come from? | 16:37 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-ui stable/ocata: Release 3.2.3 (ocata) https://review.openstack.org/555854 | 16:37 |
| bnemec | dtrainor: Control Plane. I don't think that's set by a template param though, it's just whatever the ctlplane subnet has configured. | 16:37 |
| *** pradk has quit IRC | 16:37 | |
| dtrainor | got it. | 16:38 |
| mandre | therve: excellent, do you have some doc somewhere how to use it? | 16:38 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-ui stable/pike: Release 7.4.9 (pike) https://review.openstack.org/555856 | 16:38 |
| dtrainor | bnemec, even in the context of an overcloud with ssl? | 16:39 |
| therve | Ah ah docs | 16:39 |
| d0ugal | lol | 16:39 |
| mandre | :) | 16:39 |
| therve | mandre, Do you use heat templates? | 16:39 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-ui stable/queens: Release 8.3.1 (queens) https://review.openstack.org/555858 | 16:40 |
| therve | OS::Zaqar::MistralTrigger encapsulates most of the work | 16:40 |
| bnemec | dtrainor: Yes, although note that if you're using ssl and net-iso, then you don't need to worry about the control plane vip. With net-iso it's the public vip. | 16:40 |
| dtrainor | bnemec, i thought i needed to know what it was, so that I can use the IP as the CN for generating the certificate | 16:41 |
| mandre | therve: yeah, using t-h-t... basically I want to get rid of the polling in a mistral workflow that blocks waiting for stack events | 16:41 |
| bnemec | dtrainor: Okay, let's step back. Are you using net-iso? | 16:41 |
| dtrainor | sorry - yes i am, net-iso with multiple nics | 16:41 |
| dtrainor | my intent was to also include environments/ssl/enable-tls.yaml | 16:42 |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Improve nova-ironic cellv2 discovery https://review.openstack.org/553376 | 16:42 |
| bnemec | dtrainor: Then the control plane vip doesn't matter for the cert. | 16:42 |
| bnemec | dtrainor: See https://docs.openstack.org/tripleo-docs/latest/install/advanced_deployment/ssl.html#certificate-and-public-vip-configuration | 16:42 |
| therve | mandre, https://github.com/openstack/heat-templates/blob/master/hot/autohealing/autohealing_server.yaml#L126 | 16:43 |
| bnemec | Hmm, did we rename these? | 16:43 |
| dtrainor | ok, i see. and in this instance, "public" should definitely not be confused with "external", because they're different | 16:43 |
| dtrainor | (further context: working with qe downstream tests that list some instructions for how to implement ssl, the prescribed docs may be dated) | 16:44 |
| bnemec | dtrainor: No, those are the same. | 16:44 |
| mandre | therve: awesome! that should get me started | 16:44 |
| bnemec | The public VIP is the one on the external network. | 16:44 |
| mandre | thanks | 16:44 |
| dtrainor | The instructions create a new cert and key which get stuffed in to environments/ssl/enable-tls.yaml, but the cert also uses a CN that references an IP that I'm trying to make sure is the correct one | 16:45 |
| bnemec | dtrainor: The cert should point at whatever address was specified in PublicVirtualFixedIPs. | 16:46 |
| weshay | rfolco|ruck, arxcruz|rover fyi.. I don't see openvswitch in the image elements https://logs.rdoproject.org/openstack-periodic/periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-master-upload/5dce828/undercloud/home/jenkins/overcloud_image_build.log.txt.gz#_2018-03-23_06_02_32 | 16:46 |
| weshay | rfolco|ruck, arxcruz|rover so the tempest failures in fs20 are expected | 16:47 |
| * weshay checking the very latest | 16:47 | |
| dtrainor | i see. | 16:47 |
| d0ugal | mandre: the tricky part (maybe) is that we don't want to trigger a new workflow on an even | 16:47 |
| d0ugal | event* | 16:48 |
| d0ugal | mandre: but rather we want to pause the tripleo.v1.stacks workflows and resume them on the expected event | 16:48 |
| *** zaneb has quit IRC | 16:48 | |
| d0ugal | I also wonder if we want to keep polling (but less regular?) as a backup | 16:48 |
| mandre | d0ugal: exactly we want to keep the original workflow blocking and unpause them | 16:49 |
| d0ugal | Yeah | 16:49 |
| chem | panda: oki for ffu, I don't know what is the next issue because I'm now hitting the 3h upstream timeout (because everything is going fined during 3h) and don't have a job in rdo where I could set timeout higher | 16:49 |
| d0ugal | mandre: have you seen the std.mistral_http action? It has a silly name, but otherwise has a mechanism that might be useful | 16:49 |
| rfolco|ruck | weshay, ack | 16:50 |
| *** ykarel has quit IRC | 16:50 | |
| mandre | d0ugal: anyway one safe change we could do could be to poll on the zaqar queue rather than heat stack status, I assume it takes less resources | 16:50 |
| *** jfrancoa has quit IRC | 16:50 | |
| d0ugal | mandre: yup, makes sense | 16:50 |
| d0ugal | mandre: https://docs.openstack.org/mistral/newton/dsl/dsl_v2.html#std-mistral-http | 16:51 |
| mandre | d0ugal: I didn't know about std.mistral_http, will take a look thanks | 16:51 |
| d0ugal | mandre: that action pauses (I think, I forget the state name) a workflow until it recieves a request back. | 16:51 |
| dtrainor | thanks bnemec++ | 16:53 |
| d0ugal | mandre: https://github.com/openstack/mistral/blob/master/mistral/actions/std_actions.py#L267-L268 | 16:53 |
| d0ugal | mandre: that is the key bit, it tells mistral that when the action finishes, the task isn't finished | 16:53 |
| *** marrusl has joined #tripleo | 16:54 | |
| *** gkadam_ has quit IRC | 16:54 | |
| *** bnemec is now known as beekneemech | 16:54 | |
| mandre | d0ugal: hmm, I can't find how to unblock the action afterwards... the doc just say that we need to notify mistral back | 16:56 |
| *** lucasagomes is now known as lucas-afk | 16:57 | |
| d0ugal | mandre: yeah, that bit is missing, let me check... | 16:57 |
| d0ugal | mandre: I think something like this... | 16:58 |
| *** zoli is now known as zoli|gone | 16:58 | |
| *** zoli|gone is now known as zoli | 16:58 | |
| d0ugal | mistral action-exectuion-update --state=RUNNING ACTION_ID | 16:58 |
| d0ugal | but I am going to write a quick test | 16:59 |
| mandre | d0ugal: is it possible to unpause an action execution from another mistral workflow? | 17:00 |
| d0ugal | mandre: I assume so, but I have never tried. | 17:01 |
| d0ugal | mandre: Mistral has mistral actions, like it does for other projects. There is no special internal access (there maybe should be in some cases) | 17:01 |
| mandre | so the flow would be: call the wait_for_stack_complete_or_failed workflow, which would be paused by default, then from zaqar call another workflow that unpauses it | 17:02 |
| d0ugal | lol | 17:02 |
| d0ugal | mandre: yeah, I guess that could work. Seems a little crazy | 17:02 |
| *** athomas_ has quit IRC | 17:03 | |
| bogdando | EmilienM: let's merge https://review.openstack.org/#/c/553026/5 ? | 17:04 |
| mwhahaha | we'll need a promotion and stuff | 17:04 |
| mandre | d0ugal: I don't think it's possible to change the running status directly from zaqar as we don't know the execution id | 17:04 |
| EmilienM | bogdando: no | 17:04 |
| EmilienM | we need 2 things: | 17:04 |
| EmilienM | 1) a bump in openstack/requirements | 17:04 |
| EmilienM | 2) a bump in RDO | 17:04 |
| EmilienM | 3) a promotion | 17:04 |
| bogdando | oh | 17:04 |
| EmilienM | that's 3 things | 17:04 |
| bogdando | EmilienM: that one then https://review.openstack.org/#/c/554674 :) | 17:06 |
| bogdando | and prolly https://review.openstack.org/#/c/554678 | 17:06 |
| d0ugal | mandre: yeah, that is a bit tricky. | 17:06 |
| d0ugal | mandre: how would the un-pausing workflow know the ID it needs to unpause? | 17:07 |
| chem | panda: when you have time could you have another look at https://review.openstack.org/#/c/540473/ ? | 17:07 |
| d0ugal | mandre: http://paste.openstack.org/show/710291/ | 17:07 |
| d0ugal | mandre: so that is a full example | 17:07 |
| panda | chem: ok | 17:07 |
| chem | panda: thanks | 17:07 |
| d0ugal | mandre: so the action is left in the RUNNING state and you need to manually move it to SUCCESS or ERROR. I think you can give a value to be used as the output too | 17:08 |
| EmilienM | bogdando: done | 17:08 |
| d0ugal | mandre: but yeah, I am not quite sure how this will fit into the idea :) | 17:08 |
| abishop | anyone here familiar with details of building tripleo container images? | 17:08 |
| abishop | tripleo containers are failing "rpm -V selinux-policy-targeted" | 17:08 |
| abishop | not sure who to talk to about this (I also asked in #openstack-kolla) | 17:08 |
| panda | chem: do you have the list of reviews/LP for the other jobs in the card too ? | 17:09 |
| mandre | d0ugal: I suppose we would need an action that looks for all the executions waiting for a stack status update, filtering by stack name, and resume their execution | 17:09 |
| chem | panda: hum ... so list of lp *and* review .. was confused by the title, looked like only lp | 17:10 |
| chem | panda: I can add the review np | 17:10 |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 17:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 17:10 |
| mandre | d0ugal: I'll think about it more this week end, sounds fun :) | 17:10 |
| *** ebarrera has quit IRC | 17:11 | |
| d0ugal | mandre: let me know what you find! (I am on PTO next week tho') | 17:12 |
| mandre | d0ugal: sure, if we're lucky we may even have a patch to try when you're back | 17:13 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-docs master: containers: document kernel for 3rd party https://review.openstack.org/550608 | 17:14 |
| *** dtantsur is now known as dtantsur|afk | 17:16 | |
| *** agopi|lunch is now known as agopi | 17:19 | |
| *** pcaruana has quit IRC | 17:23 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: DNM, test ovs-2.9 https://review.openstack.org/555873 | 17:23 |
| *** ebarrera_ has joined #tripleo | 17:24 | |
| *** itlinux has quit IRC | 17:27 | |
| *** agopi is now known as agopi|out | 17:28 | |
| *** jpich has quit IRC | 17:28 | |
| *** ktibi has quit IRC | 17:29 | |
| *** beekneemech has quit IRC | 17:29 | |
| *** marios has quit IRC | 17:30 | |
| *** bnemec has joined #tripleo | 17:32 | |
| *** bnemec is now known as beekneemech | 17:33 | |
| *** agopi|out has quit IRC | 17:33 | |
| *** beekneemech has quit IRC | 17:34 | |
| *** bnemec has joined #tripleo | 17:34 | |
| *** dprince has quit IRC | 17:36 | |
| *** bogdando has quit IRC | 17:36 | |
| *** trown has quit IRC | 17:40 | |
| *** holser__ has quit IRC | 17:40 | |
| alee_ | beagles, mwhahaha whats the status on the barbican CIX issue? any progress in tracking down the networking problems? | 17:42 |
| mwhahaha | alee_: shouldn't you answer that? :D | 17:43 |
| * mwhahaha has no idea was dealing with other blockers | 17:43 | |
| mwhahaha | we're seeing ssh timeouts in other cases now | 17:43 |
| *** jpena is now known as jpena|off | 17:43 | |
| *** psachin has joined #tripleo | 17:44 | |
| weshay | mwhahaha, fyi.. last check https://github.com/openstack/tripleo-common/commits/master bf4602b6 is not in tripleo promote jobs yet | 17:44 |
| alee_ | mwhahaha, I'm not sure where to look on network issues -- I was hoping beagles had some ideas. | 17:44 |
| weshay | tripleo-common is at 9f15b68 | 17:45 |
| weshay | one patch behind | 17:45 |
| mwhahaha | weshay: k | 17:45 |
| mwhahaha | so the next run | 17:45 |
| weshay | mwhahaha, should be unless the packaging guys are holding it for current -> consistent | 17:45 |
| *** derekh has quit IRC | 17:46 | |
| mwhahaha | alee_: so we've got Bug 1757556 which kinda matches the same thing, not completely sure at the moment where we're at. There was some talk about testing a new ovs | 17:47 |
| openstack | bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] https://launchpad.net/bugs/1757556 | 17:47 |
| alee_ | mwhahaha, yup - certainly seems like the same symptoms | 17:48 |
| alee_ | mwhahaha, we're calling the same tempest code to make ssh connections | 17:49 |
| panda | chem: the change looks good to me, I just want to be sure that all the failures in the jobs are unrelated | 17:52 |
| *** wojdec has left #tripleo | 17:52 | |
| chem | panda: they are unrelate | 17:52 |
| chem | d | 17:52 |
| chem | panda: that patch fixes issue we had ... weeks (month?) ago | 17:53 |
| chem | panda: the error were that we were deploying queen containers for a pike overcloud deployment | 17:54 |
| chem | panda: (at the time that was master) | 17:54 |
| chem | panda: it made the job fail very very early | 17:55 |
| chem | panda: we are well past that point now | 17:55 |
| *** ipsecguy has quit IRC | 17:55 | |
| *** radeks__ has quit IRC | 17:56 | |
| chem | panda: dunno what else to say to convice you | 17:57 |
| *** yamahata has quit IRC | 17:57 | |
| chem | panda: but for sure the current issue are not related | 17:57 |
| chem | panda: just filled the p->q upgrade current erro https://bugs.launchpad.net/tripleo/+bug/1758406 | 17:58 |
| openstack | Launchpad bug 1758406 in tripleo "Fluentd-client not found during P->Q ugrade." [Critical,Confirmed] | 17:58 |
| chem | we have a card for that as well https://trello.com/c/ml8rMPVu/440-fix-servicesfluentdclient | 17:58 |
| chem | panda: and a bz :) | 17:58 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-heat-templates master: Use ironic::inspector::dnsmasq_ip_subnets https://review.openstack.org/543582 | 17:58 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-heat-templates master: Container Undercloud - Enable neutron segments https://review.openstack.org/555424 | 17:58 |
| *** ipsecguy has joined #tripleo | 18:00 | |
| chem | panda: and I have updated https://trello.com/c/L7SqVPhg/657-current-jobs-passing-lp-for-all-known-issues as well | 18:00 |
| chem | panda: for ffu I cannot tell what will be the next error, I'm reaching upstream timeout because everything works during 3h | 18:01 |
| chem | panda: see http://logs.openstack.org/64/552064/25/experimental/tripleo-ci-centos-7-scenario001-multinode-ffu-upgrade/635fc05/ | 18:01 |
| *** ipsecguy has quit IRC | 18:01 | |
| *** ykarel has joined #tripleo | 18:02 | |
| chem | panda: so for ffu we can see here http://logs.openstack.org/64/552064/25/experimental/tripleo-ci-centos-7-scenario001-multinode-ffu-upgrade/635fc05/logs/undercloud/home/zuul/ | 18:03 |
| chem | panda: that container prep happens only during the upgrade step as only upgrade_overcloud_prep_containers.log.txt.gz is present (not overcloud_prep_containers.log.txt.gz) | 18:04 |
| chem | panda: and upgrade_overcloud_prep_containers has the expected values http://logs.openstack.org/64/552064/25/experimental/tripleo-ci-centos-7-scenario001-multinode-ffu-upgrade/635fc05/logs/undercloud/home/zuul/upgrade_overcloud_prep_containers.log.txt.gz | 18:04 |
| chem | panda: in that upgrade jobs (p->q) http://logs.openstack.org/80/552080/13/experimental/tripleo-ci-centos-7-scenario001-multinode-oc-upgrade/a17d01c/logs/undercloud/home/zuul/ | 18:05 |
| chem | panda: we can see the two logs created http://logs.openstack.org/80/552080/13/experimental/tripleo-ci-centos-7-scenario001-multinode-oc-upgrade/a17d01c/logs/undercloud/home/zuul/overcloud_prep_containers.log.txt.gz http://logs.openstack.org/80/552080/13/experimental/tripleo-ci-centos-7-scenario001-multinode-oc-upgrade/a17d01c/logs/undercloud/home/zuul/upgrade_overcloud_prep_containers.log.txt.gz | 18:06 |
| chem | panda: with the correct value in it | 18:06 |
| chem | panda: that's the best I can provide | 18:07 |
| panda | chem: I was talking more about the failure of featureset001 and rhos-12 | 18:07 |
| chem | panda: nice | 18:07 |
| panda | chem: but rhos-12 fails to spawn the test environment | 18:07 |
| panda | chem: featureset001 fails tempest | 18:08 |
| panda | chem: actually no ssh to cirros | 18:09 |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 18:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 18:10 |
| panda | chem: I see featureset001 passed on previous PS | 18:10 |
| panda | chem: PS39 | 18:11 |
| *** ipsecguy has joined #tripleo | 18:11 | |
| panda | chem: rhos-12 was passing in PS 34 | 18:12 |
| panda | chem: newton too | 18:12 |
| *** masco has quit IRC | 18:12 | |
| chem | panda: newton is working now | 18:14 |
| panda | chem: I see it passing for 45 | 18:14 |
| *** agopi|out has joined #tripleo | 18:15 | |
| chem | panda: "tripleo-quickstart-extras-gate-newton-delorean-full-minimalSUCCESS" on ps46 | 18:15 |
| *** slagle has quit IRC | 18:15 | |
| *** pkovar has quit IRC | 18:16 | |
| panda | chem: ah yeah, I see it too | 18:16 |
| chem | panda: so that leave us with featureset001 | 18:16 |
| weshay | mwhahaha, not sure if this will work, but https://review.openstack.org/#/c/555873/ | 18:17 |
| panda | chem: which is failing in the other gates too | 18:18 |
| chem | panda: yeah was checking if it was the same tempest errro | 18:19 |
| chem | panda: but opened tempest log file and firefox crashed | 18:19 |
| hjensas | anyone want to give this one a review: https://review.openstack.org/523944 , thanks! | 18:20 |
| *** cshastri has joined #tripleo | 18:22 | |
| chem | panda: so we're good :) | 18:23 |
| openstackgerrit | Merged openstack/tripleo-ui stable/pike: Release 7.4.9 (pike) https://review.openstack.org/555856 | 18:25 |
| openstackgerrit | Merged openstack/tripleo-ui stable/queens: Release 8.3.1 (queens) https://review.openstack.org/555858 | 18:25 |
| openstackgerrit | Merged openstack/tripleo-ui stable/ocata: Release 3.2.3 (ocata) https://review.openstack.org/555854 | 18:25 |
| *** psachin has quit IRC | 18:26 | |
| *** dciabrin_ has joined #tripleo | 18:26 | |
| *** dciabrin has quit IRC | 18:28 | |
| *** aputtur has quit IRC | 18:29 | |
| *** yamahata has joined #tripleo | 18:29 | |
| *** ebarrera_ has quit IRC | 18:30 | |
| panda | chem: I think we are goo | 18:31 |
| chem | \o/ :) | 18:31 |
| * panda melts | 18:31 | |
| panda | I am goo | 18:31 |
| *** itlinux has joined #tripleo | 18:32 | |
| *** gfidente has quit IRC | 18:32 | |
| chem | panda: ah, oki, no +2/w then ? | 18:33 |
| panda | chem: I am goo, I have no hands to +2 | 18:34 |
| chem | panda: oki, then | 18:34 |
| chem | panda: thanks goo :) | 18:35 |
| chem | panda: I have no idea how it "sounds" in english | 18:35 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder master: new diskimage-build command with config https://review.openstack.org/554002 | 18:40 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder master: Move fail early checks in to python https://review.openstack.org/555462 | 18:40 |
| openstackgerrit | greghaynes proposed openstack/diskimage-builder master: Move environment create/cleanup to python https://review.openstack.org/555463 | 18:40 |
| *** pchavva1 has joined #tripleo | 18:45 | |
| *** itlinux has quit IRC | 18:47 | |
| *** itlinux has joined #tripleo | 18:49 | |
| *** dbecker has joined #tripleo | 18:53 | |
| *** salmankhan has quit IRC | 18:59 | |
| openstackgerrit | Merged openstack/diskimage-builder master: enable systemd profile for Gentoo https://review.openstack.org/552952 | 19:01 |
| *** ykarel has quit IRC | 19:07 | |
| *** itlinux has quit IRC | 19:08 | |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 19:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 19:10 |
| *** athomas has quit IRC | 19:10 | |
| *** cshastri has quit IRC | 19:11 | |
| openstackgerrit | Jill Rouleau proposed openstack/python-tripleoclient master: Add missing unit tests for undercloud_deploy https://review.openstack.org/555456 | 19:12 |
| *** itlinux has joined #tripleo | 19:15 | |
| *** itlinux has quit IRC | 19:16 | |
| *** zaneb has joined #tripleo | 19:18 | |
| *** dbecker has quit IRC | 19:27 | |
| *** bkopilov has quit IRC | 19:27 | |
| *** bkopilov has joined #tripleo | 19:31 | |
| *** openstackgerrit has quit IRC | 19:34 | |
| *** agopi|out has quit IRC | 19:43 | |
| trozet | hi guys with update and upgrade, is puppet re-ran for both cases? | 19:44 |
| *** pradk has joined #tripleo | 19:48 | |
| *** etingof|afk is now known as etingof | 19:51 | |
| *** fragatina has quit IRC | 19:52 | |
| *** agopi|out has joined #tripleo | 19:53 | |
| *** moshele has joined #tripleo | 19:55 | |
| *** florianf has quit IRC | 19:57 | |
| *** openstackgerrit has joined #tripleo | 20:05 | |
| openstackgerrit | Merged openstack/os-net-config stable/queens: Adding ethtool command after binding dpdk drivers in Mellanox nics https://review.openstack.org/555718 | 20:05 |
| *** ansmith has quit IRC | 20:05 | |
| *** morazi has quit IRC | 20:06 | |
| *** itlinux has joined #tripleo | 20:07 | |
| *** itlinux has quit IRC | 20:09 | |
| *** tesseract has quit IRC | 20:09 | |
| *** sai_p has joined #tripleo | 20:09 | |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 20:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 20:10 |
| *** panda is now known as panda|off | 20:12 | |
| *** pchavva1 has quit IRC | 20:13 | |
| openstackgerrit | Merged openstack/tripleo-validations stable/queens: Fix MySQL Open Files Limit validation https://review.openstack.org/555255 | 20:15 |
| *** pchavva has quit IRC | 20:20 | |
| *** thrash is now known as thrash|g0ne | 20:21 | |
| *** mcornea is now known as mcornea|afk | 20:21 | |
| *** toure is now known as toure|gone | 20:29 | |
| openstackgerrit | Damien Ciabrini proposed openstack/tripleo-heat-templates master: HAProxy: fix bind mount to expose stats socket https://review.openstack.org/555928 | 20:30 |
| *** mcornea|afk has quit IRC | 20:32 | |
| *** raildo has quit IRC | 20:33 | |
| *** bfournie has quit IRC | 20:45 | |
| *** bfournie has joined #tripleo | 20:46 | |
| *** bfournie has quit IRC | 20:50 | |
| *** dciabrin_ has quit IRC | 20:50 | |
| *** dciabrin_ has joined #tripleo | 20:50 | |
| *** abishop has quit IRC | 20:52 | |
| *** fzdarsky|afk has quit IRC | 21:05 | |
| *** zaneb has quit IRC | 21:09 | |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 21:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 21:10 |
| *** tcw has quit IRC | 21:11 | |
| *** fragatina has joined #tripleo | 21:11 | |
| *** akrivoka has quit IRC | 21:19 | |
| *** jbadiapa has quit IRC | 21:19 | |
| *** rfolco|ruck is now known as rfolco|off | 21:20 | |
| *** moshele has quit IRC | 21:21 | |
| *** fultonj has quit IRC | 21:22 | |
| *** ansmith has joined #tripleo | 21:25 | |
| *** bnemec is now known as beekneemech | 21:32 | |
| beekneemech | My OCD is really unhappy that the columns in https://raw.githubusercontent.com/openstack/tripleo-heat-templates/master/README.rst are and even number of characters so all the X's are off-center. | 21:32 |
| beekneemech | #FridayThoughts | 21:33 |
| beekneemech | *an | 21:33 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates master: Deploy Designate in scenario003 https://review.openstack.org/555007 | 21:34 |
| *** wolverineav has joined #tripleo | 21:36 | |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates master: Add environment to enable Designate https://review.openstack.org/555006 | 21:41 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates master: Deploy Designate in scenario003 https://review.openstack.org/555007 | 21:41 |
| * beekneemech did that in the wrong order | 21:42 | |
| * beekneemech blames off-center X's | 21:42 | |
| openstackgerrit | Jill Rouleau proposed openstack/python-tripleoclient master: Make download_config backwards compatible https://review.openstack.org/555465 | 21:42 |
| *** chlong has joined #tripleo | 21:42 | |
| *** radeks__ has joined #tripleo | 21:51 | |
| *** jlabarre has quit IRC | 21:59 | |
| *** pblaho has quit IRC | 21:59 | |
| *** jmelvin has quit IRC | 22:00 | |
| *** colonwq has quit IRC | 22:04 | |
| *** colonwq has joined #tripleo | 22:08 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 22:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 22:10 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Collect installed cron jobs https://review.openstack.org/554889 | 22:17 |
| openstackgerrit | Merged openstack/instack-undercloud master: Add Nova UI endpoints to the endpoint list https://review.openstack.org/553814 | 22:17 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Add configuration for the Nova proxy endpoint https://review.openstack.org/552647 | 22:17 |
| openstackgerrit | Merged openstack/tripleo-common stable/pike: Increase the image size for security hardened images to 23G https://review.openstack.org/555726 | 22:17 |
| openstackgerrit | Merged openstack/tripleo-common stable/queens: Increase the image size for security hardened images to 23G https://review.openstack.org/555729 | 22:17 |
| *** zaneb has joined #tripleo | 22:23 | |
| *** Goneri has quit IRC | 22:25 | |
| *** radeks__ has quit IRC | 22:25 | |
| *** dparkes has joined #tripleo | 22:29 | |
| *** liverpooler has quit IRC | 22:32 | |
| *** EmilienM has quit IRC | 22:34 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-quickstart master: DNM, test ovs-2.9 https://review.openstack.org/555873 | 22:35 |
| openstackgerrit | Merged openstack/puppet-tripleo stable/queens: Allow custom per-service listen_options for haproxy https://review.openstack.org/554153 | 22:36 |
| openstackgerrit | Merged openstack/tripleo-upgrade master: Use rhos-release role pin puddle option https://review.openstack.org/555018 | 22:36 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: Include pacemaker in Telemetry role https://review.openstack.org/555739 | 22:36 |
| *** EmilienM has joined #tripleo | 22:36 | |
| *** ChanServ sets mode: +v EmilienM | 22:36 | |
| *** slagle has joined #tripleo | 22:37 | |
| *** rlandy has quit IRC | 22:43 | |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-quickstart-extras master: Do not override NeutronServicePlugins in CI https://review.openstack.org/555950 | 22:58 |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-heat-templates master: Container Undercloud - Enable neutron segments https://review.openstack.org/555424 | 23:00 |
| *** artom has quit IRC | 23:08 | |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757474 | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1757556 | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1757474 in tripleo "periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset027-master fails with undefined groups.overcloud" [Critical,In progress] - Assigned to Bogdan Dobrelya (bogdando) | 23:10 |
| openstack | Launchpad bug 1757556 in tripleo "timeouts in neutron are causing ssh failures in tempest test instances" [Critical,Triaged] | 23:10 |
| *** yamahata has quit IRC | 23:16 | |
| *** tosky has quit IRC | 23:19 | |
| *** wolverineav has quit IRC | 23:24 | |
| *** wolverineav has joined #tripleo | 23:24 | |
| *** wolverineav has quit IRC | 23:25 | |
| *** wolverineav has joined #tripleo | 23:25 | |
| *** holser__ has joined #tripleo | 23:29 | |
| *** wolverineav has quit IRC | 23:30 | |
| *** wolverineav has joined #tripleo | 23:32 | |
| *** pradk has quit IRC | 23:34 | |
| *** holser__ has quit IRC | 23:43 | |
| *** jgwentworth is now known as melwitt | 23:54 | |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/472607 | 23:55 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
