| *** ooolpbot has joined #tripleo | 00:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 00:10 |
|---|---|---|
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 00:10 |
| *** ooolpbot has quit IRC | 00:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 00:10 |
| *** michchap_ has joined #tripleo | 00:11 | |
| *** chlong has quit IRC | 00:13 | |
| *** michchap has quit IRC | 00:13 | |
| *** yolanda has joined #tripleo | 00:13 | |
| *** limao has joined #tripleo | 00:25 | |
| *** chlong has joined #tripleo | 00:26 | |
| *** dmacpher has joined #tripleo | 00:37 | |
| *** bana_k has quit IRC | 00:41 | |
| openstackgerrit | Giulio Fidente proposed openstack/python-tripleoclient: Use heatclient poll_for_events in wait_for_stack_ready https://review.openstack.org/360141 | 00:44 |
| *** bfournie has joined #tripleo | 00:49 | |
| *** bana_k has joined #tripleo | 00:50 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements https://review.openstack.org/348695 | 00:52 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Updates and fixes to NIC config template routes https://review.openstack.org/318389 | 00:52 |
| *** thrash is now known as thrash|g0ne | 00:55 | |
| *** ooolpbot has joined #tripleo | 01:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 01:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 01:10 |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 01:10 |
| *** ooolpbot has quit IRC | 01:10 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Move Swift hiera settings into composable services https://review.openstack.org/360058 | 01:11 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Mv keystone, mongo, manila out of controller.yaml https://review.openstack.org/360060 | 01:11 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Move glance/heat hiera settings to services https://review.openstack.org/360029 | 01:11 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Move Swift hiera settings into composable services https://review.openstack.org/360058 | 01:13 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Mv keystone, mongo, manila out of controller.yaml https://review.openstack.org/360060 | 01:13 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Move glance/heat hiera settings to services https://review.openstack.org/360029 | 01:13 |
| *** saneax-_-|AFK is now known as saneax | 01:16 | |
| *** yolanda has quit IRC | 01:18 | |
| *** bana_k has quit IRC | 01:18 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Implement scenario001 CI job https://review.openstack.org/360039 | 01:21 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Test new scenario001 job https://review.openstack.org/360040 | 01:21 |
| *** ebalduf has quit IRC | 01:22 | |
| *** yolanda has joined #tripleo | 01:31 | |
| *** saneax is now known as saneax-_-|AFK | 01:32 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Fix use of snmp service https://review.openstack.org/360062 | 01:36 |
| *** yolanda has quit IRC | 01:37 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Split multinode.yaml template https://review.openstack.org/360168 | 01:38 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add support for deploying Ceph RGW role https://review.openstack.org/289027 | 01:41 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Split multinode.yaml template https://review.openstack.org/360168 | 01:46 |
| *** yolanda has joined #tripleo | 01:47 | |
| *** saneax-_-|AFK is now known as saneax | 01:58 | |
| *** yamahata has quit IRC | 02:03 | |
| openstackgerrit | Xiang Chen proposed openstack/diskimage-builder: Explain difference between two envvar https://review.openstack.org/345935 | 02:04 |
| openstackgerrit | Giulio Fidente proposed openstack/python-tripleoclient: Use heatclient poll_for_events in wait_for_stack_ready https://review.openstack.org/360141 | 02:05 |
| *** jrist has quit IRC | 02:05 | |
| openstackgerrit | Xiang Chen proposed openstack/diskimage-builder: Explain difference between two envvars https://review.openstack.org/345935 | 02:06 |
| *** ooolpbot has joined #tripleo | 02:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 02:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 02:10 |
| *** ooolpbot has quit IRC | 02:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 02:10 |
| *** saneax is now known as saneax-_-|AFK | 02:17 | |
| *** yolanda has quit IRC | 02:17 | |
| *** jrist has joined #tripleo | 02:18 | |
| *** jrist has joined #tripleo | 02:18 | |
| leifmadsen | hey all! somewhat cross-posting from #heat, but I'm trying to understand composable services and roles in newton, deploying with TripleO, and wondering how much knowledge of Puppet would be useful | 02:20 |
| *** yolanda has joined #tripleo | 02:21 | |
| leifmadsen | and by that, I mean, would it be useful going forward to purchase a book on Puppet. I have a lot of DevOps experience with Chef and Ansible (and I originally learned Puppet), but it's been several years since I've used Puppet, and wondering if I should refresh | 02:21 |
| leifmadsen | as a foundation for being able to use t-h-t, heat, and composable services and roles | 02:21 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Test new scenario001 job https://review.openstack.org/360040 | 02:23 |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: Update authtoken parameters to match recent changes https://review.openstack.org/359400 | 02:25 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Update authtoken parameters to match recent changes https://review.openstack.org/359401 | 02:29 |
| *** julim has quit IRC | 02:35 | |
| *** bswartz has quit IRC | 02:38 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Fix path to sahara templates https://review.openstack.org/358607 | 02:47 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Allow a fixed IP to be allocated to RedisVirtualIP https://review.openstack.org/352219 | 02:47 |
| openstackgerrit | Merged openstack/python-tripleoclient: Add 'openstack overcloud plan list' command https://review.openstack.org/350498 | 02:59 |
| openstackgerrit | Merged openstack/python-tripleoclient: Skip post-config based on t-h-t flag https://review.openstack.org/353316 | 03:02 |
| openstackgerrit | Merged openstack/tripleo-common: Fix typo in Introspection workflow message https://review.openstack.org/352430 | 03:05 |
| *** ooolpbot has joined #tripleo | 03:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 03:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 03:10 |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 03:10 |
| *** ooolpbot has quit IRC | 03:10 | |
| *** saneax-_-|AFK is now known as saneax | 03:13 | |
| openstackgerrit | ayoung proposed openstack/puppet-tripleo: Optionally include mod auth mellon for apache https://review.openstack.org/359942 | 03:20 |
| *** chlong has quit IRC | 03:23 | |
| openstackgerrit | Tim Rozet proposed openstack/tripleo-heat-templates: Adds OpenDaylight composable service https://review.openstack.org/355147 | 03:25 |
| *** chlong has joined #tripleo | 03:35 | |
| *** chlong has quit IRC | 03:38 | |
| *** chlong has joined #tripleo | 03:38 | |
| *** rlandy|bbl is now known as rlandy | 03:39 | |
| *** rlandy has quit IRC | 03:40 | |
| *** coolsvap_ has joined #tripleo | 03:50 | |
| *** tobias_fiberdata has quit IRC | 03:57 | |
| *** tobias_fiberdata has joined #tripleo | 03:58 | |
| *** morazi has quit IRC | 03:58 | |
| *** tobias_fiberdata has quit IRC | 04:02 | |
| *** tobias_fiberdata has joined #tripleo | 04:02 | |
| *** saneax is now known as saneax-_-|AFK | 04:03 | |
| *** limao has quit IRC | 04:09 | |
| *** limao has joined #tripleo | 04:09 | |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates: Add fluentd client service https://review.openstack.org/353506 | 04:09 |
| *** ooolpbot has joined #tripleo | 04:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 04:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 04:10 |
| *** ooolpbot has quit IRC | 04:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 04:10 |
| openstackgerrit | Giulio Fidente proposed openstack/python-tripleoclient: Use heatclient poll_for_events in wait_for_stack_ready https://review.openstack.org/360141 | 04:10 |
| *** bswartz has joined #tripleo | 04:12 | |
| *** limao has quit IRC | 04:14 | |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates: Add fluentd client service https://review.openstack.org/353506 | 04:17 |
| *** links has joined #tripleo | 04:37 | |
| *** yamahata has joined #tripleo | 04:50 | |
| *** tzumainn has quit IRC | 04:50 | |
| *** morazi has joined #tripleo | 04:55 | |
| *** limao has joined #tripleo | 04:55 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements https://review.openstack.org/348695 | 04:55 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Register enabled services via global config https://review.openstack.org/359639 | 05:02 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Move service/endpoint configuration to keystone profile https://review.openstack.org/359681 | 05:02 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Conditionally expose haproxy endpoints https://review.openstack.org/359645 | 05:02 |
| *** ayoung has quit IRC | 05:03 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-cloud-config: Updated from global requirements https://review.openstack.org/350676 | 05:04 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements https://review.openstack.org/348695 | 05:06 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Populate HAProxy endpoint ports from EndpointMap https://review.openstack.org/355366 | 05:07 |
| *** masco has joined #tripleo | 05:07 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add HAProxy TLS handled by certmonger as composable service https://review.openstack.org/356430 | 05:09 |
| *** jaosorior has joined #tripleo | 05:10 | |
| *** ooolpbot has joined #tripleo | 05:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 05:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 05:10 |
| *** ooolpbot has quit IRC | 05:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 05:10 |
| openstackgerrit | Saravanan KR proposed openstack/os-net-config: Add support for OVS DPDK Bridge and Port https://review.openstack.org/338694 | 05:38 |
| *** dtrainor has quit IRC | 05:40 | |
| *** dtrainor has joined #tripleo | 05:40 | |
| openstackgerrit | Saravanan KR proposed openstack/os-net-config: Add support for OVS DPDK Bond https://review.openstack.org/349183 | 05:43 |
| *** morazi has quit IRC | 05:46 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Make Undercloud OpenStack service workers configurable https://review.openstack.org/360255 | 05:48 |
| bandini | morning | 05:49 |
| openstackgerrit | Saravanan KR proposed openstack/os-net-config: Supporting numbered nics for DPDK Port and DPDK Bond https://review.openstack.org/359610 | 05:54 |
| *** skramaja has joined #tripleo | 05:59 | |
| *** pkovar has joined #tripleo | 06:02 | |
| *** ccamacho has joined #tripleo | 06:04 | |
| *** rajinir has quit IRC | 06:05 | |
| ccamacho | Morning! /o/ | 06:06 |
| *** nyechiel has joined #tripleo | 06:09 | |
| *** ooolpbot has joined #tripleo | 06:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 06:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 06:10 |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 06:10 |
| *** ooolpbot has quit IRC | 06:10 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Move resource registry override to enable-tls.yaml https://review.openstack.org/358971 | 06:11 |
| *** akuznetsov has joined #tripleo | 06:14 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Composable Mistral services https://review.openstack.org/323436 | 06:18 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Do not merge, checking composable mistral. https://review.openstack.org/352947 | 06:19 |
| *** bana_k has joined #tripleo | 06:23 | |
| *** myoung|pto has quit IRC | 06:25 | |
| *** myoung has joined #tripleo | 06:26 | |
| *** rcernin has joined #tripleo | 06:28 | |
| *** r-mibu has quit IRC | 06:32 | |
| *** jbadiapa has joined #tripleo | 06:35 | |
| *** jubapa has joined #tripleo | 06:36 | |
| openstackgerrit | Christian Schwede proposed openstack/tripleo-heat-templates: Add system-uuid based hostname entries https://review.openstack.org/358643 | 06:38 |
| *** rasca has joined #tripleo | 06:42 | |
| *** fragatina has quit IRC | 06:46 | |
| *** bana_k has quit IRC | 06:47 | |
| *** tobias_fiberdata has quit IRC | 06:47 | |
| *** tobias_fiberdata has joined #tripleo | 06:48 | |
| *** jprovazn has joined #tripleo | 06:51 | |
| *** saneax-_-|AFK is now known as saneax | 06:55 | |
| *** tobias-fiberdata has joined #tripleo | 07:00 | |
| *** tobias_fiberdata has quit IRC | 07:03 | |
| *** numans has joined #tripleo | 07:03 | |
| *** florianf has joined #tripleo | 07:04 | |
| *** tesseract- has joined #tripleo | 07:08 | |
| openstackgerrit | Merged openstack/tripleo-ui: Use Mistral API for plan deletion https://review.openstack.org/356458 | 07:09 |
| *** ifarkas_afk is now known as ifarkas | 07:09 | |
| *** ooolpbot has joined #tripleo | 07:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 07:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 07:10 |
| *** ooolpbot has quit IRC | 07:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 07:10 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Add 'openstack overcloud plan create' command https://review.openstack.org/359784 | 07:11 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Update the 'overcloud deploy --templates' command to use Mistral https://review.openstack.org/342261 | 07:11 |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-heat-templates: Added a new composable service for enabling DPDK on compute nodes. https://review.openstack.org/344084 | 07:14 |
| *** dsariel has joined #tripleo | 07:18 | |
| *** aufi has joined #tripleo | 07:19 | |
| *** tobias-fiberdata has quit IRC | 07:20 | |
| *** tobias-fiberdata has joined #tripleo | 07:21 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Add 'openstack overcloud plan create' command https://review.openstack.org/359784 | 07:27 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Update the 'overcloud deploy --templates' command to use Mistral https://review.openstack.org/342261 | 07:27 |
| *** r-mibu has joined #tripleo | 07:27 | |
| jaosorior | anybody working with tripleo-ui around here? | 07:28 |
| *** mrunge has joined #tripleo | 07:28 | |
| *** saneax is now known as saneax-_-|AFK | 07:30 | |
| *** bootsha has joined #tripleo | 07:32 | |
| *** nyechiel has quit IRC | 07:36 | |
| openstackgerrit | Robin Cernin proposed openstack/tripleo-validations: Validate RabbitMQ HA policies in clustered environment https://review.openstack.org/358430 | 07:37 |
| d0ugal | jaosorior: florianf or jtomasek might be around | 07:38 |
| jtomasek | jaosorior: hi | 07:38 |
| openstackgerrit | Robin Cernin proposed openstack/tripleo-validations: Validation stonith device exists in OpenStack Platform HA cluster https://review.openstack.org/360102 | 07:38 |
| florianf | jaosorior: hi | 07:41 |
| jaosorior | jtomasek, florianf: tripleo-ui is now using zaqar more extensively, right? | 07:41 |
| *** jpich has joined #tripleo | 07:41 | |
| jtomasek | yes | 07:42 |
| florianf | jaosorior: yes, not just tripleo-ui, but the underlying tripleo-common workflows | 07:42 |
| jaosorior | florianf: are you guys using ther same zaqar websockets configuration as what the undercloud is currently deploying? | 07:42 |
| jaosorior | I think it's in port 9999 or something of the sort | 07:42 |
| *** zoli_gone-proxy is now known as zoliXXL | 07:43 | |
| florianf | jaosorior: yes | 07:43 |
| jaosorior | florianf, jtomasek are you using the ws:// keystone endpoint at all? | 07:43 |
| jaosorior | or just the http:// endpoint? | 07:44 |
| d0ugal | jaosorior: FWIW, tripleoclient is also using Zaqar | 07:44 |
| jtomasek | jaosorior: yes we do, there are currently 2 zaqars in endpoints we're using the zaqar-websocket one which runs on port 9000 | 07:44 |
| *** abregman has joined #tripleo | 07:44 | |
| jtomasek | jaosorior: ws endpoint | 07:44 |
| jtomasek | d0ugal: you're using also ws one, right? | 07:44 |
| jtomasek | in CLI | 07:45 |
| d0ugal | jtomasek: I assume so? | 07:45 |
| jaosorior | ah, it's 9000, right | 07:45 |
| d0ugal | jtomasek: https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/plugin.py#L72-L73 | 07:45 |
| *** hewbrocca-afk is now known as hewbrocca | 07:45 | |
| jtomasek | d0ugal: yep, cool so that is the same | 07:46 |
| d0ugal | So yeah, seems so :) | 07:46 |
| jaosorior | awesome | 07:46 |
| *** dmacpher has quit IRC | 07:46 | |
| jaosorior | so this is being tested in CI, right? | 07:46 |
| *** shardy has joined #tripleo | 07:47 | |
| hewbrocca | ... and the room suddenly got quiet | 07:47 |
| jtomasek | iiuc yes, since it is used in CLI, d0ugal^? | 07:47 |
| d0ugal | jaosorior, jtomasek: Yeah, it is | 07:48 |
| d0ugal | jaosorior: currently the node registration and introspection are using it. | 07:48 |
| d0ugal | jaosorior: When deploy lands it will be using it too | 07:48 |
| d0ugal | shardy: Hi! | 07:48 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Add `openstack overcloud deploy --plan` https://review.openstack.org/360305 | 07:49 |
| shardy | d0ugal: Hi! | 07:49 |
| shardy | morning all, how is everything going? | 07:50 |
| jaosorior | d0ugal: excellent. So then if I fuck it up I will know easier. | 07:50 |
| d0ugal | jaosorior: I assume so :-D | 07:50 |
| * shardy looks nervously at email backlog | 07:50 | |
| d0ugal | shardy: welcome back, good I think! | 07:50 |
| jaosorior | shardy: welcome back dude | 07:50 |
| d0ugal | shardy: we have been merging all sorts of things in your absence ;) | 07:50 |
| *** zoliXXL is now known as zoli|trng | 07:51 | |
| openstackgerrit | Christian Schwede proposed openstack/tripleo-docs: Deploy and Scale Swift https://review.openstack.org/293311 | 07:51 |
| shardy | d0ugal: great! (I think?) ;) | 07:51 |
| shardy | jaosorior: thanks! | 07:51 |
| *** jmiu has quit IRC | 07:51 | |
| ccamacho | nope | 07:52 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: [WIP] Add `openstack overcloud deploy --plan` https://review.openstack.org/360305 | 07:52 |
| jtomasek | welcome back shardy :) | 07:56 |
| shardy | jtomasek: thanks! :) | 07:56 |
| florianf | d0ugal: is the cli testing against a real zaqar instance? or just mocks? | 07:56 |
| jtomasek | btw. anyone have an idea why Controller role profile name on the node is called 'control'? capabilities='profile:control' | 07:57 |
| d0ugal | florianf: Real for the CI run, mocks for unit tests. | 07:57 |
| jtomasek | this is not pretty, because there is no way to get this name from anywhere in the backend (nor roles list action or parameters listing) :/ | 07:57 |
| florianf | d0ugal: cool, thanks. | 07:57 |
| *** jmiu has joined #tripleo | 07:57 | |
| hewbrocca | hey bandini and Ng have you seen https://bugs.launchpad.net/tripleo/+bug/1585275 | 08:00 |
| openstack | Launchpad bug 1585275 in tripleo "SQL errors on undercloud/overcloud cause HA deployments to fail" [Undecided,New] | 08:00 |
| *** mcornea has joined #tripleo | 08:00 | |
| * Ng looks | 08:00 | |
| hewbrocca | the folks working on OPNFV have found and fixed a number of issues like this | 08:01 |
| hewbrocca | trozet: being one of them | 08:01 |
| hewbrocca | one of the folks, not one of the issues :) | 08:01 |
| bandini | lol | 08:01 |
| * bandini looks | 08:01 | |
| bandini | trozet: do we have some failure logs with a recent tripleo? We changed so much in that regards with the composable roles (also the links to logs in the LP bug are all 404s now) | 08:06 |
| *** nyechiel has joined #tripleo | 08:06 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: [WIP] Add `openstack overcloud deploy --plan` https://review.openstack.org/360305 | 08:06 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Update the 'overcloud deploy --templates' command to use Mistral https://review.openstack.org/342261 | 08:06 |
| *** limao has quit IRC | 08:07 | |
| hewbrocca | bandini: he's US-East, so he won't get that until later | 08:07 |
| hewbrocca | bandini: worth re-raising with him when he does get in, I guess. Thanks! | 08:08 |
| *** limao has joined #tripleo | 08:08 | |
| hewbrocca | bandini: oh, do you happen to know if we killed os-cloud-config yet? | 08:08 |
| hewbrocca | os-cloud-config.die.die.die | 08:08 |
| *** limao has quit IRC | 08:08 | |
| bandini | hewbrocca: I don't think we did yet. but yes DIE DIE DIE | 08:08 |
| *** limao has joined #tripleo | 08:09 | |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-quickstart: Specify which type of config we are using for the deployment https://review.openstack.org/360315 | 08:09 |
| bandini | hewbrocca: I will shoot Tim an email, just to be sure (I will forget we chatted in like..2 hours) ;) | 08:09 |
| *** jpena|away is now known as jpena | 08:09 | |
| bandini | I am going senile at insane speeds | 08:10 |
| *** ooolpbot has joined #tripleo | 08:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 08:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 08:10 |
| *** ooolpbot has quit IRC | 08:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 08:10 |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-quickstart: Specify which type of config we are using for the deployment https://review.openstack.org/360315 | 08:11 |
| hewbrocca | shardy: GOOD MORNING | 08:12 |
| hewbrocca | and welcome back! | 08:12 |
| shardy | hewbrocca: Hi! Thanks! :) | 08:12 |
| bandini | shardy: welcome back and congrats ;) | 08:12 |
| shardy | thanks bandini :) | 08:12 |
| Ng | shardy: congrats! how are things going? | 08:14 |
| *** athomas has joined #tripleo | 08:15 | |
| shardy | Ng: thanks! All going well :) | 08:15 |
| *** skramaja has quit IRC | 08:15 | |
| Ng | :D | 08:15 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates: Availability monitoring agents support https://review.openstack.org/254788 | 08:20 |
| *** chem has joined #tripleo | 08:22 | |
| *** skramaja has joined #tripleo | 08:23 | |
| *** pgadiya has joined #tripleo | 08:23 | |
| *** abehl has joined #tripleo | 08:24 | |
| openstackgerrit | Erno Kuvaja proposed openstack/puppet-tripleo: Add Manila CephFS backend to manila class https://review.openstack.org/354047 | 08:27 |
| *** lucas-dinner is now known as lucasagomes | 08:28 | |
| *** tobias_fiberdata has joined #tripleo | 08:28 | |
| Jokke_ | anyone able to help me with this http://logs.openstack.org/94/355394/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/d338887/console.html#_2016-08-23_12_43_14_654866 I have rechecked https://review.openstack.org/#/c/355394/ bit too many times now | 08:31 |
| *** tobias-fiberdata has quit IRC | 08:31 | |
| Jokke_ | shardy: welcome back and Congratulations! | 08:31 |
| shardy | Jokke_: thanks! | 08:31 |
| *** dtantsur|afk is now known as dtantsur | 08:32 | |
| shardy | Jokke_: looks like https://bugs.launchpad.net/tripleo/+bug/1612622 | 08:33 |
| openstack | Launchpad bug 1612622 in tripleo "CI: Exception registering nodes: Node 59ecc491-41b0-4950-8d00-ddce532ba0f4 is locked by host localhost.localdomain, please retry after the current operation is completed" [Critical,Triaged] - Assigned to James Slagle (james-slagle) | 08:33 |
| *** saneax-_-|AFK is now known as saneax | 08:33 | |
| dtantsur | morning my owl friends! how is gate today? | 08:34 |
| dtantsur | also welcome back shardy | 08:35 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Terminate Zaqar websocket endpoint in HAProxy https://review.openstack.org/360329 | 08:37 |
| *** ccamacho is now known as ccamacho|afk | 08:37 | |
| *** xuao has joined #tripleo | 08:40 | |
| Jokke_ | thanks shardy, so it seems it's affecting not just periodics | 08:40 |
| xuao | Morning all~ | 08:40 |
| Jokke_ | moin xuao | 08:41 |
| xuao | hello Jokke_ | 08:42 |
| xuao | I have a question about diskimage-builder. I found a elem 'target_disk' in the file 'elements/deploy-ironic/init.d/80-deploy-ironic' and wander where it's defined. | 08:45 |
| xuao | T .T | 08:45 |
| xuao | I'm so sorry | 08:48 |
| *** openstackgerrit has quit IRC | 08:48 | |
| xuao | Please ignore it. | 08:48 |
| *** openstackgerrit has joined #tripleo | 08:48 | |
| xuao | : ). | 08:49 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Enable TLS for Zaqar's websocket endpoint https://review.openstack.org/360350 | 08:52 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Enable TLS for Zaqar's websocket endpoint https://review.openstack.org/360350 | 08:53 |
| openstackgerrit | Christian Schwede proposed openstack/tripleo-docs: Update advanced Swift deployment doc https://review.openstack.org/360353 | 08:55 |
| jpena | quick question: are we setting up lbaasv1 in tripleo? If so, we'll have to fix that because upstream has removed it -> https://review.openstack.org/286381 | 08:59 |
| openstackgerrit | Christian Schwede proposed openstack/tripleo-docs: Update advanced Swift deployment doc https://review.openstack.org/360353 | 09:00 |
| *** derekh has joined #tripleo | 09:00 | |
| *** akrivoka has joined #tripleo | 09:04 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates: Refactor upgrade checks. https://review.openstack.org/357750 | 09:04 |
| *** aqkhan_ has joined #tripleo | 09:06 | |
| *** bootsha has quit IRC | 09:07 | |
| *** aqkhan__ has quit IRC | 09:10 | |
| *** ooolpbot has joined #tripleo | 09:11 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:11 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 09:11 |
| *** ooolpbot has quit IRC | 09:11 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 09:11 |
| openstackgerrit | Tomas Sedovic proposed openstack/tripleo-validations: Validate rabbitmq limits https://review.openstack.org/353970 | 09:14 |
| *** ccamacho|afk is now known as ccamacho | 09:14 | |
| *** nyechiel has quit IRC | 09:16 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Make HAProxy terminate OpenStack internal/admin endpoints https://review.openstack.org/360366 | 09:19 |
| openstackgerrit | Merged openstack/tripleo-docs: Document Ceph repositories to be enabled https://review.openstack.org/352910 | 09:22 |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-docs: [WIP] Documentation for installing and using Ironic in overcloud https://review.openstack.org/354016 | 09:23 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Always deploy HAProxy https://review.openstack.org/360368 | 09:24 |
| openstackgerrit | Christian Schwede proposed openstack/tripleo-docs: Update advanced Swift deployment doc https://review.openstack.org/360353 | 09:27 |
| openstackgerrit | Martin André proposed openstack/puppet-tripleo: [WIP] UI profile for tripleo https://review.openstack.org/354630 | 09:27 |
| jaosorior | shardy: hey man, but the way, I'm trying to introduce global settings for the services. The profiles will have an output global_settings and they end up in all-nodes-config https://review.openstack.org/#/c/359639/ Can you check it out? Would be good to have your opinion on that | 09:30 |
| jaosorior | shardy: we can use that, for instance, for the keystone endpoints https://review.openstack.org/#/c/359680/ | 09:32 |
| openstackgerrit | Merged openstack/tripleo-specs: tripleo-sriov https://review.openstack.org/313872 | 09:32 |
| shardy | jaosorior: is the use-case only to tell which services are enabled? | 09:34 |
| shardy | jaosorior: because I think my existing all-nodes series handles that in a slightly less invasive way: | 09:34 |
| shardy | https://review.openstack.org/#/c/348974 | 09:34 |
| jaosorior | shardy: so far. But thought it would be useful in case there needed to be other information that should be exposed globally. | 09:34 |
| shardy | jaosorior: sure, I'm just trying to think of when that will be needed, other than foo_enabled | 09:35 |
| shardy | jaosorior: with my patch, if e.g nova_api is enabled, there will be a nova_api_node_ips entry in the all-nodes | 09:36 |
| shardy | and I was expecting us to use that to derive services that are enabled e.g for the endpoint creation etc | 09:36 |
| shardy | vs wiring in explicit booleans | 09:36 |
| jaosorior | I see | 09:37 |
| jaosorior | let me check that out | 09:37 |
| jaosorior | shardy: though I do think that booleans is a more intuitive way (better for readability too) of doing such a thing. | 09:37 |
| shardy | jaosorior: check https://review.openstack.org/#/c/348974/4/overcloud.yaml - there's already an EnabledServices list, which has a list of all enabled service names | 09:38 |
| shardy | (only for the Controller right now, but we'll wire it in for all roles) | 09:38 |
| shardy | jaosorior: maybe, my thought was if foo in $enabled_services is just as intutive and less invasive from a template perspective | 09:38 |
| shardy | e.g we already know a service is enabled, if it's in the list of enabled services | 09:39 |
| jaosorior | shardy: and how can I get that in the puppet manifests? | 09:39 |
| shardy | jaosorior: we can just pass a list in via hiera | 09:39 |
| shardy | same as I'm already doing for the service_ips | 09:39 |
| shardy | jaosorior: Or, if you really want a boolean, we can generate a mapping with foo_enabled: true in it | 09:39 |
| jaosorior | shardy: where do the service_ips end up? all nodes? | 09:39 |
| shardy | jaosorior: yup, all nodes | 09:40 |
| jaosorior | ok, I see nova_api_node_ips: ['172.17.0.16','172.17.0.10','172.17.0.14'] | 09:40 |
| jaosorior | being dumped there | 09:40 |
| shardy | jaosorior: Yeah, so you could just look for that, and if it's defined do something such as create endpoints | 09:41 |
| shardy | or, we could have enabled_services: nova_api,... | 09:41 |
| *** tosky has joined #tripleo | 09:42 | |
| shardy | jaosorior: one thing to note is the *_node_ips is only generated for those services defined in the ServiceNetMap | 09:42 |
| jaosorior | shardy: I just need a way to use this via puppet and I'm in. If it's intuitive I'll be happier though haha | 09:42 |
| shardy | so, e.g there will be an entry for heat_api, but not heat_engine | 09:42 |
| jaosorior | I see | 09:42 |
| *** fragatina has joined #tripleo | 09:43 | |
| jaosorior | well, that's the thing that I liked about the global_settings approach | 09:43 |
| jaosorior | whatever is needed globally can be set there | 09:43 |
| jaosorior | shardy: anyway, so these are the two use cases for now: | 09:43 |
| jaosorior | * Keystone endpoint creation https://review.openstack.org/#/c/359680/ | 09:43 |
| shardy | jaosorior: Sure, I'm not opposed to that if there are really global settings that need to be wired in | 09:43 |
| jaosorior | * haproxy endpoints https://review.openstack.org/#/c/359645/ | 09:44 |
| jaosorior | as long as those are covered I'm happy | 09:44 |
| shardy | but I don't think it's necessarily justified if it's just to determine enabled services, because we already have that data as an output from the composable services interfaces | 09:44 |
| jaosorior | shardy: I see your point. For me, as long as those two use-cases can be covered. I can drop the global_settings approach. That's not a big deal | 09:45 |
| shardy | jaosorior: ack, let me see if I can just wire in a foo_enabled map as well as the foo_node_ips stuff | 09:46 |
| shardy | then your patch should work with no changes at all | 09:46 |
| *** bootsha has joined #tripleo | 09:47 | |
| *** fragatina has quit IRC | 09:47 | |
| *** dsariel has quit IRC | 09:47 | |
| derekh | sshnaidm|afk: when your back, want to set up the dev env on rh1? | 09:50 |
| *** gfidente|afk is now known as gfidente | 09:56 | |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates: [WIP] Set CORS for tripleo-ui https://review.openstack.org/360392 | 09:59 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates: Refactor upgrade checks. https://review.openstack.org/357750 | 09:59 |
| *** limao has quit IRC | 10:01 | |
| *** sshnaidm|afk is now known as sshnaidm | 10:01 | |
| sshnaidm | derekh, hi | 10:01 |
| derekh | sshnaidm: sorry on a call now, 30 minutes | 10:02 |
| sshnaidm | derekh, sure | 10:02 |
| *** numans has quit IRC | 10:03 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: WIP: DONT MERGE Testin OOOQ job https://review.openstack.org/359146 | 10:03 |
| *** zoli|trng is now known as zoli|lunch | 10:03 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: [WIP] Set parameters https://review.openstack.org/360391 | 10:04 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: [WIP] Add `openstack overcloud deploy --plan` https://review.openstack.org/360305 | 10:04 |
| shadower | shardy: I'm trying to get a release of tripleo-validations and there seems to be some contention regarding its version number: https://review.openstack.org/#/c/359821/ | 10:05 |
| shadower | shardy: could you please chime in when you have a moment? | 10:05 |
| shardy | shadower: Will do, can we just tag it as part of the n-3 milestone next week? | 10:07 |
| shadower | shardy: normally I'd be happy, but I've been told by one of the downstream packages that their work is blocked on this | 10:08 |
| *** nyechiel has joined #tripleo | 10:08 | |
| shadower | apparently they need the tags and release artifacts to move forward | 10:08 |
| *** jubapa has quit IRC | 10:08 | |
| shadower | shardy: and this repo was'nt released at all yet | 10:09 |
| *** jbadiapa has quit IRC | 10:09 | |
| shadower | so assuming we can, I'd love to get one out asap, and then coordinate the other releases with the rest of the tripleo repos | 10:09 |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 10:10 |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 10:10 |
| d0ugal | jpich: This is a bit late, but it would be cool to include the stack status in a plan list. | 10:10 |
| jpich | d0ugal: Sounds cool! If it's not too complicated to implement sounds like that might even fit as a non-feature patch before RC too | 10:12 |
| jpich | d0ugal: Would you open a bug to track the idea? | 10:12 |
| d0ugal | jpich: I might try and sneak it in | 10:12 |
| jpich | :D | 10:12 |
| d0ugal | jpich: Fine, I'll do that and follow procedure :P | 10:12 |
| shardy | shadower: ack, agreed - I don't see why we should be prevented from tagging a release then also releasing as part of newton-3 | 10:12 |
| jpich | d0ugal: You could do both! How exciting | 10:12 |
| shardy | I've commented and we can wait for feedback from Doug re the best way to do it | 10:12 |
| shadower | shardy: agreed, thanks! | 10:13 |
| d0ugal | jpich: https://bugs.launchpad.net/tripleo/+bug/1616846 | 10:15 |
| openstack | Launchpad bug 1616846 in tripleo "Add the stack status to `openstack overcloud plan list`" [Low,Confirmed] | 10:15 |
| jpich | d0ugal: A thing of beauty | 10:15 |
| jpich | Thanks! :) | 10:15 |
| *** jschlueter has quit IRC | 10:16 | |
| *** jschlueter has joined #tripleo | 10:16 | |
| *** electrofelix has joined #tripleo | 10:17 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: [WIP] Set parameters https://review.openstack.org/360391 | 10:17 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: [WIP] Add `openstack overcloud deploy --plan` https://review.openstack.org/360305 | 10:17 |
| rook | anyone noticing when a host is scheduled and goes into ERROR - the instance UUID is never removed from said ironic node? | 10:17 |
| rook | this is causing tons of failures in my lab. | 10:17 |
| dtantsur | rook, not sure about "never", I thought it's "sometimes".. it's very bad if this is true. what was the cause of the ERROR? | 10:18 |
| rook | dtantsur seems to be rescheduling for some reason ( i haven't diagnosed the reason yet ). however it causes the deploy to fail... and you are right, it isn't always... | 10:18 |
| rook | right now it is about a 50/50 shot. | 10:18 |
| rook | I typically hold my breath during the deployment. that sees to help | 10:19 |
| rook | :) | 10:19 |
| shadower | rook: respect if you can hold it for so long | 10:19 |
| dtantsur | ROFL | 10:19 |
| *** tremble has joined #tripleo | 10:20 | |
| *** tremble has joined #tripleo | 10:20 | |
| dtantsur | rook, I think you can find the actual error in ironic-conductor logs | 10:20 |
| dtantsur | unfortunately, when ironic reports a failure, nova tears down the node thus destroying all evidence ;) | 10:20 |
| rook | https://gist.github.com/jtaleric/d67ec98aaee0c507dde954978a863dec | 10:21 |
| rook | some of what I am seeing... since I am deploying all the nodes i have in my env it is failing quite often. | 10:21 |
| rook | if I deploy lessthan the number of nodes, my guess is that I will have better luck | 10:21 |
| * rook is also trying to deploy with Ceph (3 nodes) | 10:22 | |
| rook | However, that doesn't seem to be the issue :/ | 10:22 |
| *** chem has quit IRC | 10:22 | |
| *** chem has joined #tripleo | 10:22 | |
| dtantsur | rook, I believe you :) but that does not give any hints about the actual problem ironic encountered | 10:23 |
| rook | let me check the conductor logs. | 10:23 |
| dtantsur | rook, fwiw most often people lack free disk space | 10:23 |
| rook | dtantsur oh, i have plenty, and a NVMe to boot :) | 10:23 |
| rook | resources should be the last of my problems | 10:23 |
| dtantsur | rook, also, IIUC the fix for this instance_uuid problem was merged this morning to Nova | 10:23 |
| dtantsur | soooo you may become a bit more lucky soon :) | 10:24 |
| rook | lol | 10:24 |
| dtantsur | rook, if I understand it right: https://review.openstack.org/#/c/341253/ | 10:25 |
| *** lucasagomes is now known as lucas-relocate | 10:25 | |
| rook | nothing in the conductor logs that is screaming at me :/ | 10:26 |
| dtantsur | suspicious | 10:26 |
| rook | dtantsur yeah, different deployments are experiencing weird behaviors.. Mine is the rescheduling and running out of nodes since the instance_uuid is never being cleaned from ironic. | 10:27 |
| dtantsur | meh | 10:27 |
| rook | dtantsur another lab is seeing when they do a stack delete that neutron ports aren't being cleaned up | 10:27 |
| dtantsur | neutron ports are created by nova too, so will be cool to check nova logs and maybe get someone from that side involved | 10:28 |
| hewbrocca | jaosorior: were you the one who was trying to kill os-cloud-config? | 10:28 |
| derekh | sshnaidm: ok, ready? | 10:29 |
| rook | shadower: lol yeah - i have had plenty of training deploying w/ OOO | 10:29 |
| sshnaidm | derekh, yep | 10:29 |
| jaosorior | hewbrocca: yeah | 10:29 |
| hewbrocca | jaosorior: excellent, would you put it out of my misery already :D | 10:29 |
| jaosorior | hewbrocca: this is the fifth attempt https://review.openstack.org/#/c/359680/ | 10:30 |
| hewbrocca | good god it's like it's undead or something | 10:30 |
| *** numans has joined #tripleo | 10:30 | |
| jaosorior | hewbrocca: it refuses to die somehow | 10:30 |
| dtantsur | os-undead-config | 10:30 |
| derekh | sshnaidm: cool, so we need to do 2 things(maybe more) 1. add a tripleo-user tenants/user | 10:30 |
| dtantsur | Zomby as a Service | 10:31 |
| jaosorior | lol | 10:31 |
| hewbrocca | srsly | 10:31 |
| derekh | sshnaidm: then create a "jumpbox" with a floating IP which you can create envs from | 10:31 |
| hewbrocca | well it's passing CI | 10:31 |
| sshnaidm | derekh, ok | 10:31 |
| hewbrocca | let's merge it quick before it kills again | 10:31 |
| derekh | sshnaidm: are you able to get onto the rh1 undercloud yet ? | 10:31 |
| jaosorior | hewbrocca: that depends on another commit which we unfortunately won't take | 10:31 |
| jaosorior | hewbrocca: now waiting for shardy, he said he could provide a viable alternative | 10:32 |
| sshnaidm | derekh, I'm on 10.1.8.9 | 10:32 |
| sshnaidm | derekh, afaiu it's rh1 undercloud | 10:32 |
| hewbrocca | jaosorior: ahh, right | 10:32 |
| hewbrocca | this was the earlier discussion about global parameters | 10:32 |
| derekh | sshnaidm: yup, it is, you should be the "stack" user for this | 10:33 |
| sshnaidm | derekh, yeah | 10:33 |
| jaosorior | hewbrocca: indeedo | 10:34 |
| derekh | sshnaidm: ok, can you join the screen session there "screen -x" I think | 10:34 |
| sshnaidm | derekh, I'm in | 10:34 |
| derekh | sshnaidm: I'm on tab 4, ctrl-A 4 | 10:35 |
| derekh | crtl+a 4 | 10:35 |
| sshnaidm | derekh, oh.. let me reconnect | 10:35 |
| derekh | sshnaidm: ok | 10:36 |
| jaosorior | derekh: given the OOMs we've been seeing lately. do you think it makes sense to reduce the number of workers in the undercloud? https://review.openstack.org/#/c/360255/ | 10:36 |
| sshnaidm | derekh, I'm in nested screen session :) | 10:36 |
| *** akrivoka has quit IRC | 10:37 | |
| derekh | jaosorior: maybe, but I think there was a reson we needed 2 heat services (heat-engine I think) so you probably shouldn't touch that one | 10:37 |
| *** xuao has quit IRC | 10:37 | |
| derekh | sshnaidm: nice, is it working ok or do you need to sort it out? | 10:38 |
| sshnaidm | derekh, it's ok now | 10:38 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Update authtoken parameters to match recent changes https://review.openstack.org/359401 | 10:38 |
| dtantsur | hmm, my undercloud already has 7.5 GiB eaten (HA, no SSL). I wonder if 8 GiB is enough at all... | 10:38 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: [WIP] Set parameters https://review.openstack.org/360391 | 10:39 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Add 'openstack overcloud plan create' command https://review.openstack.org/359784 | 10:39 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: [WIP] Add `openstack overcloud deploy --plan` https://review.openstack.org/360305 | 10:39 |
| dtantsur | at least probably not without swap | 10:39 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Update the 'overcloud deploy --templates' command to use Mistral https://review.openstack.org/342261 | 10:39 |
| derekh | sshnaidm: ok, so see there are a bunch of rc files, you'll want to source the overcloudrc file to be the admin user on the overcloud | 10:39 |
| jaosorior | derekh: well, I'm reducing them to two | 10:39 |
| sshnaidm | derekh, ok | 10:39 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Make OVS ARP responder feature configurable https://review.openstack.org/360053 | 10:39 |
| openstackgerrit | Merged openstack/os-net-config: Remove discover from test-requirements https://review.openstack.org/345850 | 10:39 |
| openstackgerrit | Merged openstack/python-tripleoclient: Fix 'no nodes found' message in introspect manageable nodes workflow https://review.openstack.org/352783 | 10:39 |
| derekh | jaosorior: ok, arn't they already 2? or is this in anticipation of the bump in flavor james was talking about? | 10:40 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-quickstart: Move OPT_CONFIG after release env file https://review.openstack.org/360417 | 10:40 |
| jaosorior | derekh: I actually thought it was 4 already | 10:40 |
| *** jbadiapa has joined #tripleo | 10:41 | |
| *** jubapa has joined #tripleo | 10:41 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Clean up old functions https://review.openstack.org/359979 | 10:41 |
| derekh | sshnaidm: that script I just catted out "scripts/prepare-ovb-cloud.sh" is the one used to prepare the cloud in the first place, among other things it creates the openstack-nodepool user, so we can copy the command from it | 10:42 |
| derekh | sshnaidm: and change openstack user too tripleo-user | 10:42 |
| derekh | sshnaidm: to create the new tripleo-user tenant, this is the same as what we had on rh2 and is the tenant you will use | 10:43 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Migrate to using osc-lib https://review.openstack.org/335460 | 10:45 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Migrate to using osc-lib https://review.openstack.org/335460 | 10:46 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Better namespace the list of TripleO actions https://review.openstack.org/357682 | 10:47 |
| sshnaidm | derekh, is it ok there? | 10:47 |
| derekh | sshnaidm: looks good, go for it | 10:48 |
| jaosorior | shardy: by the way, also if you have time. Can you check this out? https://review.openstack.org/#/c/356416/ It tries to get FQDNs for the different VIPs in the endpoint map. Before you could only add the public one there. | 10:48 |
| *** akrivoka has joined #tripleo | 10:49 | |
| sshnaidm | derekh, ok | 10:51 |
| *** milan has joined #tripleo | 10:52 | |
| derekh | sshnaidm: ok, next we should probably create a tripleo-userrc file, the password for tripleo-user is in "os-asserted-users" | 10:52 |
| *** ccamacho is now known as ccamacho|afk | 10:53 | |
| *** mpsairam has quit IRC | 10:55 | |
| sshnaidm | derekh, done | 10:56 |
| jistr | dtantsur: btw did you get report the problem you hit with stack-update earlier? I just hit the very same problem, and i haven't changed a thing in templates/puppet between the stack-create and stack-update. There must be something broken indeed. | 10:57 |
| jistr | s/get report/get to report/ | 10:57 |
| jaosorior | shadower: by the way, I think the hiera entries need to exist, else these lookups will fail https://review.openstack.org/#/c/359645/4/puppet/services/haproxy.yaml | 10:57 |
| jaosorior | I mean, shardy | 10:57 |
| dtantsur | jistr, no, I didn't have a chance, because I had to tear down the environment. I suspect that OOM may be related | 10:57 |
| dtantsur | jistr, just building an HA overcloud took 7.5 GiB our of 8 GiB RAM for me | 10:58 |
| dtantsur | but this is just a guess | 10:58 |
| jaosorior | shardy: seems to me that with my approach in that patch it would also fail. Probably could do it with global_settings. But not sure if we want to go that way still | 10:58 |
| shardy | jaosorior: won't they just return nil? | 10:58 |
| jistr | dtantsur: ah perhaps OOM is an issue in my env too, i have an 8 GiB undercloud too. Thanks for the hint :) | 10:58 |
| dtantsur | worth checking, at least... maybe we should start recommending 12 GiB at least | 10:59 |
| rook | dtantsur so, got past those issues only to find a new one, ever seen : overcloud-ComputeNodesPostDeployment-7eeeeaaaexxq-ComputeOvercloudServicesDeployment_Step4-srfeswoug6ps : which on the os-collect-config i see, https://gist.github.com/jtaleric/28955b67f8f046bfbf7d80a9151d33f6 | 11:00 |
| jaosorior | shardy: I need to verify. But at least according to the documentation it should throw an error https://docs.puppet.com/hiera/3.2/variables.html#the-hiera-lookup-function | 11:00 |
| derekh | sshnaidm: ok, next we need to take the ssh keys from the tripleo-user tentant on rh2 (the default keys) and mirror them to rh1 | 11:00 |
| dtantsur | rook, no, sorry, never tired ceph | 11:00 |
| * dtantsur has enough fun with ironic in overcloud :) | 11:01 | |
| rook | hm, is this issue due to ceph? | 11:01 |
| shardy | jaosorior: I think it's saying the result must be a string, not that is must exist | 11:01 |
| jaosorior | shardy: let me test that out | 11:01 |
| shardy | maybe it works differently when interpolating the result, but on the CLI it just gives nil | 11:01 |
| dtantsur | rook, well, your paste mentions ceph, this is why I assumed it's related | 11:02 |
| rook | rgr | 11:03 |
| rook | sad trombone | 11:03 |
| jaosorior | shardy: nevermind. It's alright. Just tried it out locally | 11:04 |
| jistr | rook, gfidente: maybe the missing ceph::profile::params::client_keys could be due to outdated t-h-t? | 11:04 |
| jaosorior | shardy: misunderstood the documentation then | 11:04 |
| rook | jistr: maybe? I am using the downstream product. | 11:04 |
| *** ccamacho|afk is now known as ccamacho | 11:05 | |
| jistr | rook: can you see this in t-h-t/puppet/services/ceph-base.yaml? https://github.com/openstack/tripleo-heat-templates/blob/6c537d2005b8cb5b25f63722376fca08471987ac/puppet/services/ceph-base.yaml#L80 | 11:06 |
| sshnaidm | derekh, it's either extremely slow or I can't connect to rh2 | 11:08 |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 11:10 |
| sshnaidm | derekh, can you try please? | 11:10 |
| *** lucas-relocate is now known as lucas-hungry | 11:13 | |
| derekh | sshnaidm: got it, back in a bit, gotta grab lunch http://paste.openstack.org/show/563362/ | 11:13 |
| derekh | sshnaidm: next you'll need to create a small vm as a jump host and give it a floating ip | 11:14 |
| derekh | back soon | 11:14 |
| sshnaidm | derekh, ok | 11:14 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates: Availability monitoring agents support https://review.openstack.org/254788 | 11:16 |
| *** pkovar has quit IRC | 11:20 | |
| *** ramishra has quit IRC | 11:20 | |
| *** rodrigods has quit IRC | 11:21 | |
| *** rodrigods has joined #tripleo | 11:21 | |
| *** ramishra has joined #tripleo | 11:22 | |
| karthiks_afk | shardy, sorry i missed out the second comment of https://review.openstack.org/#/c/329415/8/puppet/services/nova-compute.yaml@64. | 11:23 |
| *** karthiks_afk is now known as karthiks | 11:23 | |
| shardy | karthiks: Hi, so my question was does nova::compute::pci_passthrough expect a json serialized list of maps? | 11:24 |
| shardy | if so then my suggestion will enable exposing a json parameter to the user, and pass a json string to puppet | 11:25 |
| shardy | which I think may be a bit easier for folks to use | 11:25 |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-heat-templates: Ironic: add missing haproxy and firewall configuration https://review.openstack.org/356539 | 11:26 |
| *** bfournie has quit IRC | 11:27 | |
| karthiks | shardy, will try as per your suggestion. | 11:29 |
| *** matbu is now known as matbu|lch | 11:29 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Move Swift hiera settings into composable services https://review.openstack.org/360058 | 11:30 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Mv keystone, mongo, manila out of controller.yaml https://review.openstack.org/360060 | 11:30 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Move glance/heat hiera settings to services https://review.openstack.org/360029 | 11:30 |
| *** zoli|lunch is now known as zoli|trng | 11:36 | |
| jistr | gfidente: +2'd mon upgarde and left an optional comment on osd upgrade which would be nice to have but not a blocker imo | 11:37 |
| *** jcoufal has joined #tripleo | 11:37 | |
| *** jcoufal has quit IRC | 11:39 | |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates: Rework the pacemaker_common_functions for M..N upgrades https://review.openstack.org/321027 | 11:39 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates: WIP Add a function to upgrade from full HA to NG HA https://review.openstack.org/358626 | 11:39 |
| gfidente | jistr, ack, thanks | 11:47 |
| *** adarazs is now known as adarazs_lunch | 11:48 | |
| gfidente | d0ugal, I got undercloud nodes registration with mistral logging it's trying to get to zaqar on 8888 | 11:48 |
| gfidente | but it's configured to use the websockets | 11:48 |
| gfidente | any clue why is that, ever seen it? | 11:48 |
| d0ugal | gfidente: no, weird - that's new to me | 11:48 |
| gfidente | so the nodes registration just hangs | 11:48 |
| d0ugal | gfidente: hmm, weird. we should maybe add a sensible timeout for each command. | 11:49 |
| d0ugal | gfidente: what does your keystone endpoint list have for messaging-websocket? | 11:50 |
| gfidente | d0ugal, ws://192.0.2.1:9000/ | 11:51 |
| d0ugal | hrm, weird. | 11:51 |
| gfidente | then I have the standard zaqar endpoint too | 11:51 |
| *** jaosorior has quit IRC | 11:52 | |
| d0ugal | gfidente: I don't have any good ideas, sorry - dprince set this bit up | 11:52 |
| *** jbadiapa has quit IRC | 11:52 | |
| *** jaosorior has joined #tripleo | 11:52 | |
| d0ugal | gfidente: It's hard to see how this could go wrong: https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/plugin.py#L72 | 11:52 |
| *** jubapa has quit IRC | 11:53 | |
| gfidente | right I added some logging and it goes there | 11:53 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Always deploy HAProxy https://review.openstack.org/360368 | 11:54 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Make HAProxy terminate OpenStack internal/admin endpoints https://review.openstack.org/360366 | 11:54 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Enable TLS for Zaqar's websocket endpoint https://review.openstack.org/360350 | 11:54 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: [WIP] Set parameters https://review.openstack.org/360391 | 11:54 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Add 'openstack overcloud plan create' command https://review.openstack.org/359784 | 11:54 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: [WIP] Add `openstack overcloud deploy --plan` https://review.openstack.org/360305 | 11:54 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Update the 'overcloud deploy --templates' command to use Mistral https://review.openstack.org/342261 | 11:54 |
| *** dprince has joined #tripleo | 11:55 | |
| *** vijaykc4 has joined #tripleo | 11:57 | |
| *** masco has quit IRC | 11:58 | |
| *** vijaykc4 has joined #tripleo | 11:58 | |
| *** jpena is now known as jpena|lunch | 12:04 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: [WIP] Set parameters https://review.openstack.org/360391 | 12:04 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: [WIP] Add `openstack overcloud deploy --plan` https://review.openstack.org/360305 | 12:04 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Update the 'overcloud deploy --templates' command to use Mistral https://review.openstack.org/342261 | 12:04 |
| *** bfournie has joined #tripleo | 12:04 | |
| *** pkovar has joined #tripleo | 12:04 | |
| *** xuao has joined #tripleo | 12:04 | |
| *** pradk has joined #tripleo | 12:09 | |
| *** xuao has quit IRC | 12:09 | |
| *** ooolpbot has joined #tripleo | 12:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 12:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 12:10 |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 12:10 |
| *** ooolpbot has quit IRC | 12:10 | |
| jistr | dtantsur: FYI just in case it's useful -- it's indeed an actual bug regardless of OOM or not probably, zaneb's fix got merged into heat a few hours ago https://bugs.launchpad.net/tripleo/+bug/1616550 | 12:10 |
| openstack | Launchpad bug 1616550 in tripleo "Overcloud update is broken: UPDATE_FAILED NotFound_Remote: resources[0]: Software config with id not found" [High,Confirmed] | 12:10 |
| *** jprovazn has quit IRC | 12:10 | |
| dtantsur | jistr, cool! I'll give it a try once I unbreak the whole ironic thing >_< | 12:11 |
| zaneb | lol | 12:11 |
| hewbrocca | crazy cool | 12:12 |
| hewbrocca | heh | 12:13 |
| hewbrocca | looks like more of a TripleO bug than a Heat bug to me | 12:13 |
| hewbrocca | but I guess it depends on where you stand | 12:13 |
| openstackgerrit | Jiri Stransky proposed openstack/puppet-tripleo: Write restart flags to restart services only when necessary https://review.openstack.org/358510 | 12:14 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates: Restart only services that need it https://review.openstack.org/358511 | 12:15 |
| *** lucas-hungry is now known as lucasagomes | 12:15 | |
| slagle | "shattered into a thousand tiny islands" is the name of my emo punk sitar band | 12:16 |
| hewbrocca | ROFL | 12:16 |
| *** adarazs_lunch is now known as adarazs | 12:17 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Fix use of snmp service https://review.openstack.org/360062 | 12:17 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Move ceilometer api to run under apache wsgi https://review.openstack.org/293642 | 12:18 |
| *** mburned_out is now known as mburned | 12:19 | |
| *** rhefner has quit IRC | 12:19 | |
| *** colonwq has quit IRC | 12:19 | |
| *** colonwq has joined #tripleo | 12:20 | |
| *** matbu|lch is now known as matbu | 12:20 | |
| *** shardy has quit IRC | 12:21 | |
| *** cdearborn has joined #tripleo | 12:21 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Move Swift hiera settings into composable services https://review.openstack.org/360058 | 12:22 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Mv keystone, mongo, manila out of controller.yaml https://review.openstack.org/360060 | 12:22 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Move glance/heat hiera settings to services https://review.openstack.org/360029 | 12:22 |
| derekh | sshnaidm: back, how goes it? | 12:22 |
| dprince | jaosorior, gfidente: small fixup in the heat-engine.yaml template in the last patch above. | 12:22 |
| matbu | is somoeone can review this (and +2 btw :) ) plz https://review.openstack.org/#/c/353996/ | 12:22 |
| *** shardy has joined #tripleo | 12:22 | |
| * dprince forget to set 2016-10-14 in his template | 12:23 | |
| *** mwhahaha_ has joined #tripleo | 12:23 | |
| dprince | How would people feel if we switched the templates to '2016-10-14' everywhere? | 12:23 |
| openstackgerrit | Ana Krivokapic proposed openstack/tripleo-heat-templates: Add constraint for ControllerCount https://review.openstack.org/342272 | 12:23 |
| *** Tristitia has quit IRC | 12:23 | |
| *** mwhahaha has quit IRC | 12:23 | |
| *** mwhahaha_ is now known as mwhahaha | 12:24 | |
| matbu | jaosorior: thx :) | 12:24 |
| *** bootsha has quit IRC | 12:24 | |
| *** jaosorior is now known as jaosorior_away | 12:25 | |
| sshnaidm | derekh, how do I create the jumphost? | 12:25 |
| *** Tristit1a has joined #tripleo | 12:26 | |
| derekh | sshnaidm: you just want to create a normal vm with nova boot ... | 12:27 |
| openstackgerrit | Giulio Fidente proposed openstack/python-tripleoclient: Use heatclient poll_for_events in wait_for_stack_ready https://review.openstack.org/360141 | 12:27 |
| sshnaidm | derekh, from CentOS-7-x86_64-GenericCloud ? | 12:27 |
| derekh | sshnaidm: yes, iirc you wont need to specifiy any networks as it should get the default one | 12:28 |
| *** jdob has joined #tripleo | 12:28 | |
| *** bootsha has joined #tripleo | 12:28 | |
| sshnaidm | derekh, m1.small? | 12:29 |
| derekh | sshnaidm: yup, that will be plenty | 12:29 |
| sshnaidm | ok | 12:30 |
| *** weshay_bbiab is now known as weshay | 12:30 | |
| *** ccamacho is now known as ccamacho|lunch | 12:33 | |
| sshnaidm | derekh, ready | 12:33 |
| *** thrash|g0ne is now known as thrash | 12:33 | |
| derekh | sshnaidm: now | 12:33 |
| derekh | nova floating-ip-create | 12:33 |
| derekh | nova floating-ip-associate undercloud FIP | 12:33 |
| shardy | dprince: +1 on aligning all templates with newton HOT | 12:34 |
| dprince | shardy: ack. I might throw that up. I've caused myself CI failures more than once now using new functions but not updating the version | 12:35 |
| EmilienM | good morning | 12:37 |
| *** morazi has joined #tripleo | 12:37 | |
| *** rlandy has joined #tripleo | 12:37 | |
| Jokke_ | dprince: GMorning. I think I managed to address your wish as well on https://review.openstack.org/#/c/354047/7 It's hitting on the nonha test bug though so no rush | 12:37 |
| slagle | gfidente: do you think we should wait for your tripleoclient patch or go ahead and make the CI changes to give the undercloud more resources? | 12:37 |
| derekh | sshnaidm: ok, whats left is basically stuff to make the whats documented here work - https://etherpad.openstack.org/p/tripleo-ci-devenvs | 12:38 |
| dprince | Jokke_: boom, +2 | 12:39 |
| derekh | sshnaidm: for example, if you do a nova list you wont see the image that infra are using, | 12:39 |
| dprince | Jokke_: thanks | 12:39 |
| derekh | sshnaidm: on rh2 I added a cron job to make it public so people could use it | 12:39 |
| derekh | sshnaidm: http://paste.openstack.org/show/563355/ | 12:40 |
| Jokke_ | :D | 12:40 |
| *** jcoufal has joined #tripleo | 12:40 | |
| derekh | sshnaidm: so we'll need that on rh1 also, infra upload a new image daily, its private the cron job makes it public | 12:40 |
| sshnaidm | derekh, ok, so to do the same here? | 12:40 |
| derekh | sshnaidm: yup | 12:41 |
| derekh | sshnaidm: that image is what you'll use for your undercloud | 12:41 |
| sshnaidm | derekh, btw, where is this image from? | 12:41 |
| derekh | sshnaidm: infra build it and upload a new one daily | 12:41 |
| derekh | sshnaidm: from nodepool | 12:41 |
| sshnaidm | derekh, infra - do you mean openstack-infra? | 12:41 |
| *** rhallisey has joined #tripleo | 12:41 | |
| derekh | sshnaidm: yup | 12:42 |
| sshnaidm | derekh, and they build centos image | 12:42 |
| sshnaidm | that we use for our undercloud | 12:42 |
| *** Tristit1a is now known as Tristitia | 12:42 | |
| derekh | sshnaidm: yes, correct, | 12:42 |
| sshnaidm | derekh, do you know if I can download it via web..? | 12:43 |
| EmilienM | can someone approve https://review.openstack.org/#/c/359400/ please? | 12:43 |
| derekh | sshnaidm: the image has all of the git repos in it for example | 12:43 |
| derekh | sshnaidm: once you make it public with that cron job you can "glance download .." it, I don't think they have it anywhere for download | 12:43 |
| derekh | sshnaidm: brb, gotta restart something | 12:44 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Upgrade ceph-osd https://review.openstack.org/357312 | 12:44 |
| derekh | *everything | 12:44 |
| *** derekh has quit IRC | 12:44 | |
| gfidente | slagle, honestly no, it doesn't seem to the cause | 12:46 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates: Add fluentd client service https://review.openstack.org/353506 | 12:46 |
| gfidente | slagle, it doesn't look like repeated runs of get_stack themselves leak, though they use a little memory | 12:46 |
| slagle | ok, i'm going to go ahead with the changes then | 12:46 |
| *** jcoufal has quit IRC | 12:47 | |
| *** derekh has joined #tripleo | 12:47 | |
| *** apetrich has quit IRC | 12:48 | |
| *** jprovazn has joined #tripleo | 12:49 | |
| pradk | can i get some re +2's on this ... just rebased https://review.openstack.org/#/c/293642/ | 12:51 |
| *** apetrich has joined #tripleo | 12:52 | |
| *** bootsha has quit IRC | 12:54 | |
| *** mgagne_ is now known as mgagne | 12:54 | |
| *** bootsha has joined #tripleo | 12:55 | |
| *** [1]cdearborn has joined #tripleo | 12:55 | |
| *** jpena|lunch is now known as jpena | 12:56 | |
| *** asbishop has joined #tripleo | 12:56 | |
| *** sanjayu|AFK is now known as sanjayu_ | 12:57 | |
| *** morazi has quit IRC | 12:57 | |
| rook | Nooo! | 12:57 |
| rook | The dreaded dracut! | 12:57 |
| rook | rebooted my undercloud, and now it is stuck in dracut :*( | 12:57 |
| *** tvignaud has quit IRC | 12:58 | |
| hewbrocca | rook, slagle did we fix that horrible LVM bug in undercloud config? | 12:59 |
| rook | this is newton, so not yet... | 12:59 |
| hewbrocca | GRRRRRRRRRRR | 12:59 |
| hewbrocca | jpeeler: ^^^ you were looking at this, right? | 12:59 |
| slagle | hewbrocca: which bug? link me | 12:59 |
| *** ccamacho|lunch is now known as ccamacho | 12:59 | |
| * hewbrocca digs | 12:59 | |
| *** jrist has quit IRC | 13:00 | |
| *** [2]cdearborn has joined #tripleo | 13:00 | |
| sshnaidm | derekh, something broke in screen session there.. | 13:00 |
| *** bootsha has quit IRC | 13:00 | |
| derekh | sshnaidm: opps, maybe I closed the tab by accident | 13:00 |
| slagle | gfidente: fwiw i tested your patch in my environment and it fails the same way | 13:01 |
| gfidente | slagle, which patch? | 13:01 |
| hewbrocca | https://bugzilla.redhat.com/show_bug.cgi?id=1323024 | 13:01 |
| openstack | bugzilla.redhat.com bug 1323024 in instack-undercloud "Undercloud installation disables LVM activation, which causes boot failure on systems with more than root and swap LVs" [Unspecified,New] - Assigned to jslagle | 13:01 |
| sshnaidm | derekh, and who is "--owner 462104cc..." in your script? | 13:01 |
| slagle | gfidente: python-tripleoclient | 13:01 |
| hewbrocca | slagle: that's the one | 13:01 |
| gfidente | slagle, ah yeah I am on 8G so I don't see it failing either way | 13:02 |
| slagle | lol, you're killing me | 13:02 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart: Update configuration docs https://review.openstack.org/356700 | 13:02 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart: Update section header indicators https://review.openstack.org/360522 | 13:02 |
| gfidente | slagle, I was looking at the memory consumed by heat with heatclient poll_ and tripleoclient poll_ | 13:02 |
| *** jbadiapa has joined #tripleo | 13:02 | |
| *** jubapa has joined #tripleo | 13:02 | |
| derekh | sshnaidm: ahh, thats probably the openstack-nodepool user, so that needs to change | 13:02 |
| gfidente | slagle, I think it's worth using the version in the heatclient but it's not the cause of the memory issues | 13:02 |
| slagle | hewbrocca: in the bug, jpeeler says it's fixed. the patch landed in newton and is not backported to any other releaes | 13:03 |
| slagle | gfidente: ok, wfm. i've made the flavor changes in rh1 btw. i'll reply to the thread | 13:05 |
| sshnaidm | derekh, do you have a screen session attached? | 13:06 |
| slagle | pabelanger: has the nodepool restart happened yet to pick up the fix for deleting the stale instances? | 13:06 |
| derekh | sshnaidm: I've just reattached now, I think I closed it by accident sorry, I'm back in tab 4 | 13:07 |
| *** morazi has joined #tripleo | 13:07 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Upgrade ceph-osd https://review.openstack.org/357312 | 13:08 |
| *** paramite has joined #tripleo | 13:09 | |
| *** cdearborn has quit IRC | 13:09 | |
| *** julim_ has joined #tripleo | 13:09 | |
| hewbrocca | slagle: huh, maybe we should move the bug to... some other state | 13:10 |
| *** ooolpbot has joined #tripleo | 13:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 13:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 13:10 |
| *** ooolpbot has quit IRC | 13:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 13:10 |
| sshnaidm | derekh, hmm.. which rc file to source for images? there are plenty of them.. | 13:10 |
| *** skramaja has quit IRC | 13:10 | |
| hewbrocca | rook: do you believe you are suffering from https://bugzilla.redhat.com/show_bug.cgi?id=1323024 ? | 13:10 |
| openstack | bugzilla.redhat.com bug 1323024 in instack-undercloud "Undercloud installation disables LVM activation, which causes boot failure on systems with more than root and swap LVs" [Unspecified,New] - Assigned to jslagle | 13:10 |
| *** skramaja has joined #tripleo | 13:11 | |
| *** tvignaud has joined #tripleo | 13:11 | |
| slagle | hewbrocca: probably. it is not on my radar fwiw | 13:11 |
| slagle | (osp8 bugs) | 13:11 |
| *** jeckersb_gone is now known as jeckersb | 13:11 | |
| slagle | looking at bugs is pretty far down the list of "stuff" | 13:12 |
| *** jrist has joined #tripleo | 13:13 | |
| *** jrist has joined #tripleo | 13:13 | |
| rook | hewbrocca it could very well be this. | 13:13 |
| *** [1]cdearborn has quit IRC | 13:14 | |
| *** julim_ has quit IRC | 13:15 | |
| *** tzumainn has joined #tripleo | 13:16 | |
| derekh | sshnaidm: the script sources /home/stack/overcloudrc , is that what you mean? | 13:16 |
| *** [2]cdearborn has quit IRC | 13:17 | |
| *** julim_ has joined #tripleo | 13:17 | |
| sshnaidm | derekh, yeah | 13:18 |
| sshnaidm | derekh, should it be nodepoolrc in our case? | 13:18 |
| openstackgerrit | Martin André proposed openstack/puppet-tripleo: [WIP] UI profile for tripleo https://review.openstack.org/354630 | 13:18 |
| jpeeler | hewbrocca: should the lvm bug be in post or modified? | 13:19 |
| mandre | EmilienM: ^^ | 13:19 |
| sshnaidm | derekh, because with overcloudrc I don't see any centos templates | 13:19 |
| derekh | sshnaidm: taking a look | 13:21 |
| openstackgerrit | Ana Krivokapic proposed openstack/python-tripleoclient: Fix unbracket_ipv6() https://review.openstack.org/360533 | 13:21 |
| *** links has quit IRC | 13:22 | |
| sshnaidm | derekh, I see | 13:22 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Mv ceilo, aodh, and gnocchi out of controller.yaml https://review.openstack.org/360536 | 13:23 |
| hewbrocca | rook: there is a workaround described in the bug | 13:25 |
| hewbrocca | jpeeler: check with mburned when he gets on | 13:25 |
| beagles | shardy, dprince: a question about order of operations, can I reference values that are defined in the resource_registry in an environment file used in the deploy command? | 13:25 |
| dprince | beagles: I think we guarantee that via our environment ordering | 13:26 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Enable gnocchi systemd services https://review.openstack.org/356111 | 13:26 |
| dprince | beagles: so I'd say yes | 13:26 |
| beagles | shardy, dprince: e.g. in neutron-ovs-dvr if OS::TripleO::Compute::Net::SoftwareConfig has to match OS::TripleO::Controller:Net::SoftwareConfig, can I just: | 13:26 |
| beagles | OS::TripleO::Compute::Net::SoftwareConfig: OS::TripleO::Controller::Net::SoftwareConfig ? | 13:27 |
| beagles | cuz that'd be kinda cool | 13:27 |
| dprince | beagles: I think so | 13:27 |
| beagles | I'll try it a bit later and see what happens, thanks | 13:27 |
| beagles | dprince, ^^ | 13:27 |
| beagles | nice | 13:27 |
| derekh | sshnaidm: I think I know the problem | 13:28 |
| derekh | sshnaidm: try now | 13:28 |
| saneax | Hi Folks, these THT's are must have for sr-iov and they already have/had +2 and +1's | 13:29 |
| rook | hewbrocca: i don't have that line in the lvm.conf | 13:29 |
| rook | so my guess is,this is different? | 13:29 |
| saneax | https://review.openstack.org/#/c/332191/ and https://review.openstack.org/#/c/332190/ | 13:29 |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-docs: Switch docs to use tripleo-build-images https://review.openstack.org/337669 | 13:29 |
| sshnaidm | derekh, API ver related changes? | 13:30 |
| rook | hewbrocca: i also have no /home | 13:30 |
| derekh | sshnaidm: yup, that --visibility flag was added in a more recent version then what we had set | 13:30 |
| derekh | sshnaidm: I was left scratching my head for a long time on that one before | 13:31 |
| rook | https://gist.github.com/jtaleric/d322ea310bb12babcd9631554c3203f0 hewbrocca | 13:31 |
| *** lblanchard has joined #tripleo | 13:31 | |
| sshnaidm | derekh, ok, great | 13:31 |
| sshnaidm | derekh, so what is now? creating instackenv.json? | 13:31 |
| hewbrocca | poor rook, all this and he's living out of his car too | 13:33 |
| * rook lives out of a car?! | 13:33 | |
| rook | ahh | 13:33 |
| rook | lol | 13:33 |
| rook | i have no /home | 13:33 |
| rook | jesus I am slow today | 13:33 |
| derekh | sshnaidm: you should be able to start on line 16 of the etherpad, some packages might need to be installed on the jump host and you'll need to git clone "tripleo-ci" into /home/centos | 13:33 |
| hewbrocca | rook: I guess if you don't have the suspect line in lvm.conf then this is a different issue | 13:34 |
| sshnaidm | derekh, ok | 13:34 |
| openstackgerrit | Mathieu Mitchell proposed openstack/diskimage-builder: Generate and use upper-constraints for ironic-agent https://review.openstack.org/360547 | 13:35 |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-ui: Use Mistral/Zaqar for folder-based plan-creation https://review.openstack.org/360550 | 13:35 |
| derekh | sshnaidm: once thats done, i.e. you can use the jump host to create OVB envs, then eveything afterwards (undercloud, baremetal envs, instack.json etc...) is disposable, everytime you need a new env you just delete the undercloud and env you had and create a new one | 13:35 |
| *** maeca1 has joined #tripleo | 13:36 | |
| derekh | sshnaidm: the only thing that remains long term is the jumphost | 13:36 |
| rook | hewbrocca: seems that way... | 13:36 |
| sshnaidm | derekh, ok | 13:36 |
| derekh | sshnaidm: so if for example next week ci jobs start failing because something is broken on the undercloud and we can't reproduce it, you can create an undercloud with the latest image from infra and take a look | 13:37 |
| sshnaidm | derekh, yeah, I was doing this with rh2, it's a huge help | 13:37 |
| derekh | sshnaidm: yup | 13:38 |
| *** fultonj has joined #tripleo | 13:39 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: WIP: Ceilometer Wsgi Mitaka->Newton upgrades https://review.openstack.org/360004 | 13:39 |
| shardy | beagles: Hey, yes that should work I think | 13:40 |
| rook | hewbrocca: ok, found the issue | 13:40 |
| rook | someone fat fingered a regex | 13:40 |
| rook | in the /etc/lvm/lvm.conf | 13:40 |
| beagles | shardy, thx | 13:41 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Fix use of snmp service https://review.openstack.org/360062 | 13:42 |
| hewbrocca | O NOES | 13:42 |
| hewbrocca | you guys, rook can move out of his car now | 13:42 |
| thrash | s/car/home/ | 13:43 |
| thrash | too soon? | 13:43 |
| *** tobias-fiberdata has joined #tripleo | 13:43 | |
| rook | ugh | 13:44 |
| sshnaidm | derekh, one more question, what is --nic net-id=7b866.. in the doc? which net is it? | 13:44 |
| *** limao has joined #tripleo | 13:45 | |
| d0ugal | thrash: btw, I never figured out that CI failure :( I got this change in tho' so hopefully I will know soon. It's a weird one. https://review.openstack.org/#/c/359937/ | 13:45 |
| *** jobewan has joined #tripleo | 13:45 | |
| thrash | d0ugal: nice | 13:45 |
| *** tobias_fiberdata has quit IRC | 13:46 | |
| thrash | d0ugal: was actually just about to look at the latest failure. | 13:46 |
| d0ugal | thrash: no idea how long it will take to get through to CI | 13:46 |
| *** jobewan is now known as jobewan_away | 13:46 | |
| *** jobewan_away is now known as jobewan | 13:46 | |
| thrash | d0ugal: interesting... there are additional commands that failed now. | 13:46 |
| *** limao_ has joined #tripleo | 13:46 | |
| d0ugal | thrash: weridly, one of jpich's patches landed before mine and I rebased on it, now it reports that command as failing too | 13:46 |
| d0ugal | thrash: Yeah, same file. I wonder if I have introduced a bad character somewhere? | 13:46 |
| thrash | d0ugal: maybe... in setup.cfg? | 13:47 |
| d0ugal | thrash: I was thinking the Python file - the two commands in that file claim to fail | 13:47 |
| jaosorior_away | EmilienM: got a +A for https://review.openstack.org/#/c/359604/ ? :D there was a merge conflict when it was merging so I had to fix that | 13:47 |
| thrash | d0ugal: but also overcloud_deploy | 13:47 |
| d0ugal | thrash: oh really? I didn't see that. | 13:47 |
| *** jaosorior_away is now known as jaosorior | 13:48 | |
| *** zoli|trng is now known as zoli_gone-proxy | 13:48 | |
| jaosorior | EmilienM: thanks dude | 13:48 |
| thrash | d0ugal: 4th to last line in the console log | 13:48 |
| jaosorior | EmilienM: and, when you get time, can you take a look at this? https://review.openstack.org/#/c/356416/ | 13:48 |
| d0ugal | thrash: oh yeah, weird, I was looking at the first patch in that series | 13:48 |
| d0ugal | so it gets worse as you go on >_< | 13:48 |
| thrash | d0ugal: apparently. :) | 13:48 |
| thrash | d0ugal: but that means it may be introduced in the first one. | 13:49 |
| d0ugal | thrash: Yup, that would make sense. | 13:49 |
| *** limao has quit IRC | 13:49 | |
| d0ugal | thrash: I need to grab some lunch before it gets closer to dinner time, back in a bit | 13:50 |
| jaosorior | EmilienM: got some reviews that need checking out? | 13:50 |
| thrash | d0ugal: go for it... I'll take a look. Fresh eyes and all. | 13:50 |
| jistr | jaosorior: https://review.openstack.org/#/c/356457/ | 13:50 |
| EmilienM | jaosorior: not now, thanks. I'll look your patch | 13:50 |
| jistr | jaosorior: sorry couldn't resist :)) | 13:50 |
| jaosorior | jistr: no biggie. Looking | 13:50 |
| jistr | jaosorior: thanks :) | 13:51 |
| sshnaidm | do we have a dive today? | 13:52 |
| *** cdearborn has joined #tripleo | 13:52 | |
| *** rhallisey has quit IRC | 13:54 | |
| jaosorior | jistr: I thought we weren't symlinking anymore. Since we're using sepparate packages for the puppet modules nowadays, no? | 13:55 |
| rook | hewbrocca jpeeler This is the line that caused me pain in /etc/lvm/lvm.conf -- global_filter = "[ r|^/dev/disk/by-path/ip.*iscsi.*\.org\.openstack:.*| ]" | 13:56 |
| jistr | jaosorior: yea we're using either packages in /usr or modules from source in /opt, but we're still symlinking from /etc. Would be nice if we got rid of symlinking eventually i think. | 13:56 |
| jistr | jaosorior: the change preserves priority in the sense that whatever is in /etc is still the highest priority | 13:56 |
| *** abehl has quit IRC | 13:56 | |
| *** rhallisey has joined #tripleo | 13:57 | |
| *** ebalduf has joined #tripleo | 13:57 | |
| jpeeler | rook: well that's not good... | 13:58 |
| jaosorior | dprince: Zaqar's websocket endpoint TLS https://review.openstack.org/#/c/360350/ :D | 13:58 |
| sshnaidm | slagle, hi, do you know maybe why rh2 is not accessible? do you work on it? | 13:58 |
| *** maeca1 has quit IRC | 13:58 | |
| trozet | dprince: thanks for reviewing 355147, I had a couple of questions about it | 13:59 |
| dprince | jaosorior: ack | 14:00 |
| dprince | trozet: whats up? | 14:01 |
| *** NachoDuck has quit IRC | 14:01 | |
| slagle | sshnaidm: it's accessible for me | 14:01 |
| slagle | sshnaidm: what issue are you seeing? | 14:02 |
| trozet | dprince: so in the service opendaylight.yaml, am I able to reference OpenDaylightApiNetwork, even though it is not defined int hat file? I added it to network/service_net_map->ServiceNetMapDefaults, is that OK? | 14:02 |
| dprince | trozet: exactly | 14:02 |
| dprince | trozet: if you have a look in the puppet/services/database/mysql.yaml file you'll see an example | 14:03 |
| dprince | trozet: there is a nice comment above it to explain the magic that happens and we are putting that comment mostly everywhere | 14:03 |
| *** NachoDuck has joined #tripleo | 14:04 | |
| sshnaidm | slagle, I can't run "nova list" using regular credentials, neither derekh does | 14:04 |
| trozet | dprince: OK, cool. I updated a new patch set last night, but it seems nonHA gate just fails all the time... | 14:04 |
| *** maeca1 has joined #tripleo | 14:04 | |
| dprince | trozet: yep, there is an LP on that and a mailing list thread about the instability of the nonha job. | 14:05 |
| trozet | dprince: and jenkins still +1s...so is that OK? Or do i need to just keep rechecking? | 14:05 |
| dprince | trozet: nonha uses SSL and that apparently consumes more RAM | 14:05 |
| dprince | trozet: I'll review it again, and if it is good recheck it | 14:05 |
| derekh | slagle: from the outside world http://paste.openstack.org/show/563388/ | 14:06 |
| *** limao_ has quit IRC | 14:06 | |
| trozet | dprince: thanks! I also keep rechecking 335173 (the corresponding puppet changes), but random failures occur in puppet syntax that have nothign to do with my patch, like wrong version of ruby | 14:06 |
| *** limao has joined #tripleo | 14:06 | |
| trozet | dprince: and then if i can get it to pass all the initial jobs, it usually will fail on the nonha | 14:06 |
| mburned | jpeeler: you had a question? | 14:08 |
| dprince | trozet: +2, and I've just rechecked it | 14:08 |
| *** jaosorior has quit IRC | 14:08 | |
| mburned | jpeeler: very intermittent today, but i'll answer as i can | 14:08 |
| trozet | dprince: thanks | 14:08 |
| jpeeler | mburned: i was going to ask what state a bugzilla bug should be in with an upstream fix, though unfortunately i see it's not ready anyway | 14:09 |
| *** ooolpbot has joined #tripleo | 14:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 14:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 14:10 |
| *** ooolpbot has quit IRC | 14:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 14:10 |
| mburned | jpeeler: generally on_dev when the patch is in the review process | 14:10 |
| mburned | jpeeler: and POST when it merges | 14:10 |
| shardy | jtomasek: Hey, if you get a moment, can you pls check https://review.openstack.org/#/c/356240/ - I'd like to get some more concrete examples of where allowing parameters to exist in multiple parameter_groups would help building a UI | 14:11 |
| openstackgerrit | Giulio Fidente proposed openstack/python-tripleoclient: Update overcloud passwords on update command https://review.openstack.org/338213 | 14:11 |
| jtomasek | shardy: will do | 14:11 |
| mburned | jpeeler: post when merged on the right branch depending on the bug, so if it's mitaka, then on stable/mitaka, if it's newton, then master | 14:11 |
| thrash | d0ugal: from tripleo_common.utils import tarball\nImportError: cannot import name tarball | 14:12 |
| thrash | d0ugal: File "/home/bcrochet/workspace/tripleo/python-tripleoclient/tripleoclient/workflows/plan_management.py", line 15 | 14:13 |
| shadower | Anyone seen this CI issue? It's failing all our validation jobs (and seems unrelated) http://logs.openstack.org/98/353898/4/check/gate-tripleo-ci-centos-7-undercloud/e44f860/console.html#_2016-08-24_18_26_33_691667 | 14:13 |
| shadower | also seems unrelated to bug 1616144 | 14:14 |
| openstack | bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] https://launchpad.net/bugs/1616144 | 14:14 |
| sshnaidm | shadower, it means delroean couldn't build your patch | 14:14 |
| sshnaidm | shadower, you can see more detailed logs in delorean archive in logs directory | 14:14 |
| sshnaidm | shadower, http://logs.openstack.org/98/353898/4/check/gate-tripleo-ci-centos-7-undercloud/e44f860/logs/delorean_repos.tar.xz | 14:15 |
| slagle | derekh: still a problem? | 14:15 |
| shadower | sshnaidm: thanks | 14:15 |
| sshnaidm | slagle, works for me now | 14:16 |
| *** pgadiya has quit IRC | 14:17 | |
| sshnaidm | slagle, did something happen there? or just network issue | 14:17 |
| *** Ryjedo has joined #tripleo | 14:17 | |
| shadower | sshnaidm: ha so turns out it was an issue with the patch. Thanks! | 14:17 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates: Add fluentd client service https://review.openstack.org/353506 | 14:17 |
| slagle | sshnaidm: derekh : i think it was the iptables rules i was experimenting with to try and workaround the ntp issue | 14:17 |
| slagle | sshnaidm: derekh : i had added a MASQUERADE rule...it's gone now | 14:18 |
| slagle | that was probably it | 14:18 |
| sshnaidm | shadower, np :) | 14:18 |
| sshnaidm | slagle, so now ntp work there, right? | 14:18 |
| derekh | slagle: yup, that was it, nova list working for me now | 14:19 |
| slagle | sshnaidm: it still doesnt work. the rules i added didn't fix it anyway | 14:19 |
| hewbrocca | jpeeler: ugh, you working on debugging that regex? | 14:20 |
| slagle | sshnaidm: derekh : i had added a rule on the bastion to redirect ntp traffic to 172.16.1.1, and a postrouting masquerade rule, but none of that worked | 14:20 |
| sshnaidm | slagle, I think it's worth to open a ticket to engops about it | 14:20 |
| jpeeler | hewbrocca: sadly, yes. the regex is proper, but lvm doesn't like it. (and lvmconfig --validate did not report any problem either) | 14:21 |
| openstackgerrit | Martin André proposed openstack/tripleo-common: Allow running validation against different plans https://review.openstack.org/318194 | 14:21 |
| openstackgerrit | Martin André proposed openstack/tripleo-common: Workflows to run validations https://review.openstack.org/353899 | 14:21 |
| openstackgerrit | Martin André proposed openstack/tripleo-common: Workflows to load validations https://review.openstack.org/353898 | 14:21 |
| openstackgerrit | Martin André proposed openstack/tripleo-common: Disable retry files for ansible validations https://review.openstack.org/329039 | 14:21 |
| openstackgerrit | Martin André proposed openstack/tripleo-common: Workflow to setup validations https://review.openstack.org/313632 | 14:21 |
| slagle | sshnaidm: yes indeed. i was trying to find someone directly responsible, but failing that, the ticket is probably the right course of action | 14:21 |
| slagle | sshnaidm: are you famiilar with opening one? | 14:21 |
| hewbrocca | slagle: I don't even know who set up the trystack rack, unfortunately | 14:21 |
| hewbrocca | I've been trying to figure it out | 14:22 |
| slagle | yea me too :) | 14:22 |
| slagle | it's like this hardware magically appeared | 14:22 |
| slagle | and for some reason, despite that we can't find someone responsible for it, we've chosen to be anal about ntp | 14:22 |
| dtantsur | folks, are images at http://8.43.87.241/builds/current-tripleo/ still updated? I thought we had a promotion recently, but these are from 3 weeks ago... | 14:22 |
| slagle | seems an odd choice of prioritization :) | 14:23 |
| derekh | dtantsur: I guess that because we're no longer using rh2, this is the url for the same thing on rh1 http://66.187.229.18/builds/current-tripleo/ | 14:24 |
| dtantsur | aha, talking to quickstart folks about not using it then :) | 14:24 |
| sshnaidm | dtantsur, replied you in #oooq | 14:24 |
| thrash | d0ugal: looks like it's pulling an old version of tripleo-common | 14:25 |
| derekh | slagle: for the record, I don't have a clue either kambiz set up the switches for us but it looks like its beyond his control | 14:25 |
| derekh | slagle: by the way, your email about updating the undercloud flavor , I don't think you can update flavors iirc | 14:26 |
| sshnaidm | slagle, yeah, I have a long history of fighting there :)) | 14:27 |
| sshnaidm | slagle, I'll handle this then | 14:27 |
| derekh | slagle: so you may need to delete and recreate, which will cause an ID change, you might want to check with infra that this doesn't screw up nodepool | 14:27 |
| slagle | sshnaidm: thanks! | 14:27 |
| slagle | derekh: i think it did | 14:27 |
| derekh | slagle: opps, sorry didn't think of it earlier | 14:28 |
| derekh | slagle: you might be able to recreate it again with the old id instead of "auto" | 14:29 |
| derekh | slagle: if you have it | 14:29 |
| slagle | derekh: yep, trying tha tnow | 14:29 |
| *** mlupton has joined #tripleo | 14:30 | |
| derekh | ack, I'll shut up now | 14:30 |
| *** limao has quit IRC | 14:31 | |
| slagle | it's done, i'll see if that clears it up | 14:31 |
| *** limao has joined #tripleo | 14:31 | |
| pradk | x | 14:31 |
| dmsimard | EmilienM: are you guys going to be updating TripleO with the new authtoken stuff ? It's breaking Packstack every once in a while | 14:31 |
| EmilienM | dmsimard: I already did | 14:32 |
| dmsimard | EmilienM: ok, was just curious. | 14:32 |
| EmilienM | undercloud & overcloud are updated. | 14:32 |
| thrash | d0ugal: if you update requirements.txt to have tripleo-common>=5.0.0.0b2 and it works. | 14:32 |
| *** rajinir has joined #tripleo | 14:32 | |
| EmilienM | dmsimard: what broke? sorry about that | 14:32 |
| EmilienM | dmsimard: the work was supposed to be backward compatible | 14:32 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates: Add fluentd client service https://review.openstack.org/353506 | 14:33 |
| dmsimard | EmilienM: I think it's mostly Packstack-specific issues but since you ask https://review.openstack.org/#/c/355601/ and https://review.openstack.org/#/c/360325/ | 14:33 |
| openstackgerrit | Martin André proposed openstack/instack-undercloud: Setup CORS settings for tripleo-ui https://review.openstack.org/360593 | 14:34 |
| *** jbadiapa has quit IRC | 14:34 | |
| *** thrash is now known as thrash|biab | 14:35 | |
| EmilienM | dmsimard: weird, it should not have break :( | 14:35 |
| EmilienM | dmsimard: the parameters are backward compatible I don't get how it breaks | 14:37 |
| EmilienM | dmsimard: our p-o-i still use old parameters | 14:37 |
| EmilienM | and it works | 14:37 |
| dmsimard | EmilienM: in the case of gnocchi it was a duplicate keystone authtoken config error (packstack used it exclusively) and for ceilometer I'm not sure | 14:37 |
| EmilienM | dmsimard: ok for gnocchi it makes sense and it's not our fault. For ceilomter I'm curious | 14:37 |
| EmilienM | this thing is super critical I would like to know if something is wrong because we applied the same code to all modules | 14:38 |
| hewbrocca | slagle: just talked with mgalgoci | 14:38 |
| hewbrocca | he is going to respond on that thread within the next three hours | 14:38 |
| dmsimard | EmilienM: [1;31mError: Invalid parameter keystone_auth_uri on Class[Ceilometer::Api] at /var/tmp/packstack/601c5bb0479e4bcb97375329a99af5b5/modules/packstack/manifests/ceilometer.pp:73 on node n50.ci.centos.org | 14:38 |
| dmsimard | Wrapped exception: | 14:38 |
| hewbrocca | with some alternatives to actually opening that port, which he would very much like not to do | 14:39 |
| dmsimard | EmilienM: they removed keystone_auth_uri | 14:39 |
| jpeeler | hewbrocca: the regex issue rook is seeing is due to being improperly quoted - am fixing now. i thought that lvmconfig --validate would catch this, won't make that mistake again. | 14:39 |
| *** aufi has quit IRC | 14:39 | |
| hewbrocca | jpeeler: well, #winning then | 14:40 |
| EmilienM | dmsimard: let me look | 14:40 |
| dmsimard | EmilienM: keystone_auth_uri was already deprecated I guess | 14:40 |
| hewbrocca | just don't make rook have to live out of his car again | 14:40 |
| hewbrocca | he gets very grumpy | 14:40 |
| EmilienM | dmsimard: keystone_auth_uri was deprecated indeed | 14:40 |
| EmilienM | dmsimard: so, sorry if it broke you but we respected backward compatibility policy | 14:40 |
| numans | EmilienM, Hi, the review https://review.openstack.org/#/c/314875/ has passed the CI. If you can add this to your review Queue it would be awesome :) | 14:42 |
| EmilienM | numans: ack will do | 14:43 |
| numans | EmilienM, thank you | 14:43 |
| EmilienM | numans: +2 | 14:43 |
| *** aufi has joined #tripleo | 14:43 | |
| numans | EmilienM, that's great. Thanks. | 14:44 |
| openstackgerrit | Giulio Fidente proposed openstack/python-tripleoclient: Use heatclient poll_for_events in wait_for_stack_ready https://review.openstack.org/360141 | 14:44 |
| *** tobias-fiberdata has quit IRC | 14:47 | |
| *** limao has quit IRC | 14:48 | |
| *** newcomer25 has joined #tripleo | 14:48 | |
| *** aufi has quit IRC | 14:48 | |
| larsks | EmilienM: question for you re: https://review.openstack.org/#/c/359401/. Now my deploy is failing because "Error: Must pass admin_password to Class[Nova::Api] at ..." Do I need to change something? | 14:48 |
| *** openstackgerrit has quit IRC | 14:48 | |
| *** openstackgerrit has joined #tripleo | 14:49 | |
| larsks | EmilienM: hmmm, maybe a merge went awry locally. Checking... | 14:50 |
| larsks | EmilienM: not, that's not it. | 14:50 |
| dprince | pabelanger, derekh, sshnaidm, EmilienM: would it be reasonable to propose a CI change moratorium? | 14:50 |
| dprince | pabelanger, derekh, sshnaidm, EmilienM: like not adding any new features to CI until after the release? | 14:50 |
| EmilienM | I disagree | 14:51 |
| EmilienM | I'm pushing on having upgrade testing | 14:51 |
| sshnaidm | dprince, I'd merge image caching... | 14:51 |
| EmilienM | and thanks to my work, we found a bug in Heat that prevented the upgrade to work | 14:51 |
| dprince | pabelanger, derekh, sshnaidm, EmilienM: it would of course be okay to fix regressions, and for example increase the memory of the Undercloud to resovel the non-ha memory issue (SSL) | 14:51 |
| EmilienM | see https://review.openstack.org/360122 | 14:51 |
| dprince | EmilienM: upgrade testing in a new, non-voting job would be fine I think | 14:51 |
| EmilienM | I would like to continue my work on scenario001 and the update/upgrade jobs | 14:51 |
| EmilienM | dprince: none of my work is voting now | 14:52 |
| EmilienM | so I'm fine | 14:52 |
| dprince | EmilienM: I don't think we would stifle effort on those things | 14:52 |
| bnemec | It seems reasonable to me to observe feature freeze for CI too. | 14:52 |
| dprince | I would like to stop infra related changes in particular that might impact us | 14:52 |
| dprince | like for example the credentials change we made last week. Lets hold on those sorts of things for now | 14:52 |
| dprince | pabelanger: ^? | 14:52 |
| EmilienM | dprince: +1 on this thing | 14:52 |
| slagle | dprince: we might like to merge the image caching patch | 14:53 |
| EmilienM | we don't need more downtime. | 14:53 |
| EmilienM | +1 with sshnaidm and slagle on image caching thouh | 14:53 |
| slagle | dprince: although there is risk either way (using cached images vs jobs timing out) | 14:53 |
| dprince | slagle: okay, if we can demonstrate that it is safe enough and there is a huge impact perhaps | 14:53 |
| slagle | if we want safe, let's hold off | 14:54 |
| *** limao has joined #tripleo | 14:54 | |
| dprince | slagle: I want safe I think | 14:54 |
| slagle | b/c there is no way to effectively demonstrate | 14:54 |
| dprince | slagle: I would like to increase the size of the UC per your mailing list thread | 14:54 |
| dprince | slagle: so perhaps we do that and then hold ground on the "voting" jobs then? | 14:54 |
| slagle | dprince: that's already in progress, i've recreated the flavor and new instances are building now | 14:54 |
| dprince | slagle: perfect | 14:55 |
| EmilienM | sshnaidm: do we still have a way to manually do a tripleo promotion? | 14:55 |
| EmilienM | sshnaidm: I see all patches are abandonned | 14:55 |
| sshnaidm | EmilienM, yeah, did it 2 days before | 14:55 |
| EmilienM | sshnaidm: and I need a new package merged last night | 14:55 |
| sshnaidm | EmilienM, in order not to promote it not intentionally | 14:55 |
| EmilienM | sshnaidm: can we run the job now to get promotion? | 14:56 |
| d0ugal | thrash|biab: oh, interesting - but it should be pulling it from git. I guess it is cached somewhere? :/ | 14:56 |
| sshnaidm | EmilienM, I think so | 14:56 |
| sshnaidm | slagle, dprince - no objections to run promotion now? | 14:56 |
| *** saneax is now known as saneax-_-|AFK | 14:56 | |
| pabelanger | So, I have some thoughts on that. While I understand the reason for putting a freeze on CI changes, we really need to continue making some of them as they are affecting openstack-infra. For example, we do need a to land https://review.openstack.org/#/c/357308/ https://review.openstack.org/#/c/357341/1 because it is exposing private infrastruture to nodepool.o.o. And this is the last task I have to achieve | 14:57 |
| pabelanger | this cycle. | 14:57 |
| *** maeca1 has quit IRC | 14:57 | |
| sshnaidm | although it's big chance it will timeouted | 14:57 |
| slagle | sshnaidm: EmilienM : i'd like to make sure there is a way to roll back if the manual promotion goes wrong | 14:57 |
| slagle | sshnaidm: EmilienM : last time, it took almost a full day just to get the link restored | 14:57 |
| pabelanger | We are also working on a addressing the FIP issues that tripleo-test-cloud-rh1 is seeing | 14:58 |
| sshnaidm | slagle, EmilienM I can fix only image folder, repository depends on #rdo guys | 14:58 |
| pabelanger | which will require nodepool restarts | 14:58 |
| dtantsur | folks, when I have several environment files setting ControllerExtraConfig, will the result be a merge of two lists or will the second just replace the first? | 14:58 |
| slagle | sshnaidm: EmilienM : so I am not in favor of trying manual promotes until we have a way to recover from them | 14:58 |
| pabelanger | and possibly back and forth, depending if the issue is fixes | 14:58 |
| pabelanger | fixed* | 14:58 |
| sshnaidm | slagle, I think jpena knows now how to fix it :) | 14:58 |
| dtantsur | i.e. is it safe/reasonable to have ControllerExtraConfig in more than one connected environment file? | 14:58 |
| EmilienM | slagle: fair | 14:59 |
| gfidente | dtantsur, no they are not merged, last one wins | 14:59 |
| gfidente | you have to merge them into a single .yaml | 14:59 |
| dtantsur | sigh.. then I'd better just document the resulting environment, I guess. thanks gfidente | 14:59 |
| sshnaidm | slagle, dprince, EmilienM we have a problem with timeouts on promotion jobs that build images anyway, run undercloud idempotent and testing overcloud delete.. I'm not sure it could be fixed without any code change | 15:01 |
| dprince | sshnaidm: why are the promition jobs taking longer? | 15:01 |
| *** sai has joined #tripleo | 15:02 | |
| dprince | sshnaidm: rather than making changes should we turn something off? | 15:02 |
| sshnaidm | dprince, because they run tasks that don't run on gate jobs I think | 15:02 |
| dprince | sshnaidm: turn off tempest then | 15:02 |
| dprince | sshnaidm: would that buy us enough time? | 15:02 |
| *** thrash|biab is now known as thrash | 15:02 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Convert overcloud.yaml to support jinja2 templating https://review.openstack.org/315679 | 15:03 |
| sshnaidm | dprince, I don't talk about tempest job | 15:03 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Move AllNodesDeployments into jinja template loop https://review.openstack.org/337267 | 15:03 |
| dprince | sshnaidm: or something else? | 15:03 |
| openstackgerrit | Jiri Stransky proposed openstack/puppet-tripleo: Write restart flags to restart services only when necessary https://review.openstack.org/358510 | 15:03 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Move AllNodesValidationDeployments into jinja template loop https://review.openstack.org/337587 | 15:03 |
| slagle | tempest doesnt run on promote jobs | 15:03 |
| pabelanger | I also think, stopping CI development for 2 months is pretty unrealistic too. | 15:03 |
| sshnaidm | dprince, I'm talking about ovb ha and non-ha which promotes current-tripleo repo | 15:03 |
| slagle | sshnaidm: the larger undercloud flavor might help with that | 15:03 |
| *** sanjayu_ is now known as sanjayu|AFK | 15:04 | |
| sshnaidm | slagle, you mean adding CPUs as well? | 15:04 |
| dprince | exactly, those jobs should be the same specs, and tests as our normal jobs | 15:04 |
| *** jpich has quit IRC | 15:04 | |
| slagle | sshnaidm: yes, it went from 2 to 4 vcpus | 15:04 |
| sshnaidm | dprince, they run undercloud additional installation and test overcloud delete, which doesn't run in any of gate jobs | 15:05 |
| slagle | dprince: the jobs have not been the same. they also test overcloud delete and undercloud idempotent install | 15:05 |
| *** mlupton_ has joined #tripleo | 15:05 | |
| slagle | they've been that way for quite a while. see toci_gate_test | 15:06 |
| dprince | slagle: I'm thinking that having a separate job for those test might be better. | 15:06 |
| sshnaidm | 10 last runs of ha job were cut with timeout | 15:06 |
| dprince | slagle: I really like the idea of what promotes being what gates | 15:06 |
| *** mlupton has quit IRC | 15:07 | |
| sshnaidm | It's worth to do code freeze when it's stable, and it's not really now | 15:07 |
| *** bnemec has quit IRC | 15:08 | |
| slagle | dprince: yea, i think it'd be fine to remove them | 15:08 |
| openstackgerrit | Adriano Petrich proposed openstack/tripleo-heat-templates: GATE TEST, please ignore https://review.openstack.org/360618 | 15:09 |
| *** dprince has quit IRC | 15:09 | |
| *** mlupton_ has quit IRC | 15:09 | |
| jistr | dtantsur: the Heat patch alone didn't do the trick for me https://bugs.launchpad.net/tripleo/+bug/1616550/comments/7 | 15:10 |
| openstack | Launchpad bug 1616550 in tripleo "Overcloud update is broken: UPDATE_FAILED NotFound_Remote: resources[0]: Software config with id not found" [High,Confirmed] | 15:10 |
| *** ooolpbot has joined #tripleo | 15:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 15:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 15:10 |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 15:10 |
| *** ooolpbot has quit IRC | 15:10 | |
| jistr | "this time it fails a different way" | 15:10 |
| dtantsur | jistr, ooops | 15:10 |
| sshnaidm | slagle, CI problems are not related to CI code changes, how would help a code freeze? | 15:10 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Implement non-ovb overcloud update job - Newton -> Newton https://review.openstack.org/351330 | 15:11 |
| pabelanger | sshnaidm: some problems have been a result of CI code changes, but I would say it has been minimal | 15:11 |
| slagle | sshnaidm: i'm not proposing a freeze on that | 15:11 |
| slagle | will have to get dprince to confirm what he meant when he returns | 15:12 |
| *** bnemec has joined #tripleo | 15:13 | |
| *** limao has quit IRC | 15:14 | |
| *** maeca1 has joined #tripleo | 15:15 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Remove EPEL usage https://review.openstack.org/347499 | 15:15 |
| *** limao has joined #tripleo | 15:15 | |
| hewbrocca | +10000000 | 15:16 |
| hewbrocca | setting all these people's keyboards on fire is becoming tiresome | 15:16 |
| EmilienM | dtantsur: i'm investigating an undercloud upgrade issue Mitaka -> Newton for ironic API FYI http://logs.openstack.org/95/346995/44/experimental/gate-tripleo-ci-centos-7-undercloud-upgrades-nv/23ed518/console.html#_2016-08-24_17_22_46_379758 | 15:17 |
| openstackgerrit | Jiri Stransky proposed openstack/puppet-tripleo: Write restart flags to restart services only when necessary https://review.openstack.org/358510 | 15:17 |
| sshnaidm | slagle, about a periodic jobs, maybe it's better to distribute a time-taking features on more jobs? for example another job that will test overcloud delete, but not more additional features, and so for every job? | 15:17 |
| EmilienM | dtantsur: I'll let you know | 15:17 |
| dtantsur | thanks... | 15:17 |
| dtantsur | bugs, bugs everywhere | 15:17 |
| dtantsur | EmilienM, of good news: it seems like we're able to fix the auth issue on ironic side | 15:18 |
| * hewbrocca sends exterminator to dtantsur | 15:18 | |
| *** vijaykc4 has quit IRC | 15:18 | |
| dtantsur | too dangerous, I can exterminate EVERYTHING | 15:18 |
| EmilienM | dtantsur: awesome news | 15:18 |
| *** maeca1 has quit IRC | 15:18 | |
| EmilienM | dtantsur: lol | 15:18 |
| * bnemec makes obligatory "take off and nuke it from orbit" joke | 15:19 | |
| *** mlupton has joined #tripleo | 15:19 | |
| * dtantsur is ready to not take off and just nuke everything for greater justice | 15:19 | |
| *** links has joined #tripleo | 15:20 | |
| EmilienM | dtantsur: dude that looks terrible | 15:20 |
| EmilienM | dtantsur: when I upgrade Ironic from Mitaka to newton and I try api again, I got this http://paste.openstack.org/show/R7e21sFdZWN57WctNevl/ | 15:21 |
| slagle | sshnaidm: more periodic jobs testing individual features is good imo | 15:21 |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-ui: Use Mistral/Zaqar for folder-based plan-creation https://review.openstack.org/360550 | 15:21 |
| *** weshay is now known as weshay_afk | 15:21 | |
| dtantsur | EmilienM, this sounds like python-ironic-lib was not updated | 15:22 |
| dtantsur | any chances it may be the case? | 15:22 |
| openstackgerrit | Adriano Petrich proposed openstack/tripleo-quickstart: WIP gate upgrade https://review.openstack.org/342161 | 15:22 |
| slagle | sshnaidm: we dont even need ovb to test undercloud idempotent | 15:22 |
| EmilienM | dtantsur: let me check that | 15:22 |
| slagle | sshnaidm: we could in fact be testing that in the undercloud job | 15:22 |
| sshnaidm | slagle, do you want to make periodic multinode? | 15:22 |
| slagle | which is in the gate | 15:22 |
| sshnaidm | ah, ok | 15:22 |
| EmilienM | dtantsur: after upgrade, I got python-ironic-lib-2.0.0-0.20160802195403.6fec307.el7.centos.noarch | 15:23 |
| bnemec | Yeah, although if we do that we have very little verification that things still work after the idempotent run. | 15:23 |
| sshnaidm | bnemec, and what do we do after idem.. run in periodic job? | 15:24 |
| bnemec | But at least we have verification that it's possible to re-run the undercloud install and have it not blow up. | 15:24 |
| *** maeca1 has joined #tripleo | 15:24 | |
| *** mlupton has quit IRC | 15:24 | |
| bnemec | sshnaidm: I think just the ping test, and then overcloud delete in periodic jobs. | 15:25 |
| *** vijaykc4 has joined #tripleo | 15:25 | |
| dtantsur | EmilienM, 2.0.0 must be fine, strange | 15:25 |
| slagle | true, either way what we have now is not buying us much. b/c it's passed maybe 3 times in a month | 15:25 |
| dtantsur | EmilienM, is it possible ironic gets updated before the lib? | 15:25 |
| EmilienM | dtantsur: yeah, I'm invstigating | 15:25 |
| bnemec | I see only five active testenvs for a queue of 60. Is this still the fip thing? | 15:27 |
| *** akuznetsov has quit IRC | 15:27 | |
| *** rhallisey_ has joined #tripleo | 15:28 | |
| *** mlupton has joined #tripleo | 15:28 | |
| slagle | yea was just testing the fip creation, which is fine | 15:28 |
| slagle | pabelanger: any way to tell what nodepool is up to? | 15:28 |
| slagle | pabelanger: we have 60 freshly spawned instances just sitting in ACTIVE | 15:28 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: WIP: Add the ability to customize undercloud with virt-customize https://review.openstack.org/360637 | 15:29 |
| *** derekh has quit IRC | 15:31 | |
| *** rhallisey has quit IRC | 15:32 | |
| colonwq | EmilienM, scenario004 will need to be IPv4 until this fix in inline everywhere: https://github.com/ceph/ceph/pull/10579 | 15:32 |
| EmilienM | colonwq: ok | 15:33 |
| bnemec | slagle: I do see a few testenvs spinning up, so maybe zuul is just running a little behind sending us jobs. | 15:33 |
| bnemec | It's probably pissed at us for breaking things all the time. :-) | 15:33 |
| beagles | shardy, is there an example where we *generate* a script from contents in a parameter in the repo somewhere? | 15:34 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Fix multiple default exports in single module https://review.openstack.org/360641 | 15:34 |
| *** derekh has joined #tripleo | 15:34 | |
| *** milan has quit IRC | 15:35 | |
| pabelanger | slagle: waiting to assign FIPs | 15:35 |
| pabelanger | slagle: this is the problem | 15:35 |
| *** limao has quit IRC | 15:35 | |
| *** newcomer25 has quit IRC | 15:36 | |
| pabelanger | we can reduce capacity again, so nodepool isn't so bursty, which might offer some relief | 15:36 |
| *** maeca1 has joined #tripleo | 15:36 | |
| *** limao has joined #tripleo | 15:36 | |
| EmilienM | dtantsur: updated in same time | 15:36 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Do not merge, checking composable mistral. https://review.openstack.org/352947 | 15:36 |
| EmilienM | dtantsur: maybe ironic api is started before updating the package | 15:36 |
| EmilienM | dtantsur: /me checking | 15:37 |
| dtantsur | yeah, worth checking | 15:37 |
| slagle | pabelanger: waiting on what though? | 15:37 |
| slagle | creating and assigning fip's takes seconds | 15:37 |
| larsks | EmilienM: re: my earlier question, are upgrades necessary to opm packages other than puppet-nova? ceilometer::api is calling validate_string($keystone_password) which is failing because keystone_password is a boolean. I am updating puppet-ceilometer right now, but curious if I am going to need any more... | 15:37 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Update validations action name https://review.openstack.org/360643 | 15:37 |
| EmilienM | dtantsur: ok that's it | 15:37 |
| EmilienM | dtantsur: ironic-api is updated by Puppet and Puppet starts the service right away but ironic-lib is not yet updated | 15:38 |
| dtantsur | yeah | 15:38 |
| EmilienM | larsks: update them all | 15:38 |
| EmilienM | dtantsur: I'm investigating if it's a packaging issue | 15:38 |
| larsks | EmilienM: fair enough. | 15:38 |
| pabelanger | slagle: for the FIP to attach | 15:38 |
| EmilienM | so python-ironic-lib is dep of ironic-common which is a dep of ironic-api | 15:39 |
| shardy | beagles: Hey, sorry not sure I follow - you want to pass a string parameter then have a SoftwareConfig resource reference it? | 15:40 |
| pabelanger | slagle: https://github.com/openstack-infra/shade/commit/7dd138c8f4c8b8761b0b50d3ce9c7c4f770f8c61 | 15:40 |
| beagles | shardy, actually I think I found something that illustrates how in major_upgrade_pacemaker_init.yaml | 15:41 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: WIP: Add the ability to customize undercloud with virt-customize https://review.openstack.org/360637 | 15:41 |
| beagles | shardy, context is we've run into a snag in the SR-IOV support and I'm brainstorming alternatives | 15:41 |
| EmilienM | dtantsur: it's an orchestration bug, I think I found it | 15:42 |
| EmilienM | dtantsur: nothing you can do on ironic side | 15:42 |
| *** tremble has quit IRC | 15:43 | |
| dtantsur | ack, lemme know if I can help | 15:43 |
| *** rhallisey_ is now known as rhallisey | 15:45 | |
| openstackgerrit | Steven Hardy proposed openstack-infra/tripleo-ci: Implement scenario001 CI job https://review.openstack.org/360039 | 15:47 |
| shardy | EmilienM: ^^ WIP, not yet tested but shows the basic approach | 15:47 |
| EmilienM | shardy: thanks | 15:47 |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Use relative paths in ci-scripts for quickstart.sh https://review.openstack.org/360649 | 15:48 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Limit Heat engine workers to 4 in rh1 https://review.openstack.org/360650 | 15:48 |
| EmilienM | shardy: tenantvm_server.yaml and tenantvm_server_boot_from_volume.yaml are the same files | 15:48 |
| openstackgerrit | Steven Hardy proposed openstack-infra/tripleo-ci: Implement scenario001 CI job https://review.openstack.org/360039 | 15:49 |
| shardy | EmilienM: Yeah, fixed | 15:49 |
| shardy | you can modify them however you like though provided the parameters interface remains compatible with tenantvm_floatingip.yaml | 15:50 |
| EmilienM | dtantsur: it sounds like when puppet updates ironic-api with "yum update", the dependencies are not updated immediatly | 15:50 |
| EmilienM | shardy: ok I see | 15:50 |
| openstackgerrit | Steven Hardy proposed openstack-infra/tripleo-ci: Implement scenario001 CI job https://review.openstack.org/360039 | 15:51 |
| karthiks | shardy, regarding https://review.openstack.org/#/c/329415/ , for input | 15:52 |
| karthiks | NovaPCIPassthrough: | 15:52 |
| karthiks | - vendor_id: 8086 | 15:52 |
| karthiks | product_id: 154c | 15:52 |
| karthiks | devname: eth0 | 15:52 |
| karthiks | getting an error [{"devname"=>"eth0", "vendor_id"=>8086, "product_id"=>"154c"}] is not a String at /etc/puppet/modules/nova/manifests/compute.pp:264 | 15:53 |
| openstackgerrit | Steven Hardy proposed openstack-infra/tripleo-ci: Implement scenario001 CI job https://review.openstack.org/360039 | 15:54 |
| EmilienM | dtantsur: I'll try a workaround in puppet-ironic | 15:54 |
| shardy | karthiks: have you changed the parameter type from string to json? | 15:54 |
| shardy | then used str_replace to convert it back to a string for the hieradata? | 15:55 |
| gfidente | merge anyone https://review.openstack.org/#/c/353945/ ? | 15:55 |
| karthiks | yes | 15:55 |
| karthiks | nova::compute::pci_passthrough: | 15:56 |
| karthiks | str_replace: | 15:56 |
| karthiks | template: JSON_PARAM | 15:56 |
| karthiks | params: | 15:56 |
| karthiks | JSON_PARAM: {get_param: NovaPCIPassthrough} | 15:56 |
| gfidente | this one too is clean port to liberty https://review.openstack.org/#/c/355882/ | 15:56 |
| *** limao has quit IRC | 15:57 | |
| gfidente | karthiks, you can pass the map with get_param to pci_passthrough as is | 15:57 |
| *** limao has joined #tripleo | 15:57 | |
| *** matbu is now known as matbu|afk | 15:57 | |
| *** rasca has quit IRC | 15:58 | |
| d0ugal | thrash: I don't know how to fix this :( | 15:58 |
| gfidente | karthiks, actually looks like it should be an array of hashes | 15:58 |
| d0ugal | thrash: Putting a beta version in the requirements.txt seems wrong | 15:58 |
| karthiks | gfidente, yes, array of hashes enclosed within " " | 15:59 |
| gfidente | karthiks, but I would not make any mangling in heat and rather assume the parameter is given with proper formatting | 15:59 |
| shardy | gfidente: Yeah, it was getting passed as a string with a list of maps in json, but I wanted to instead use a json parameter | 15:59 |
| shardy | https://review.openstack.org/#/c/329415/8/puppet/services/nova-compute.yaml | 15:59 |
| thrash | d0ugal: it's that or ignore that failure. | 16:00 |
| shardy | having folks write json strings in environment files is really error prone | 16:00 |
| gfidente | shardy, ah didn't know you were doing that, thought I could do something useful :P | 16:00 |
| *** mcornea has quit IRC | 16:00 | |
| d0ugal | thrash: or find a way to have the check respect the requirements.txt properly... but that seems hard :( | 16:00 |
| *** links has quit IRC | 16:00 | |
| d0ugal | thrash: I guess the requirements.txt bot will complain if I put a beta version in | 16:01 |
| *** mlupton has quit IRC | 16:01 | |
| thrash | d0ugal: seems like it's pulling from global-requirements.txt | 16:01 |
| d0ugal | thrash: I don't think so | 16:01 |
| *** numans has quit IRC | 16:01 | |
| d0ugal | thrash: cd python-tripleoclient; pip install -e . | 16:01 |
| d0ugal | thrash: That wont pull from git | 16:01 |
| d0ugal | thrash: just tested in a fresh venv | 16:02 |
| gfidente | shardy, karthiks so I think hiera() is doing magic conversion, you actually might have to add " around get_param in template: ? | 16:02 |
| *** mlupton has joined #tripleo | 16:02 | |
| gfidente | I've seen it happening with [ipv6] addresses passed as arrays | 16:02 |
| *** derekh has quit IRC | 16:02 | |
| karthiks | gfidente, you have an example ? | 16:02 |
| gfidente | karthiks, commenting on the submission | 16:03 |
| karthiks | Thanks gfidente | 16:03 |
| shardy | karthiks: it may help to run hiera | 16:04 |
| shardy | nova::compute::pci_passthrough | 16:04 |
| shardy | sorry hiera nova::compute::pci_passthrough on the node | 16:04 |
| shardy | as gfidente says it may be a string quoting issue | 16:04 |
| openstackgerrit | Ben Nemec proposed openstack/instack-undercloud: Pre-install packages before running puppet https://review.openstack.org/353121 | 16:04 |
| gfidente | commented, but looks like this should be fixed in the puppet module | 16:05 |
| gfidente | anyway that "'workaround'" should do | 16:05 |
| *** maeca1 has quit IRC | 16:05 | |
| shardy | thanks gfidente | 16:05 |
| thrash | d0ugal: ahh | 16:06 |
| thrash | d0ugal: that's the prob I guess.. no git | 16:06 |
| openstackgerrit | Steven Hardy proposed openstack-infra/tripleo-ci: Implement scenario001 CI job https://review.openstack.org/360039 | 16:08 |
| karthiks | gfidente, shardy Thanks. Will check. | 16:10 |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 16:10 |
| openstackgerrit | Steven Hardy proposed openstack-infra/tripleo-ci: Implement scenario001 CI job https://review.openstack.org/360039 | 16:11 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 16:11 |
| EmilienM | dtantsur: trying again ^ | 16:11 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates: Add support for configuring the OVS firewall driver https://review.openstack.org/357556 | 16:11 |
| *** bana_k has joined #tripleo | 16:12 | |
| openstackgerrit | Steven Hardy proposed openstack-infra/tripleo-ci: Implement scenario001 CI job https://review.openstack.org/360039 | 16:14 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 16:15 |
| EmilienM | dtantsur: ok got my local undercloud upgraded :) | 16:15 |
| EmilienM | w00t ! | 16:15 |
| dtantsur | \o/ | 16:16 |
| *** limao has quit IRC | 16:18 | |
| *** limao has joined #tripleo | 16:18 | |
| *** dprince has joined #tripleo | 16:19 | |
| d0ugal | shardy: Hey, do you have a minute? | 16:19 |
| d0ugal | shardy: if not, it can wait :) | 16:19 |
| shardy | d0ugal: sure | 16:19 |
| d0ugal | shardy: I have a dilema - check-osc-plugin is broken here: https://review.openstack.org/#/c/359784 | 16:20 |
| *** hewbrocca is now known as hewbrocca-afk | 16:20 | |
| d0ugal | shardy: it is broken because of how we list tripleo-common in the requirements.txt | 16:20 |
| d0ugal | shardy: https://github.com/openstack/python-tripleoclient/blob/master/requirements.txt#L20 | 16:20 |
| d0ugal | shardy: basically, it gets an older version of tripleo-common from PyPI and the check fails. | 16:21 |
| *** dhill_ has quit IRC | 16:21 | |
| d0ugal | shardy: and I don't know how to fix this, at least, not in the short term | 16:21 |
| d0ugal | shardy: (The only quick fix I can think of is to duplicate the code I want from triple_common, it isn't that much) | 16:22 |
| shardy | d0ugal: can we just release tripleo-common so the pypi version is updated? | 16:22 |
| gfidente | slagle, bnemec do you know why we need selinux/permissive on centos? | 16:22 |
| d0ugal | shardy: Sure, that would work - it would mean that we need to bump the PyPI release every time tripleoclient uses something new | 16:22 |
| bnemec | gfidente: Nope, I was actually quite interested to see your patch yesterday. | 16:23 |
| d0ugal | shardy: but it turns out, that isn't that often since I am just hitting this. | 16:23 |
| gfidente | bnemec, ack | 16:23 |
| shardy | d0ugal: Yeah making the check job pull it from git would definitely be better | 16:23 |
| shardy | I'm not familiar with how that job is implemented tbh | 16:23 |
| d0ugal | shardy: https://github.com/openstack-infra/project-config/blob/master/jenkins/scripts/check_osc_commands.py | 16:23 |
| bnemec | I feel like there was a reason the check jobs use deps from pip. | 16:23 |
| slagle | gfidente: there is some aspect of not wanting to be blocked everytime we need an selinux policy update | 16:24 |
| slagle | and i'm not a fan of carrying custom policy | 16:24 |
| slagle | but, maybe we should just be blocked when it happens | 16:24 |
| d0ugal | shardy: the problem is that "pip install -e ." wont pull git deps, I think it is a limitation of pbr | 16:24 |
| gfidente | slagle, right I see your point | 16:24 |
| gfidente | bnemec, pls https://review.openstack.org/#/c/353945/ and https://review.openstack.org/#/c/355882/ :) | 16:24 |
| karthiks | gfidente, shardy It works. Thanks | 16:25 |
| slagle | pabelanger: is it possible to see the nodepool logs of when it's requesting to attch the fip's? | 16:25 |
| gfidente | karthiks, ^_^ cool :) | 16:26 |
| gfidente | karthiks, but then we should try to make things better in the module I think | 16:26 |
| slagle | pabelanger: i'd like to see if i can correspond that to something on the server side (as is, i'm not seeing any requests but i might be looking at the wrong places) | 16:26 |
| openstackgerrit | Steven Hardy proposed openstack-infra/tripleo-ci: Implement scenario001 CI job https://review.openstack.org/360039 | 16:26 |
| shardy | EmilienM: Sorry for the churn, I think ^^ works now, or at least it starts the pingtest stack | 16:27 |
| karthiks | gfidente, you mean puppet-nova ? | 16:27 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: [WIP] Set parameters https://review.openstack.org/360391 | 16:27 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Add 'openstack overcloud plan create' command https://review.openstack.org/359784 | 16:27 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: [WIP] Add `openstack overcloud deploy --plan` https://review.openstack.org/360305 | 16:27 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Update the 'overcloud deploy --templates' command to use Mistral https://review.openstack.org/342261 | 16:27 |
| d0ugal | shardy: ^ I just tried setting the version to 5.0.0.0b2 (suggestion from thrash) but I half expect another job to complain about a beta release :) | 16:28 |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Add -L option for curl during image downloads https://review.openstack.org/360665 | 16:28 |
| gfidente | karthiks, puppet-nova yes | 16:28 |
| gfidente | maybe check_array_of_hash might pass with both string or an actual array | 16:29 |
| karthiks | gfidente, ok will do it post N3 :) | 16:29 |
| gfidente | karthiks, sure | 16:29 |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Add -L option for curl during image downloads https://review.openstack.org/360665 | 16:30 |
| pabelanger | slagle: not much in nodepool logs currently. We have shade debug logs disabled. | 16:30 |
| pabelanger | basically use just get: | 16:30 |
| pabelanger | 2016-08-25 16:28:29,989 DEBUG nodepool.ProviderManager: Manager osic-cloud1 running task ServerGet (queue: 6) | 16:31 |
| pabelanger | 2016-08-25 16:28:30,436 DEBUG nodepool.ProviderManager: Manager osic-cloud1 ran task ServerGet in 0.287037849426s | 16:31 |
| pabelanger | for an hour on the node | 16:31 |
| pabelanger | then we delete and try again | 16:31 |
| pabelanger | change osic-cloud1 with tripleo-test-cloud-rh1 | 16:31 |
| slagle | pabelanger: odd, i just dont see any requests coming in to associate floating ip's | 16:31 |
| pabelanger | ever? | 16:32 |
| EmilienM | shardy: ok thanks | 16:32 |
| thrash | d0ugal: worth a shot tho | 16:33 |
| slagle | pabelanger: well right now we are pretty much sitting idle | 16:33 |
| *** mlupton has quit IRC | 16:33 | |
| d0ugal | thrash: I agree, that's why I tried :) | 16:33 |
| pabelanger | slagle: yes, we are busting tripleo-test-cloud-rh1 with 57 nodes | 16:34 |
| pabelanger | which is likely part of the issue | 16:34 |
| slagle | pabelanger: the instances are built. they're ACTIVE | 16:34 |
| pabelanger | right, but not from nodepool POV | 16:34 |
| pabelanger | because we haven't attached a FIP | 16:35 |
| pabelanger | I just deleted 5 servers | 16:35 |
| pabelanger | that should kick off the relaunch | 16:35 |
| slagle | just wondering if anything has actually asked to attach a FIP | 16:36 |
| slagle | there is no bottleneck in attaching fip's afaict | 16:36 |
| *** dprince has quit IRC | 16:37 | |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-docs: [WIP] Documentation for installing and using Ironic in overcloud https://review.openstack.org/354016 | 16:38 |
| bnemec | Weird. We have a bunch of active testenvs, but 0 slaves with fips. | 16:38 |
| dtantsur | folks, anyone (dsneddon?) has an idea why flat network won't work for instances? or at least how to debug? | 16:38 |
| *** mlupton has joined #tripleo | 16:40 | |
| *** sshnaidm is now known as sshnaidm|afk | 16:40 | |
| *** jpich has joined #tripleo | 16:40 | |
| *** ifarkas is now known as ifarkas_afk | 16:42 | |
| *** limao has quit IRC | 16:43 | |
| *** limao has joined #tripleo | 16:44 | |
| pabelanger | bnemec: when bust clouds with FIPs we see this issue. We need to stagger the launches so FIP assignment doesn happen all at once | 16:44 |
| gfidente | bnemec, wrong channel, I didn't realize it does | 16:44 |
| *** rcernin has quit IRC | 16:45 | |
| *** bana_k has quit IRC | 16:46 | |
| *** fragatina has joined #tripleo | 16:47 | |
| *** fragatina has quit IRC | 16:47 | |
| *** fragatina has joined #tripleo | 16:48 | |
| openstackgerrit | Chris Krelle proposed openstack/diskimage-builder: WIP: A hardware burn-in element. https://review.openstack.org/355675 | 16:48 |
| *** fragatina has quit IRC | 16:49 | |
| *** florianf has quit IRC | 16:50 | |
| *** fragatina has joined #tripleo | 16:50 | |
| *** hjensas has quit IRC | 16:52 | |
| *** dprince has joined #tripleo | 16:53 | |
| *** tesseract- has quit IRC | 16:56 | |
| EmilienM | shardy: why did you add pingtest in test-environments? | 16:57 |
| shardy | EmilienM: because we need a default environment file for tripleo.sh to use | 16:58 |
| shardy | it could go in templates/ if you prefer | 16:58 |
| *** akrivoka has quit IRC | 16:59 | |
| *** pkovar has quit IRC | 17:00 | |
| EmilienM | shardy: I guess it's fine I'm just wondering why we have multiple folders | 17:00 |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: WIP: Document Third Party CI and developer usage https://review.openstack.org/360007 | 17:00 |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: WIP: Document Third Party CI and developer usage https://review.openstack.org/360007 | 17:01 |
| *** athomas has quit IRC | 17:01 | |
| *** limao has quit IRC | 17:02 | |
| *** weshay_afk is now known as weshay | 17:03 | |
| EmilienM | shardy: I don't think this is going to work | 17:03 |
| EmilienM | in https://review.openstack.org/#/c/360039/11/toci_gate_test.sh we look for a pingtest that is called scenario001-multinode | 17:03 |
| EmilienM | but you removed my file | 17:04 |
| shardy | EmilienM: Yeah, you'll need to change that to pass the environment file via PINGTEST_ENV | 17:04 |
| shardy | EmilienM: https://review.openstack.org/#/c/360039/11/scripts/tripleo.sh | 17:06 |
| *** skramaja has quit IRC | 17:06 | |
| shardy | so instead of overriding PINGTEST_TEMPLATE, you'll override PINGTEST_ENV to point to a no-cinder version | 17:06 |
| shardy | (or, if you pass the required mapping in scenario001-multinode.yaml you can just set it to an empty string or dummy template | 17:07 |
| EmilienM | mhh this is not what I wanted | 17:07 |
| EmilienM | I wanted a pingtest per scenario but with common bits | 17:07 |
| shardy | EmilienM: thats | 17:07 |
| shardy | what this does | 17:07 |
| EmilienM | ok | 17:07 |
| shardy | you can pass an environment file which modifies the pingtest stack that is deployed | 17:07 |
| EmilienM | the names are just confusing to me | 17:08 |
| shardy | give it a try and feel free to erase my changes from history if you don't like them :) | 17:08 |
| EmilienM | no no I want to keep your stuff | 17:08 |
| *** skramaja has joined #tripleo | 17:08 | |
| EmilienM | I just feel like the dog & compute atm | 17:08 |
| shardy | I did it pretty quickly to illustrate the approach, perhaps the naming can be improved | 17:08 |
| chem | bandini: hi, about the https://review.openstack.org/354713 | 17:08 |
| EmilienM | shardy: https://s-media-cache-ak0.pinimg.com/736x/30/e4/c6/30e4c6141cee4f15281591f22db0b249.jpg | 17:08 |
| EmilienM | shardy: ok I'll polish it how I can | 17:09 |
| chem | bandini: M/N mysql upgrade do you need more info ? | 17:09 |
| shardy | lol @ the dog :D | 17:09 |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 17:10 |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| shardy | EmilienM: basically I just abstracted the non-common bits behind the OS::PingTest::Server, so all common bits are now in templates/tenantvm_floatingip.yaml | 17:10 |
| *** mlupton has quit IRC | 17:10 | |
| shardy | it's similar to what we do in many places in t-h-t | 17:10 |
| EmilienM | shardy: excellent, I'll just do some renames to make things obvious | 17:11 |
| *** vijaykc4 has quit IRC | 17:11 | |
| shardy | then we redefine OS::PingTest::Server, depending on whether you want boot-from-volume or not | 17:11 |
| shardy | EmilienM: ack, sounds good | 17:11 |
| *** vijaykc4 has joined #tripleo | 17:12 | |
| *** ebalduf has quit IRC | 17:12 | |
| ccamacho | going away. | 17:12 |
| *** lucasagomes is now known as lucas-dinner | 17:12 | |
| *** chem is now known as chem|off | 17:13 | |
| *** bana_k has joined #tripleo | 17:16 | |
| *** mlupton has joined #tripleo | 17:16 | |
| *** abregman has quit IRC | 17:18 | |
| *** vijaykc4 has quit IRC | 17:22 | |
| *** maeca1 has joined #tripleo | 17:24 | |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart: Update configuration docs https://review.openstack.org/356700 | 17:28 |
| dsneddon | dtantsur, Can you explain your problem with flat networks? | 17:30 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Move Redis, Memcached, Sahara, OVS agent out of role templates https://review.openstack.org/360697 | 17:31 |
| dsneddon | dtantsur, Flat networks should work with instances, but you need to make sure that your bridges are set up correctly. You need to have the same bridge mapped on both controllers and computes. | 17:31 |
| dtantsur | dsneddon, the problem with flat networks was dead simple: VMs did not react on anything (pings, ssh) no matter what I do. "No route to host". | 17:31 |
| dtantsur | well, I guess tripleo creates bridges, not me... | 17:31 |
| *** vijaykc4 has joined #tripleo | 17:32 | |
| dsneddon | dtantsur, Not true. You need an external bridge on the computes to use flat networking. You can put a br-ex on the computes and use that. | 17:32 |
| karthiks | gfidente, shardy if I need a config like pci_passthrough_whitelist = [{"product_id":"0001", "vendor_id":"8086"}, | 17:32 |
| karthiks | # {"product_id":"0002", "vendor_id":"8086"}] | 17:32 |
| dtantsur | dsneddon, should I do it manually by logging into computes? | 17:33 |
| *** vijaykc4 has quit IRC | 17:33 | |
| karthiks | how shall the input be given in environment.yaml ? | 17:33 |
| dsneddon | dtantsur, You can do that. Can you point me to your compute.yaml NIC config? | 17:33 |
| openstackgerrit | Dmitry Tantsur proposed openstack/tripleo-docs: [WIP] Documentation for installing and using Ironic in overcloud https://review.openstack.org/354016 | 17:33 |
| dtantsur | dsneddon, I don't know what you're referring to, sorry. I'm pretty sure I never did it. This ^^^ is everything I've done (what succeeded) | 17:33 |
| dsneddon | dtantsur, Depending on which interface you want to use, it can be tricky, because if you want br-ex on the provisioning interface, you will lose connectivity while trying to set it up (unless you change the ifcfg files and reboot). | 17:34 |
| dtantsur | as you see, I ended up creating another (non-flat) network for instances, and it works, at least partly | 17:34 |
| dtantsur | well, fair :) | 17:34 |
| dtantsur | dsneddon, thanks for help, I really have to bail out already. could you please check https://review.openstack.org/354016 to see if I'm doing something really stupid? | 17:35 |
| dsneddon | dtantsur, Sure, will do. | 17:35 |
| shardy | karthiks: won't it just be a yaml list of maps, like I put on the review? | 17:35 |
| shardy | karthiks: http://paste.openstack.org/show/563434/ | 17:35 |
| *** dtantsur is now known as dtantsur|afk | 17:36 | |
| karthiks | Will check it shardy. Multiple entries are supported . | 17:37 |
| shardy | karthiks: it should be the same as before, only you can just write the yaml structure without caring about all the json quoting etc | 17:38 |
| beagles | karthiks, is what is giving you trouble is you need it to be transported as a string to the puppet module? | 17:39 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add global_config_settings to services' output https://review.openstack.org/336542 | 17:39 |
| beagles | karthiks, ie, not interpreted as JSON? | 17:39 |
| beagles | I think gfidente posted some feedback about that... if that's not what it is, sorry for the noise | 17:39 |
| *** dhill_ has joined #tripleo | 17:41 | |
| karthiks | OK .... shardy. Will check it for my part. beagles gfidente comments helped and it converted the json params as strings for puppet | 17:41 |
| *** StevenK has quit IRC | 17:41 | |
| *** StevenK has joined #tripleo | 17:42 | |
| *** tosky has quit IRC | 17:42 | |
| *** maeca1 has quit IRC | 17:45 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add global_config_settings to services' output https://review.openstack.org/336542 | 17:47 |
| openstackgerrit | Karthik S proposed openstack/tripleo-heat-templates: Configure the pci_passthrough_whitelist via THT https://review.openstack.org/329415 | 17:48 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Implement scenario001 CI job https://review.openstack.org/360039 | 17:54 |
| *** rcernin has joined #tripleo | 17:54 | |
| *** oshvartz has quit IRC | 17:56 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Implement scenario001 CI job https://review.openstack.org/360039 | 17:59 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Test new scenario001 job https://review.openstack.org/360040 | 17:59 |
| *** jcoufal has joined #tripleo | 17:59 | |
| *** maeca1 has joined #tripleo | 18:02 | |
| *** rhallisey has quit IRC | 18:03 | |
| *** yamahata has quit IRC | 18:06 | |
| *** mlupton has quit IRC | 18:08 | |
| *** ooolpbot has joined #tripleo | 18:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 18:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 18:10 |
| *** ooolpbot has quit IRC | 18:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 18:10 |
| *** mlupton has joined #tripleo | 18:11 | |
| openstackgerrit | Ben Swartzlander proposed openstack/puppet-tripleo: Add manila-netapp backend to manila class https://review.openstack.org/354014 | 18:12 |
| *** maeca1 has quit IRC | 18:12 | |
| *** kbyrne has quit IRC | 18:13 | |
| gfidente | bnemec, commented on https://review.openstack.org/#/c/360097/ looks like we just need two booleans and an allow for redis_t | 18:14 |
| *** david-lyle_ has joined #tripleo | 18:15 | |
| *** julim_ has quit IRC | 18:15 | |
| *** david-lyle_ has quit IRC | 18:15 | |
| *** oshvartz has joined #tripleo | 18:16 | |
| *** kbyrne has joined #tripleo | 18:16 | |
| *** fragatina has quit IRC | 18:17 | |
| *** jubapa has quit IRC | 18:21 | |
| gfidente | bnemec, for more selinux fun: https://bugzilla.redhat.com/show_bug.cgi?id=1367926#c3 | 18:22 |
| openstack | bugzilla.redhat.com bug 1367926 in openstack-selinux "label /srv/data to work with ceph-selinux polixy" [Unspecified,New] - Assigned to rhallise | 18:22 |
| EmilienM | gfidente: I reported something like this already | 18:23 |
| EmilienM | a sec | 18:23 |
| EmilienM | http://tracker.ceph.com/issues/16674 | 18:23 |
| thrash | d0ugal: https://review.openstack.org/#/c/359784/10/requirements.txt | 18:23 |
| EmilienM | ah not this one | 18:23 |
| *** mlupton has quit IRC | 18:24 | |
| gfidente | EmilienM, subject is wrong, the /srv/data problem is fixed in puppet-ceph already | 18:24 |
| EmilienM | gfidente: right, nevermind | 18:24 |
| gfidente | EmilienM, it's the other comments in the BZ which make the story interesting | 18:24 |
| EmilienM | ok | 18:24 |
| EmilienM | gfidente: weird, sounds like a packaging sync issue | 18:24 |
| gfidente | yeah | 18:24 |
| larsks | EmilienM: I have a question for you re: the puppet/services/pacemaker/ceilometer-* templates you worked on. These patches all "inherit" from ../ceilometer-base.yaml rather than the corresponding service template (e.g., ../ceilometer-agent-central.yaml), which is different from pretty much everything else. | 18:24 |
| gfidente | EmilienM, but we're almost there | 18:25 |
| larsks | Is there a reason for that, or was that just an accident of how they were created? | 18:25 |
| EmilienM | larsks: most of services inherits from a base service | 18:25 |
| EmilienM | it's on purpose | 18:25 |
| * EmilienM brb lunch | 18:25 | |
| larsks | EmilienM: waaiiiiit! | 18:26 |
| larsks | :/ | 18:26 |
| larsks | EmilienM: hopefully you see this when you get back. I think you misunderstood my question. | 18:26 |
| EmilienM | larsks: what is different? | 18:27 |
| larsks | EmilienM: most of the services in the pacemaker/ directory inherit from the corresponding service in the parent directory. E.g., pacemaker/nova-api.yaml instantiates ../nova-api.yaml, and pacemaker/glance-api.yaml inherits from ../glance-api.yaml. | 18:27 |
| * EmilienM cooking spaguettis | 18:27 | |
| gfidente | EmilienM with what? | 18:27 |
| larsks | The ceilometer services, though, all inherit from ../ceilometer-base.yaml. | 18:27 |
| EmilienM | the ceilometer pacemaker services are going to disappear | 18:27 |
| EmilienM | ceilometer is not managed by systemd anymore | 18:28 |
| gfidente | EmilienM, there is different ways | 18:28 |
| larsks | Oh? | 18:28 |
| larsks | Well, that makes my life easier then. | 18:28 |
| EmilienM | the new HA spec | 18:28 |
| larsks | Sweet. | 18:28 |
| EmilienM | can I go eat my spaguettis? | 18:28 |
| larsks | Go eat spaguettis! | 18:28 |
| larsks | Thanks! | 18:28 |
| *** mlupton has joined #tripleo | 18:28 | |
| EmilienM | merci | 18:28 |
| *** vijaykc4 has joined #tripleo | 18:28 | |
| gfidente | though it's 'spaghetti' | 18:29 |
| gfidente | without the u, and without the s | 18:29 |
| EmilienM | I'll trust the italian guy this time :P | 18:30 |
| gfidente | right and what are you having them with? | 18:30 |
| *** electrofelix has quit IRC | 18:30 | |
| *** maeca1 has joined #tripleo | 18:31 | |
| EmilienM | I don't know how to spell it but I know how to eat very well | 18:32 |
| EmilienM | I'm doing my own sauce with tomatoes and beef | 18:32 |
| EmilienM | basilic, etc | 18:32 |
| *** rhallisey has joined #tripleo | 18:33 | |
| *** paramite has quit IRC | 18:33 | |
| vijaykc4 | hello cores need your attention on this patch https://review.openstack.org/#/c/332190/ | 18:36 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Fix base service type inheriting ceilometer service pcmk templates https://review.openstack.org/360729 | 18:38 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Fix base service type inheriting gnocchi service templates https://review.openstack.org/360731 | 18:40 |
| *** marcusvrn_ has joined #tripleo | 18:41 | |
| gfidente | EmilienM, so that usually goes without beef | 18:41 |
| EmilienM | damn | 18:42 |
| gfidente | *but* | 18:42 |
| EmilienM | I'm also adding a bit of cheese in the sauce | 18:42 |
| gfidente | no no no | 18:42 |
| gfidente | no cheese | 18:42 |
| gfidente | instead, listen to me | 18:42 |
| gfidente | you *do* use the beef to make the tomato saurce | 18:42 |
| gfidente | but then, on the spaghetti you use the sauce without the beef | 18:42 |
| gfidente | and you eat the beef cooked in the sauce, *after* with salad | 18:43 |
| EmilienM | oh wow | 18:43 |
| EmilienM | is there a spec for that? | 18:43 |
| gfidente | right | 18:43 |
| *** yamahata has joined #tripleo | 18:43 | |
| EmilienM | I was so proud of my lunch and now I feel like newbie | 18:44 |
| *** mlupton has quit IRC | 18:44 | |
| *** paramite has joined #tripleo | 18:45 | |
| openstackgerrit | Chris Krelle proposed openstack/diskimage-builder: WIP: A hardware burn-in element. https://review.openstack.org/355675 | 18:46 |
| gfidente | EmilienM, I am always like that | 18:47 |
| EmilienM | i guess I'm better at cooking puppet manifests | 18:48 |
| EmilienM | well I hope | 18:48 |
| gfidente | EmilienM, are you saying that the spaghetti aren't good as you wished? | 18:51 |
| *** jcoufal has quit IRC | 18:51 | |
| EmilienM | they are excellent for me :P | 18:51 |
| gfidente | cool :) | 18:51 |
| *** mlupton has joined #tripleo | 18:52 | |
| dprince | lucas-dinner, dtantsur|afk: I'm hitting this now :/ https://bugs.launchpad.net/ironic/+bug/1617033 | 18:52 |
| openstack | Launchpad bug 1617033 in tripleo "Ironic fails to download glance ramdisk/kernel with V3" [Undecided,New] | 18:52 |
| openstackgerrit | Ben Swartzlander proposed openstack/tripleo-heat-templates: Add integration with NetApp Manila driver https://review.openstack.org/354019 | 18:52 |
| *** krotscheck has left #tripleo | 18:52 | |
| *** maeca1 has quit IRC | 18:55 | |
| *** maeca1 has joined #tripleo | 18:55 | |
| *** paramite has quit IRC | 18:57 | |
| *** maeca1 has quit IRC | 18:57 | |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates: Add fluentd client service https://review.openstack.org/353506 | 18:58 |
| *** rkolli has joined #tripleo | 19:00 | |
| *** fragatina has joined #tripleo | 19:02 | |
| *** maeca1 has joined #tripleo | 19:06 | |
| *** ooolpbot has joined #tripleo | 19:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 19:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 19:10 |
| *** ooolpbot has quit IRC | 19:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 19:10 |
| openstackgerrit | Julie Pichon proposed openstack/tripleo-common: Add Mistral action & workflow for root device configuration https://review.openstack.org/355947 | 19:12 |
| openstackgerrit | Julie Pichon proposed openstack/tripleo-common: Add boot option argument to the Node Registration Mistral workflow https://review.openstack.org/348875 | 19:12 |
| openstackgerrit | Julie Pichon proposed openstack/tripleo-common: Add Mistral action for boot configuration https://review.openstack.org/354174 | 19:12 |
| slagle | dprince: bnemec : EmilienM : i think we could merge the patch to use cached images | 19:13 |
| slagle | it has cumulatively passed all jobs | 19:13 |
| slagle | https://review.openstack.org/#/c/359481/ | 19:13 |
| EmilienM | ok | 19:13 |
| EmilienM | +A | 19:14 |
| dprince | slagle: I thought we agreed earlier not to do this now | 19:14 |
| dprince | slagle: just the fix to increase the UC size | 19:15 |
| dprince | EmilienM: I need more than 30 seconds to reply ;) | 19:15 |
| slagle | oh did we? | 19:16 |
| slagle | it felt like we were just talking | 19:16 |
| slagle | i dunno :) | 19:16 |
| slagle | dprince: there were some questions about what you were proposing after you left, fwiw | 19:17 |
| dprince | slagle: no changes unless needed | 19:17 |
| slagle | i think "freeze" was reacted to negatively | 19:17 |
| dprince | slagle: i.e. moratorium on any new functionality in the "gating" jobs | 19:17 |
| slagle | ok, well, needed is kind of subjective | 19:17 |
| slagle | dprince: again, that doesnt add any clarity, b/c this is not new | 19:18 |
| dprince | slagle: lets put it this way | 19:18 |
| dprince | slagle: I'll be really sad if Thursday and Friday go like last week | 19:18 |
| slagle | dprince: dude, we've barely had a passing job all week | 19:18 |
| slagle | what ci are you looking at??? | 19:19 |
| dprince | slagle: right, so lets fix the nonha job first | 19:19 |
| dprince | slagle: and then do the caching at a later time I think | 19:19 |
| dprince | slagle: one step at a time | 19:19 |
| dprince | slagle: I'd like more than an hour or 2 between these things I think at this point in the release cycle | 19:19 |
| slagle | stop the patch then | 19:20 |
| dprince | slagle: lets talk about it on the list | 19:21 |
| slagle | that would probably be a good idea | 19:21 |
| slagle | you didnt even reply to my email about the current failure | 19:21 |
| slagle | a little odd to be coming up with a policy | 19:21 |
| dprince | slagle: I had nothing to add at the time... | 19:21 |
| dprince | slagle: no need for me to reply if good replies are already there I think... | 19:21 |
| dprince | slagle: what I mean is I fully support what you did for the nonha memory increase | 19:22 |
| dprince | slagle: and now is the time to make that change | 19:22 |
| *** rhallisey has quit IRC | 19:25 | |
| *** rhallisey has joined #tripleo | 19:26 | |
| slagle | dprince: ok well the next issue to look into is probably why instances arent getting floating ips | 19:27 |
| *** oshvartz has quit IRC | 19:27 | |
| slagle | it's hurting the number of simultaneous jobs we can have running | 19:27 |
| dprince | slagle: I'd agree that addressing that is probably of high importance | 19:28 |
| dprince | slagle: could be a "fun" one | 19:28 |
| slagle | it might be | 19:29 |
| *** mlupton has quit IRC | 19:38 | |
| *** lblanchard has quit IRC | 19:39 | |
| EmilienM | dtantsur|afk: https://review.openstack.org/#/c/346995/ | 19:42 |
| EmilienM | gate-tripleo-ci-centos-7-undercloud-upgrades-nv is green \o/ | 19:42 |
| sshnaidm|afk | dprince, did you mean in https://review.openstack.org/#/c/359481/ ha job? because nonha passed there | 19:42 |
| *** mlupton has joined #tripleo | 19:43 | |
| dprince | sshnaidm|afk: hi, I just meant that I'd like to give some time to improve our CI stats in general before landing that. We've had a bad week so far :/ | 19:43 |
| dprince | sshnaidm|afk: i.e. lets not make 2 CI changes within hours of each other... | 19:43 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: More cleanup of controller.yaml https://review.openstack.org/360751 | 19:46 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Fix use of snmp service https://review.openstack.org/360062 | 19:47 |
| -openstackstatus- NOTICE: The Gerrit service on review.openstack.org is restarting to implement some performance tuning adjustments, and should return to working order momentarily. | 19:47 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 19:47 |
| sshnaidm|afk | dprince, bad week, that's right.. though this patch means exactly the improvement of CI stats :) it would solve timeout issues for us | 19:48 |
| openstackgerrit | Julie Pichon proposed openstack/python-tripleoclient: Use Mistral for baremetal boot configuration https://review.openstack.org/358012 | 19:48 |
| openstackgerrit | Julie Pichon proposed openstack/python-tripleoclient: Add 'openstack overcloud node import' command https://review.openstack.org/339727 | 19:48 |
| openstackgerrit | Julie Pichon proposed openstack/python-tripleoclient: Add 'openstack overcloud node configure' command https://review.openstack.org/344875 | 19:48 |
| dprince | sshnaidm|afk: and we may land it... but lets give at least some time to see if nonha improves with more memory | 19:49 |
| sshnaidm|afk | dprince, sure, do we have already 8gb there? | 19:49 |
| dprince | sshnaidm|afk: otherwise it is harder to tell what causes our outages | 19:49 |
| *** vijaykc4 has quit IRC | 19:49 | |
| dprince | sshnaidm|afk: yes, I think it landed earlier right slagle? | 19:49 |
| *** skramaja has quit IRC | 19:50 | |
| *** jprovazn has quit IRC | 19:50 | |
| sshnaidm|afk | dprince, it should be configured on place, isn't it? | 19:50 |
| sshnaidm|afk | oh, sorry, I forgot I can check it by my own now | 19:51 |
| slagle | dprince: sshnaidm|afk : it's live. 8GB ram and 4vcpus for the undercloud | 19:52 |
| *** mlupton has quit IRC | 19:52 | |
| openstackgerrit | Julie Pichon proposed openstack/python-tripleoclient: Use Mistral for baremetal boot configuration https://review.openstack.org/358012 | 19:53 |
| openstackgerrit | Julie Pichon proposed openstack/python-tripleoclient: Add 'openstack overcloud node configure' command https://review.openstack.org/344875 | 19:53 |
| *** mlupton has joined #tripleo | 19:54 | |
| openstackgerrit | Julie Pichon proposed openstack/python-tripleoclient: Use Ironic's generic service name in help messages https://review.openstack.org/360755 | 19:55 |
| gfidente | slagle, curious if that makes puppet apply on undercloud any faster? | 19:56 |
| dprince | jpich: what do you have against "Ironic" :) | 19:56 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Update undercloud flavor stats https://review.openstack.org/360756 | 19:57 |
| jpich | dprince: Nothing in particular, just listening to people who know more about when it's ok to use that name than I apparently do :) https://review.openstack.org/#/c/344875/4/tripleoclient/v1/overcloud_node.py | 19:58 |
| slagle | gfidente: possibly | 19:59 |
| dprince | jpich: its cool, +2 | 19:59 |
| jpich | dprince: Cheers! | 19:59 |
| *** jpich has quit IRC | 20:00 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Use modulepath for Puppet https://review.openstack.org/356457 | 20:01 |
| slagle | gfidente: i wouldnt expect it to have much impact tbh. that's not where we've typically been going into swap, and i dont think puppet really makes use of multiple cpus | 20:01 |
| slagle | i dont think it parallelizes equivalent tasks with no depdenencies | 20:01 |
| gfidente | yeah I was interested in the puppet part as I remember times 3/5 times better on 'multinode' | 20:01 |
| slagle | i dont see how it could given how slow it is | 20:02 |
| dprince | slagle: it might help out multi-process stuff in general | 20:02 |
| openstackgerrit | Merged openstack/instack-undercloud: Update authtoken parameters to match recent changes https://review.openstack.org/359400 | 20:02 |
| gfidente | slagle, and the multinode job has 4cpus I remember | 20:02 |
| dprince | slagle: like much of the puppet slowness is related to it exec'ing other commands which may well benefit here | 20:02 |
| slagle | gfidente: multinode job has 8 cpus | 20:02 |
| gfidente | slagle, oh even more | 20:02 |
| slagle | i'd like to eventually use the same for all nodes | 20:03 |
| gfidente | so somehow the cpu type or the number of cpus makes it 5 times faster | 20:03 |
| gfidente | in doing same task, and I mean puppet compile | 20:03 |
| gfidente | now on puppet apply maybe there is less parallelism | 20:03 |
| gfidente | unless it's about the cpu type, not number | 20:04 |
| slagle | gfidente: it's something to investigate | 20:04 |
| slagle | rh1 is slower compared to equivalent tasks in the multinode job, and even in my local env | 20:04 |
| slagle | across the board pretty much | 20:04 |
| *** mlupton has quit IRC | 20:04 | |
| slagle | yet i see no reason | 20:04 |
| gfidente | is nova maybe emulating with kvm some inconvenient cpu type? | 20:05 |
| gfidente | are we using kvm at all? | 20:05 |
| slagle | yes | 20:05 |
| gfidente | yeah so how do we compare emuation type? cpu flags and frequency? | 20:06 |
| gfidente | even bogomips? | 20:06 |
| slagle | /proc/cpuinfo is in the devstack-setup-host log | 20:06 |
| gfidente | of multinode? | 20:06 |
| slagle | of both | 20:06 |
| slagle | but yes | 20:07 |
| gfidente | nice | 20:07 |
| *** dprince has quit IRC | 20:07 | |
| slagle | we can see the libvirt xml for our ovb jobs | 20:07 |
| slagle | but we aren't going to get that for multinode jobs | 20:07 |
| gfidente | model name: Intel Xeon E312xx (Sandy Bridge) | 20:08 |
| gfidente | locally I have QEMU Virtual CPU | 20:08 |
| gfidente | and the OVB one has a lot more of hardware flags than I do | 20:08 |
| gfidente | and even an higher bogomips | 20:08 |
| gfidente | :( | 20:09 |
| *** rhallisey has quit IRC | 20:09 | |
| *** ooolpbot has joined #tripleo | 20:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 20:10 |
| *** ooolpbot has quit IRC | 20:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,Triaged] | 20:10 |
| *** bfournie has quit IRC | 20:13 | |
| EmilienM | gfidente: you don't wait for ovb jobs to finish? | 20:14 |
| EmilienM | you approved dprince's patches | 20:14 |
| gfidente | EmilienM, no the one above passed | 20:14 |
| EmilienM | oki doc | 20:14 |
| gfidente | EmilienM, https://review.openstack.org/#/c/360697/1 | 20:15 |
| gfidente | also we can approve https://review.openstack.org/#/c/360536/1 | 20:15 |
| EmilienM | gfidente: reviewing it right now | 20:15 |
| EmilienM | but can you teach me again how to make pastas before? | 20:15 |
| gfidente | after? | 20:16 |
| EmilienM | ok after | 20:16 |
| gfidente | EmilienM+ | 20:16 |
| *** mlupton has joined #tripleo | 20:17 | |
| gfidente | I am dealing with an overcloud | 20:17 |
| gfidente | where after a few seconds from nodes boot | 20:17 |
| gfidente | I can't login as heat-admin anymore | 20:17 |
| gfidente | and I am hating it | 20:17 |
| gfidente | not sure if you guys have ever seen this | 20:17 |
| gfidente | (interestingly I *can* login from some seconds, then not anymore) | 20:18 |
| slagle | how long did you boil it for? | 20:18 |
| EmilienM | sounds like HAproxy? | 20:18 |
| EmilienM | ah heat admin | 20:19 |
| EmilienM | err, I thought API sorry | 20:19 |
| gfidente | slagle, the overcloud? | 20:21 |
| gfidente | or EmilienM's pasta? | 20:21 |
| EmilienM | lol | 20:21 |
| gfidente | AND WHY ON EARTH IS ALWAYS THE SAME NODE WHICH LOCKS ME OUT | 20:23 |
| gfidente | OOM in overcloud? | 20:24 |
| gfidente | (too) | 20:24 |
| bnemec | gfidente: Networking problem? I've seen that "can only log in for a little while" behavior if os-net-config fails when it runs. | 20:25 |
| gfidente | nice thanks will try that though this is without netiso | 20:26 |
| slagle | EmilienM: commented on https://review.openstack.org/#/c/360039/ | 20:27 |
| *** rhallisey has joined #tripleo | 20:28 | |
| EmilienM | slagle: thx so much for your +2 | 20:28 |
| EmilienM | slagle: what you're asking will make us copy/paste all multinode parameters, right? | 20:31 |
| EmilienM | ie: NODECOUNT=1 etc | 20:31 |
| slagle | i don't think so | 20:31 |
| slagle | what is $TOCI_JOBTYPE? | 20:31 |
| slagle | the case statement is a loop, so as long as multinode is in there, the vars will be set | 20:32 |
| slagle | and then if scenario* comes after, you can override the vars as needed | 20:32 |
| EmilienM | slagle: it's a variable defined in project-config | 20:32 |
| EmilienM | it's exported by ZUUL | 20:32 |
| slagle | i know | 20:33 |
| slagle | i'm asking what the actual value is | 20:33 |
| EmilienM | ah sorry | 20:34 |
| EmilienM | slagle: scenario001-multinode | 20:35 |
| slagle | ok, so it wont work since scenario comes first | 20:35 |
| slagle | we should change that | 20:35 |
| gfidente | bnemec, fwiw, I actually get the permission denied error by the sshd and can ping the node IPs | 20:38 |
| openstackgerrit | Merged openstack/instack-undercloud: Enable SSL for ironic inspector https://review.openstack.org/359787 | 20:39 |
| slagle | gfidente: sounds like it is either cloud-init not configuring the ssh key | 20:40 |
| slagle | or maybe the node booting an old image | 20:40 |
| *** mlupton has quit IRC | 20:43 | |
| dsneddon | gfidente, What slagle said. You know, if there are problems with the DHCP server, the nodes can fail through to an old image. | 20:44 |
| gfidente | dsneddon, hey :) | 20:44 |
| EmilienM | slagle: you want me to change something? | 20:46 |
| *** maeca1 has quit IRC | 20:46 | |
| gfidente | dsneddon, there are problems with the dhcp | 20:46 |
| *** morazi has quit IRC | 20:46 | |
| beagles | does anyone have any ideas on what might to add filters to lvm.conf? | 20:47 |
| beagles | puppet or packaging... | 20:47 |
| dsneddon | gfidente, Good luck, I actually just popped in to make sure I didn't miss anything here before I pop out to lunch. | 20:47 |
| * beagles realizes he can rule out puppet pretty quickly | 20:47 | |
| slagle | EmilienM: not sure. am what I'm saying making sense? | 20:47 |
| gfidente | dsneddon, ahaha thanks, I am not sure how this causes permission denied in ssh but ... | 20:48 |
| dsneddon | gfidente, Because when you redeployed, the SSH key for heat-admin was updated locally, but the old image has the old one... | 20:48 |
| gfidente | no the key in the image is fine | 20:48 |
| *** mlupton has joined #tripleo | 20:48 | |
| dsneddon | gfidente, Ah, then it's not the root cause that I thought it was. | 20:49 |
| gfidente | dsneddon, no worries, thanks :) | 20:50 |
| EmilienM | slagle: I don't have strong opinion on where to put the conditional | 20:50 |
| EmilienM | slagle: the "case" statement doesn't look the right place on this case I think | 20:51 |
| EmilienM | except if you want to duplicate multinode parameters | 20:51 |
| EmilienM | and add a new "scenario" case | 20:51 |
| *** jeckersb is now known as jeckersb_gone | 20:51 | |
| rkolli | slagle: sorry for reaching out to you w/o context. I have https://review.openstack.org/#/c/351358/ waiting for a 2nd +2. Would it be possible for you to take a look | 20:54 |
| EmilienM | rkolli: reviewed | 20:56 |
| slagle | lgtm :) | 20:56 |
| rkolli | EmilienM: thanks | 20:57 |
| sshnaidm|afk | I want to change some parameter in ironic.conf for CI jobs, how can I do it in right way? | 20:59 |
| sshnaidm|afk | changing it in the heat template / puppet / or anywhere else? | 21:01 |
| *** asbishop has quit IRC | 21:03 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Use delorean's exit code to decide on retry https://review.openstack.org/348790 | 21:04 |
| openstackgerrit | Merged openstack/tripleo-common: Updated from global requirements https://review.openstack.org/348695 | 21:05 |
| openstackgerrit | Jeff Peeler proposed openstack/instack-undercloud: Modify improper augeas set command https://review.openstack.org/360790 | 21:05 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Disable the Mon/OSD/Client resources in puppet-ceph-external https://review.openstack.org/351753 | 21:05 |
| *** mlupton has quit IRC | 21:06 | |
| *** mlupton has joined #tripleo | 21:06 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Move out of roles any Ceph setting https://review.openstack.org/356657 | 21:06 |
| openstackgerrit | Merged openstack/python-tripleoclient: Use non-bracketed version of addresses for keystone.initialize https://review.openstack.org/355882 | 21:07 |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1616144 | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1616144 in tripleo "tripleo-ci: nonha jobs failing with Unable to establish connection to https://192.0.2.2:13004/v1/a90407df1e7f4f80a38a1b1671ced2ff/stacks/overcloud/f9f6f712-8e89-4ea9-a34b-6084dc74b5c1" [Critical,In progress] - Assigned to James Slagle (james-slagle) | 21:10 |
| gfidente | to me it sounds like cloud-init doesn't get or can't get to metadata | 21:20 |
| *** mlupton has quit IRC | 21:20 | |
| openstackgerrit | Giulio Fidente proposed openstack/python-tripleoclient: Use heatclient poll_for_events in wait_for_stack_ready https://review.openstack.org/360141 | 21:22 |
| *** mhenkel has quit IRC | 21:24 | |
| *** mlupton has joined #tripleo | 21:26 | |
| shardy | Hey all, two SR-IOV patches look ready to land, see https://review.openstack.org/#/c/332190/ and https://review.openstack.org/#/c/345352/ | 21:31 |
| gfidente | so the proble is that the nodes have multiple disks and on reboot they're not booting from the disk where they just dumped the image ... | 21:32 |
| gfidente | ... so when they (re)boot they come with shit totally different from the overcloud image | 21:34 |
| bnemec | gfidente: Maybe you need http://docs.openstack.org/developer/tripleo-docs/advanced_deployment/root_device.html ? | 21:38 |
| gfidente | bnemec, yeah i've just been through that ; 'largest' | 21:40 |
| *** pradk has quit IRC | 21:41 | |
| * gfidente has hops | 21:42 | |
| gfidente | *hopes :) | 21:42 |
| *** rlandy is now known as rlandy|bbl | 21:43 | |
| *** cdearborn has quit IRC | 21:50 | |
| *** mlupton has quit IRC | 21:53 | |
| *** shardy has quit IRC | 21:57 | |
| *** mlupton has joined #tripleo | 21:57 | |
| *** mburned is now known as mburned_out | 22:08 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Adding optional aci-opflex hiera data https://review.openstack.org/351358 | 22:08 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Implement scenario001 CI job https://review.openstack.org/360039 | 22:11 |
| *** mlupton has quit IRC | 22:13 | |
| *** chem|off has quit IRC | 22:16 | |
| *** chem|off has joined #tripleo | 22:18 | |
| *** Goneri has joined #tripleo | 22:22 | |
| *** mlupton has joined #tripleo | 22:22 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Move network bind IPs out of compute.yaml https://review.openstack.org/359940 | 22:24 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Move glance/heat hiera settings to services https://review.openstack.org/360029 | 22:28 |
| *** mlupton has quit IRC | 22:34 | |
| *** mlupton has joined #tripleo | 22:35 | |
| *** Goneri has quit IRC | 22:39 | |
| *** chem|off` has joined #tripleo | 22:46 | |
| openstackgerrit | Giulio Fidente proposed openstack/python-tripleoclient: Use heatclient poll_for_events in wait_for_stack_ready https://review.openstack.org/360141 | 22:47 |
| *** chem|off has quit IRC | 22:48 | |
| openstackgerrit | Merged openstack/puppet-tripleo: SR-IOV nic agent changes in line with composable roles. https://review.openstack.org/332191 | 22:53 |
| *** rhallisey has quit IRC | 22:57 | |
| *** chlong has quit IRC | 22:58 | |
| *** mlupton_ has joined #tripleo | 22:59 | |
| *** mlupton has quit IRC | 23:00 | |
| *** mpsairam has joined #tripleo | 23:00 | |
| *** mlupton_ has quit IRC | 23:10 | |
| *** chlong has joined #tripleo | 23:11 | |
| *** mlupton has joined #tripleo | 23:11 | |
| *** jobewan has quit IRC | 23:15 | |
| *** Ryjedo_ has joined #tripleo | 23:16 | |
| *** Ryjedo has quit IRC | 23:17 | |
| *** Ryjedo_ is now known as Ryjedo | 23:17 | |
| *** ebalduf has joined #tripleo | 23:18 | |
| *** mlupton has quit IRC | 23:19 | |
| *** mlupton has joined #tripleo | 23:21 | |
| *** yamahata has quit IRC | 23:25 | |
| *** chem|off` has quit IRC | 23:25 | |
| *** chem|off` has joined #tripleo | 23:26 | |
| *** mlupton has quit IRC | 23:29 | |
| *** mlupton has joined #tripleo | 23:31 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Allow scheduler_available_filter to be customizable https://review.openstack.org/344829 | 23:31 |
| *** chlong has quit IRC | 23:35 | |
| *** yamahata has joined #tripleo | 23:37 | |
| openstackgerrit | Merged openstack/tripleo-puppet-elements: Ceph Jewel repos split packages into ceph-{mon,osd,radosgw} https://review.openstack.org/353945 | 23:39 |
| *** mlupton_ has joined #tripleo | 23:40 | |
| *** mlupton has quit IRC | 23:40 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: [NO MERGE] Test Ceph RadosGW as replacement for Swift https://review.openstack.org/357182 | 23:42 |
| *** mlupton_ has quit IRC | 23:48 | |
| *** mlupton has joined #tripleo | 23:50 | |
| *** rcernin has quit IRC | 23:53 | |
| *** mlupton has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
