| *** Sarath has joined #tripleo | 00:05 | |
| *** mlupton has quit IRC | 00:11 | |
| *** mlupton has joined #tripleo | 00:14 | |
| *** tosky has quit IRC | 00:16 | |
| *** mlupton has quit IRC | 00:18 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: [mitaka-only] mysql: never add brackets to mysql_bind_host https://review.openstack.org/371029 | 00:21 |
|---|---|---|
| *** mlupton has joined #tripleo | 00:21 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Convert deploy steps to jinja2 loop https://review.openstack.org/365796 | 00:24 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Convert UpdateWorkflow to support composable roles https://review.openstack.org/367282 | 00:24 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Convert AllNodesExtraConfig to support composable roles https://review.openstack.org/367295 | 00:24 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Add fluentd client service https://review.openstack.org/353506 | 00:24 |
| EmilienM | larsks: ok, let's try again :) | 00:24 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Deploy TripleO with Puppet 4 https://review.openstack.org/371209 | 00:26 |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient: Migrate to using osc-lib https://review.openstack.org/370517 | 00:34 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: mysql: never add brackets to mysql_bind_host https://review.openstack.org/369369 | 00:35 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: get_host_info: get repos list https://review.openstack.org/369305 | 00:35 |
| *** mlupton has quit IRC | 00:39 | |
| *** Sarath has quit IRC | 00:44 | |
| *** skramaja_ has joined #tripleo | 00:47 | |
| *** skramaja has quit IRC | 00:47 | |
| *** mlupton has joined #tripleo | 00:50 | |
| *** mlupton has quit IRC | 00:52 | |
| *** dmacpher has quit IRC | 00:54 | |
| *** jcoufal has joined #tripleo | 00:55 | |
| *** bana_k has quit IRC | 01:14 | |
| *** mlupton has joined #tripleo | 01:32 | |
| *** mlupton has quit IRC | 01:36 | |
| *** kbyrne has quit IRC | 01:40 | |
| *** dmacpher has joined #tripleo | 01:43 | |
| *** kbyrne has joined #tripleo | 01:47 | |
| *** fultonj has quit IRC | 01:48 | |
| *** limao has joined #tripleo | 01:58 | |
| *** thrash is now known as thrash|g0ne | 01:59 | |
| *** egafford has quit IRC | 02:08 | |
| *** trozet has quit IRC | 02:11 | |
| *** jlinkes has quit IRC | 02:15 | |
| *** kbyrne has quit IRC | 02:15 | |
| *** kbyrne has joined #tripleo | 02:16 | |
| *** mlupton has joined #tripleo | 02:17 | |
| *** limao has quit IRC | 02:18 | |
| *** tzumainn has quit IRC | 02:18 | |
| *** limao has joined #tripleo | 02:19 | |
| *** jlinkes has joined #tripleo | 02:23 | |
| ayoung | EmilienM, credentials looks good. Think we can risk un-pegging Keystone | 02:24 |
| *** trozet has joined #tripleo | 02:25 | |
| *** mlupton has quit IRC | 02:27 | |
| *** absubram has quit IRC | 02:28 | |
| *** mlupton has joined #tripleo | 02:31 | |
| *** absubram has joined #tripleo | 02:40 | |
| *** mlupton has quit IRC | 02:41 | |
| *** absubram has quit IRC | 02:51 | |
| *** jlinkes has quit IRC | 02:52 | |
| *** jlinkes has joined #tripleo | 02:52 | |
| *** pmannidi has quit IRC | 02:59 | |
| *** pmannidi has joined #tripleo | 02:59 | |
| dtrainor | I'm trying to introspect some nodes. I see the ipxe dialogue, they start to come up, and then they infinitely fail trying to download agent.kernel | 03:02 |
| *** mlupton has joined #tripleo | 03:09 | |
| dtrainor | https://bugzilla.redhat.com/show_bug.cgi?id=1364079 | 03:11 |
| openstack | bugzilla.redhat.com bug 1364079 in ipxe "iPXE hangs with an infinite stream of different errors" [Unspecified,Closed: worksforme] - Assigned to rhos-maint | 03:11 |
| dtrainor | hmmmm | 03:11 |
| *** mlupton has quit IRC | 03:15 | |
| *** TicToc has quit IRC | 03:20 | |
| *** TicToc has joined #tripleo | 03:21 | |
| *** limao has quit IRC | 03:25 | |
| *** jlinkes has quit IRC | 03:48 | |
| *** bana_k has joined #tripleo | 04:00 | |
| *** jlinkes has joined #tripleo | 04:04 | |
| *** TicToc has quit IRC | 04:21 | |
| *** TicToc has joined #tripleo | 04:25 | |
| *** rcernin has quit IRC | 04:37 | |
| *** jlinkes has quit IRC | 04:42 | |
| *** absubram has joined #tripleo | 04:43 | |
| *** absubram_ has joined #tripleo | 04:44 | |
| *** absubram has quit IRC | 04:48 | |
| *** absubram_ is now known as absubram | 04:48 | |
| *** absubram has quit IRC | 04:50 | |
| *** jaosorior has joined #tripleo | 04:57 | |
| *** jlinkes has joined #tripleo | 04:58 | |
| *** radeks has joined #tripleo | 05:04 | |
| *** radeks has quit IRC | 05:07 | |
| *** mlupton has joined #tripleo | 05:12 | |
| *** bvandenh has quit IRC | 05:12 | |
| *** absubram has joined #tripleo | 05:13 | |
| *** absubram_ has joined #tripleo | 05:14 | |
| *** fragatina has quit IRC | 05:15 | |
| *** mlupton has quit IRC | 05:16 | |
| *** absubram has quit IRC | 05:18 | |
| *** absubram_ is now known as absubram | 05:18 | |
| *** pmannidi has quit IRC | 05:21 | |
| *** mpsairam has joined #tripleo | 05:24 | |
| *** TicToc has quit IRC | 05:25 | |
| *** TicToc has joined #tripleo | 05:26 | |
| *** mlupton has joined #tripleo | 05:29 | |
| *** pmannidi has joined #tripleo | 05:30 | |
| *** skramaja_ is now known as skramaja | 05:34 | |
| *** bvandenh_ has joined #tripleo | 05:38 | |
| *** dsariel has joined #tripleo | 05:38 | |
| *** rlandy|bbl is now known as rlandy | 05:39 | |
| *** benoit has quit IRC | 05:40 | |
| *** pgadiya has joined #tripleo | 05:40 | |
| *** rcernin has joined #tripleo | 05:43 | |
| *** bana_k has quit IRC | 05:43 | |
| *** benoit has joined #tripleo | 05:44 | |
| *** benoit has quit IRC | 05:49 | |
| *** benoit has joined #tripleo | 05:50 | |
| *** bana_k has joined #tripleo | 05:55 | |
| *** benoit has quit IRC | 05:56 | |
| *** jlinkes has quit IRC | 05:57 | |
| *** jlinkes has joined #tripleo | 05:57 | |
| *** benoit has joined #tripleo | 05:58 | |
| *** benoit has quit IRC | 06:03 | |
| *** benoit has joined #tripleo | 06:05 | |
| *** rajinir has quit IRC | 06:05 | |
| *** benoit has quit IRC | 06:10 | |
| *** benoit has joined #tripleo | 06:11 | |
| *** rcernin has quit IRC | 06:14 | |
| *** benoit has quit IRC | 06:17 | |
| *** benoit has joined #tripleo | 06:18 | |
| *** rcernin has joined #tripleo | 06:19 | |
| *** flepied has quit IRC | 06:19 | |
| *** jprovazn has joined #tripleo | 06:20 | |
| *** rasca has joined #tripleo | 06:20 | |
| *** pcaruana has joined #tripleo | 06:23 | |
| *** benoit has quit IRC | 06:24 | |
| *** benoit has joined #tripleo | 06:25 | |
| *** TicToc has quit IRC | 06:26 | |
| *** absubram has quit IRC | 06:29 | |
| *** fragatina has joined #tripleo | 06:29 | |
| *** aufi has joined #tripleo | 06:29 | |
| *** TicToc has joined #tripleo | 06:30 | |
| *** jlinkes has quit IRC | 06:30 | |
| *** pmannidi has quit IRC | 06:32 | |
| *** fragatina has quit IRC | 06:33 | |
| *** benoit has quit IRC | 06:38 | |
| *** jbadiapa has joined #tripleo | 06:38 | |
| *** benoit has joined #tripleo | 06:38 | |
| *** pmannidi has joined #tripleo | 06:42 | |
| *** jlinkes has joined #tripleo | 06:42 | |
| *** matbu|brb is now known as matbu | 06:43 | |
| *** benoit has quit IRC | 06:44 | |
| *** benoit has joined #tripleo | 06:45 | |
| *** bana_k has quit IRC | 06:45 | |
| *** benoit has quit IRC | 06:51 | |
| *** jtomasek_ has joined #tripleo | 06:52 | |
| *** benoit has joined #tripleo | 06:52 | |
| *** dciabrin|away has quit IRC | 06:55 | |
| *** florianf has joined #tripleo | 06:57 | |
| dtrainor | I have some nodes in ironic that I can't delete. When I try, I'm told: Failed to delete node 77b3a651-b395-4d70-85c4-a7053d725899: Node 77b3a651-b395-4d70-85c4-a7053d725899 is associated with instance bdaf8737-14b3-4ee5-94bb-724407db9df3. (HTTP 409) | 06:57 |
| dtrainor | I don't know which Instance UUID displayed in 'ironic node-list' this is referring to | 06:58 |
| *** benoit has quit IRC | 06:58 | |
| dtrainor | The only thing I could think of which it would be referring to is a deployment or stack in heat - I have neither of those. | 06:59 |
| *** david-lyle_ has joined #tripleo | 07:00 | |
| *** david-lyle has quit IRC | 07:00 | |
| *** benoit has joined #tripleo | 07:00 | |
| *** flepied has joined #tripleo | 07:03 | |
| *** flepied has quit IRC | 07:04 | |
| *** dtantsur|afk is now known as dtantsur | 07:05 | |
| *** flepied has joined #tripleo | 07:06 | |
| *** dciabrin|away has joined #tripleo | 07:09 | |
| *** benoit has quit IRC | 07:12 | |
| *** mcornea has joined #tripleo | 07:12 | |
| *** Jokke_ has quit IRC | 07:13 | |
| *** benoit has joined #tripleo | 07:13 | |
| *** bandini has joined #tripleo | 07:16 | |
| *** dciabrin|away is now known as dciabrin | 07:17 | |
| *** dmacpher has quit IRC | 07:17 | |
| *** rhefner has quit IRC | 07:18 | |
| *** adarazs has quit IRC | 07:19 | |
| *** zoli_gone-proxy has quit IRC | 07:19 | |
| *** cschwede has quit IRC | 07:19 | |
| *** psanchez has quit IRC | 07:20 | |
| *** jmiu has quit IRC | 07:20 | |
| *** marios has quit IRC | 07:20 | |
| *** stevebaker has quit IRC | 07:20 | |
| *** gchamoul has quit IRC | 07:20 | |
| *** tobias_fiberdata has joined #tripleo | 07:20 | |
| *** myoung|bbl has quit IRC | 07:20 | |
| *** jschlueter has quit IRC | 07:21 | |
| *** slagle has quit IRC | 07:21 | |
| *** psanchez has joined #tripleo | 07:21 | |
| *** slagle has joined #tripleo | 07:22 | |
| d0ugal | Morning | 07:22 |
| d0ugal | How is CI lookin'? | 07:22 |
| *** marios has joined #tripleo | 07:22 | |
| *** myoung has joined #tripleo | 07:22 | |
| *** zoli_gone-proxy has joined #tripleo | 07:22 | |
| *** stevebaker has joined #tripleo | 07:23 | |
| *** tobias-fiberdata has quit IRC | 07:23 | |
| *** jmiu has joined #tripleo | 07:23 | |
| shadower | d0ugal: http://tripleo.org/cistatus.html paints a lovely red picture | 07:23 |
| *** adarazs has joined #tripleo | 07:23 | |
| *** jlinkes has quit IRC | 07:23 | |
| *** gchamoul has joined #tripleo | 07:23 | |
| *** jlinkes has joined #tripleo | 07:23 | |
| d0ugal | shadower: yay | 07:24 |
| *** jschlueter has joined #tripleo | 07:24 | |
| *** cschwede has joined #tripleo | 07:25 | |
| *** absubram has joined #tripleo | 07:25 | |
| openstackgerrit | Dougal Matthews proposed openstack/instack-undercloud: Verify that the Deployment Plan creation was successful https://review.openstack.org/369247 | 07:26 |
| *** jpena|off is now known as jpena | 07:26 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Remove the old, deprecated Mistral action names. https://review.openstack.org/366529 | 07:27 |
| *** david-lyle has joined #tripleo | 07:28 | |
| *** david-lyle_ has quit IRC | 07:29 | |
| *** akuznetsov has joined #tripleo | 07:29 | |
| *** TicToc has quit IRC | 07:30 | |
| *** abehl has joined #tripleo | 07:32 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Pass the timeout to the deploy workflow https://review.openstack.org/370186 | 07:34 |
| *** TicToc has joined #tripleo | 07:34 | |
| *** tobias_fiberdata has quit IRC | 07:35 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Remove the environments from Mistral when removing from Swift https://review.openstack.org/369486 | 07:36 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Add an optional timeout when waiting for websocket messages https://review.openstack.org/364252 | 07:36 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Remove the get_hiera_key function https://review.openstack.org/367367 | 07:36 |
| *** pmannidi_ has joined #tripleo | 07:36 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Update the Mistral action names https://review.openstack.org/366519 | 07:37 |
| *** pmannidi has quit IRC | 07:37 | |
| *** jtomasek_ has quit IRC | 07:39 | |
| *** florianf_ has joined #tripleo | 07:39 | |
| d0ugal | When patches are proposed to master, do people propose them to newton at the same time? | 07:40 |
| *** panda|Zz is now known as panda | 07:40 | |
| d0ugal | I ask, because I have been waiting until they are about to merge - but that often means I do it late if they merge when I am not around | 07:40 |
| d0ugal | and then EmilienM beats me to it and I bet he is mad that I'm not doing it :) | 07:40 |
| *** florianf has quit IRC | 07:44 | |
| *** florianf_ is now known as florianf | 07:44 | |
| *** florianf_ has joined #tripleo | 07:44 | |
| marios | d0ugal: yeah depends on the case, but i have in the passed simultanously proposed to stable, but then -1 it so is clear we are waiting for master first | 07:45 |
| marios | past even wow | 07:45 |
| d0ugal | marios: k, thanks - I guess I should start doing that | 07:46 |
| d0ugal | Thanks | 07:46 |
| marios | np | 07:46 |
| *** athomas has joined #tripleo | 07:48 | |
| *** tobias_fiberdata has joined #tripleo | 07:49 | |
| shadower | wait, so any unmerged newton patches must be submitted against the newton branch, too? | 07:50 |
| jaosorior | shadower: for python-tripleo and tripleo-common | 07:50 |
| jaosorior | AFAIK | 07:50 |
| jaosorior | t-h-t and puppet-tripleo are not branched yet | 07:50 |
| d0ugal | shadower: EmilienM has probably been doing all of yours too lol | 07:51 |
| shadower | ah okay | 07:51 |
| * shadower will have a look | 07:51 | |
| marios | shadower: d0ugal you've been emilien'd | 07:51 |
| d0ugal | marios: actually, has tripleo-common been branched? | 07:51 |
| shadower | what about instack-undercloud? | 07:51 |
| d0ugal | now I am confused. | 07:52 |
| d0ugal | Maybe it is just tripleoclient | 07:52 |
| marios | d0ugal: i am not sure.. i thought as jaosorior said, common and pyuthon-tripleo but github can quickly answer your question... | 07:52 |
| shadower | doesn't appear so: https://github.com/openstack/tripleo-common/branches | 07:52 |
| d0ugal | Seems -common doesn't have a newton branch yet. | 07:52 |
| d0ugal | yeah | 07:52 |
| shadower | all I see is liberty and mitaka | 07:52 |
| marios | https://github.com/openstack/instack-undercloud no newton here yet afaics | 07:52 |
| shadower | ya | 07:53 |
| shadower | phew :-) | 07:53 |
| d0ugal | I don't understand why we branched the client only? Isn't that normally the last to be done :/ | 07:53 |
| jaosorior | d0ugal: supposedly libraries come first | 07:53 |
| *** tobias-fiberdata has joined #tripleo | 07:53 | |
| *** florianf_ has quit IRC | 07:53 | |
| d0ugal | jaosorior: oh, I guess that sorta makes sense. I am not aware of anyone using it as a library tho' - so maybe we can remove that status? | 07:54 |
| d0ugal | tripleo-common is more of a library | 07:54 |
| d0ugal | tripleoclient isn't even in global-requirements :) | 07:54 |
| *** tobias_fiberdata has quit IRC | 07:56 | |
| *** yamahata has joined #tripleo | 07:57 | |
| jaosorior | d0ugal: gotta talk to shardy about it I guess | 07:58 |
| d0ugal | Yeah, probably a bit late :) | 07:58 |
| *** hjensas has joined #tripleo | 07:58 | |
| *** shardy has joined #tripleo | 07:58 | |
| *** akuznetsov has quit IRC | 08:00 | |
| *** jpich has joined #tripleo | 08:01 | |
| *** jkraj has joined #tripleo | 08:01 | |
| *** gfidente has joined #tripleo | 08:02 | |
| *** ohamada has joined #tripleo | 08:03 | |
| *** openstackgerrit has quit IRC | 08:03 | |
| *** openstackgerrit has joined #tripleo | 08:04 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Update the Mistral action names https://review.openstack.org/366519 | 08:07 |
| *** ccamacho has joined #tripleo | 08:09 | |
| *** fragatina has joined #tripleo | 08:15 | |
| *** dtantsur is now known as dtantsur|bbl | 08:16 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Add template processing to the update plan workflow. https://review.openstack.org/371027 | 08:17 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Fix the default plan creation https://review.openstack.org/371347 | 08:17 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Return the result of create_plan in create_deployment_plan workflow https://review.openstack.org/371348 | 08:17 |
| openstackgerrit | Dougal Matthews proposed openstack/instack-undercloud: Verify that the Deployment Plan creation was successful https://review.openstack.org/369247 | 08:18 |
| *** fragatina has quit IRC | 08:19 | |
| *** lucas-dinner is now known as lucasagomes | 08:21 | |
| *** yamahata has quit IRC | 08:21 | |
| *** dbecker has quit IRC | 08:24 | |
| *** derekh has joined #tripleo | 08:26 | |
| *** dbecker has joined #tripleo | 08:28 | |
| *** _milan_ has joined #tripleo | 08:30 | |
| *** TicToc has quit IRC | 08:35 | |
| *** shardy has quit IRC | 08:40 | |
| *** TicToc has joined #tripleo | 08:42 | |
| *** saneax-_-|AFK is now known as saneax | 08:42 | |
| *** electrofelix has joined #tripleo | 08:43 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Fix the default plan creation https://review.openstack.org/371347 | 08:47 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Return the result of create_plan in create_deployment_plan workflow https://review.openstack.org/371348 | 08:47 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Add template processing to the update plan workflow. https://review.openstack.org/371027 | 08:47 |
| *** ohamada has quit IRC | 08:57 | |
| *** ohamada_ has joined #tripleo | 08:57 | |
| *** ohamada_ has quit IRC | 08:57 | |
| *** ohamada_ has joined #tripleo | 08:58 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Populate vnc_api_lib.ini on compute nodes with OpenContrail https://review.openstack.org/367497 | 08:58 |
| openstackgerrit | Merged openstack/python-tripleoclient: Updated from global requirements https://review.openstack.org/361875 | 08:59 |
| openstackgerrit | Saravanan KR proposed openstack/os-net-config: Add mac address to the DPDK mapping file https://review.openstack.org/370012 | 09:00 |
| *** tobias-fiberdata has quit IRC | 09:09 | |
| *** ooolpbot has joined #tripleo | 09:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 09:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1624274 | 09:10 |
| *** ooolpbot has quit IRC | 09:10 | |
| openstack | Launchpad bug 1624274 in tripleo "CI: OVB jobs consistently fail to get environments" [Critical,Triaged] | 09:10 |
| *** pmannidi_ has quit IRC | 09:12 | |
| derekh | jistr: fyi, I restarted rabbit on the rh1 controller, I think that has worted the problem, see the comment I added to your bug | 09:13 |
| jistr | derekh: awesome, thanks! | 09:14 |
| derekh | jistr: that was over an hour ago, so we should see some passes soon | 09:15 |
| *** abregman has joined #tripleo | 09:17 | |
| openstackgerrit | Merged openstack/tripleo-ui: Update version to match current release https://review.openstack.org/367587 | 09:17 |
| shadower | derekh: should we start trying rechecks or wait a bit longer? | 09:21 |
| shadower | mine failed but that was about 2 hrs ago | 09:21 |
| derekh | shadower: a couple wouldn't do any harm but I wouldn't go crazy, the jobs are now getting testenvs, to that problem is solved | 09:22 |
| derekh | shadower: but now I'm looking at errors creating the overcloud | 09:22 |
| derekh | 2016-09-16 09:02:45.558651 | 2016-09-16 09:02:42Z [CephStorage]: CREATE_FAILED ResourceInError: resources.CephStorage.resources[0].resources.CephStorage: Went to status ERROR due to "Message: Unknown, Code: Unknown" | 09:22 |
| shadower | oh that's a fantastic error explanation | 09:23 |
| * shadower has two +Ad patches that are waiting for a gate pass for days now. I'll try reverifying one | 09:23 | |
| derekh | 2016-09-16 09:03:15.000 | | eac4cf4b-23fa-4333-bd3c-813c30832378 | overcloud-cephstorage-0 | ERROR | - | NOSTATE | | | 09:24 |
| derekh | well that would do it | 09:24 |
| *** sshnaidm has quit IRC | 09:25 | |
| *** shardy has joined #tripleo | 09:26 | |
| tbarron | marios: when you get a chance, https://review.openstack.org/#/c/366760/7 needs a little syntax fix | 09:27 |
| *** sshnaidm has joined #tripleo | 09:27 | |
| marios | tbarron: thanks ack gimme 2 mins will sort it out | 09:27 |
| tbarron | marios: np, i know you are doing many things :) | 09:28 |
| *** dtantsur|bbl is now known as dtantsur | 09:28 | |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-ui: Migrate Deploy action to Mistral https://review.openstack.org/357125 | 09:37 |
| openstackgerrit | Saravanan KR proposed openstack/os-net-config: Add mac address to the DPDK mapping file https://review.openstack.org/370012 | 09:39 |
| jaosorior | aww... still seeing failures getting environment in ovb :( | 09:48 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Unset Keystone public_endpoint https://review.openstack.org/368969 | 09:48 |
| openstackgerrit | Merged openstack/python-tripleoclient: Use the hexdigest of the path to make the filename unique in swift https://review.openstack.org/369859 | 09:48 |
| openstackgerrit | Merged openstack/tripleo-quickstart: Remove external requirements https://review.openstack.org/370264 | 09:49 |
| *** tobias-fiberdata has joined #tripleo | 09:50 | |
| derekh | jaosorior: can you point me at one | 09:53 |
| openstackgerrit | Merged openstack/python-tripleoclient: Add `openstack overcloud plan deploy` https://review.openstack.org/360305 | 09:53 |
| jaosorior | derekh: http://logs.openstack.org/93/322893/20/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/8a18fa3/console.html | 09:55 |
| jaosorior | derekh: and http://logs.openstack.org/93/322893/20/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/5b5ff43/console.html | 09:55 |
| *** tosky has joined #tripleo | 09:56 | |
| derekh | jaosorior: thanks | 09:56 |
| *** akrivoka has joined #tripleo | 09:58 | |
| derekh | jaosorior: 2 different errors http://paste.openstack.org/show/579309/ http://paste.openstack.org/show/579310/ looking | 09:59 |
| openstackgerrit | Marios Andreou proposed openstack/puppet-tripleo: Add manila-netapp backend to manila class and tidy up generic https://review.openstack.org/354014 | 09:59 |
| openstackgerrit | Marios Andreou proposed openstack/puppet-tripleo: Fixup manila-cephfs native backend defaults https://review.openstack.org/366760 | 10:00 |
| *** zoli_gone-proxy is now known as zoliXXL | 10:03 | |
| openstackgerrit | Merged openstack/python-tripleoclient: Tripleoclient leaks temporary files https://review.openstack.org/330638 | 10:05 |
| *** tobias_fiberdata has joined #tripleo | 10:08 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Update the Mistral action names https://review.openstack.org/366519 | 10:08 |
| *** bvandenh_ has quit IRC | 10:08 | |
| d0ugal | derekh: The second is the error we had before in CI | 10:08 |
| d0ugal | derekh: we now know that to be https://bugs.launchpad.net/mistral/+bug/1624284 /cc therve | 10:09 |
| openstack | Launchpad bug 1624284 in Mistral "MessagingTimeout when executing mistral actions" [Undecided,Confirmed] | 10:09 |
| *** kbyrne has quit IRC | 10:09 | |
| *** ooolpbot has joined #tripleo | 10:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 10:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1624274 | 10:10 |
| *** tobias-fiberdata has quit IRC | 10:10 | |
| *** ooolpbot has quit IRC | 10:10 | |
| openstack | Launchpad bug 1624274 in tripleo "CI: OVB jobs consistently fail to get environments" [Critical,Triaged] | 10:10 |
| derekh | d0ugal: the second error I pasted ? that is an error on the RH1 cloud controller, not in a CI job | 10:10 |
| d0ugal | derekh: This one: http://paste.openstack.org/show/579310/ | 10:10 |
| d0ugal | derekh: oh :) | 10:10 |
| therve | Yeah it doesn't seem to come from mistral | 10:10 |
| d0ugal | sorry, I read that too quickly | 10:11 |
| derekh | d0ugal: no prob | 10:11 |
| *** kbyrne has joined #tripleo | 10:11 | |
| *** TicToc has quit IRC | 10:11 | |
| marios | gfidente: revisit please when yuo get a chance https://review.openstack.org/#/c/354014/ | 10:12 |
| openstackgerrit | Merged openstack/instack-undercloud: Introduce 'enable_validations' option https://review.openstack.org/322893 | 10:16 |
| *** zoliXXL is now known as zoli|lunch | 10:18 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Pass the timeout to the deploy workflow https://review.openstack.org/370186 | 10:19 |
| derekh | ok, I think all of the env stacks that failed overnight are causing extra load on things as heat had been trying to delete them (and their many resources) but failing, | 10:19 |
| derekh | cleaning things up now | 10:19 |
| *** abregman has quit IRC | 10:21 | |
| *** skramaja has quit IRC | 10:22 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Add an optional timeout when waiting for websocket messages https://review.openstack.org/364252 | 10:22 |
| openstackgerrit | Florian Fuchs proposed openstack/tripleo-ui: Migrate Deploy action to Mistral https://review.openstack.org/357125 | 10:23 |
| d0ugal | therve: So, you know how we create the default plan at install time? | 10:24 |
| therve | d0ugal, Somewhat | 10:24 |
| d0ugal | therve: The only reason that CI has been working, is because that fails | 10:24 |
| d0ugal | https://review.openstack.org/#/c/371347/ | 10:24 |
| d0ugal | As soon as I fix it, the problem comes back | 10:24 |
| therve | So we really need a quick solution in mistral | 10:25 |
| d0ugal | Yeah. | 10:25 |
| d0ugal | therve: Any ideas? :) | 10:25 |
| d0ugal | therve: and I assume since we are seeing this in CI consistently - that there is a good chance users will hit it | 10:26 |
| therve | Yeah it's a fundamental issue. Mistral doesn't hit it in its gate because there is not enough testing | 10:27 |
| tbarron | marios: i deleted my overcloud stack, virt-customized the overcloud-full image again with https://review.openstack.org/#/c/366760/8 this tiime, and attempted to redeploy but hit http://paste.fedoraproject.org/428757/14740209/ immediately - and 'heat stack-list' shows empty. | 10:27 |
| tbarron | marios: i guess there is an issue with re-deploys? I nuked all my vms and started with freshly updated packages and git clones this morning and | 10:28 |
| therve | d0ugal, Using anything but the threading executor would make the CI happy, I believe | 10:28 |
| therve | Whether or not it's correct in the general is another issue | 10:28 |
| tbarron | marios: can do that again, but just want to check real quick if there's a less drastic approach to picking up the latest update | 10:29 |
| marios | tbarron: not sure what happened with that paste... i was going to say perhaps the heat stac wasn't gone yet by te time you started a redeploy, so it thought it was updating? | 10:29 |
| d0ugal | therve: I guess we could also do something like this: http://paste.openstack.org/show/579317/ Very pseudocode-y, but hopefully makes sense. | 10:31 |
| tbarron | marios: well, i did 'heat stack-delete'; 'watch heat stack-list' till i saw it deleted, then picked up the latest manila.pp, virt-customized overcloud, uploaded to glance, and re-deployed. so the heat stack should have been gone. | 10:31 |
| d0ugal | but that would be a big change for us at this point. | 10:31 |
| therve | Yeah. And mistral would still be broken | 10:32 |
| d0ugal | therve: :) | 10:32 |
| tbarron | marios: and there are of course no overcloud.yaml and overcloud-without-mergepy.yaml in my THT, as i menationed everything was built fresh this morning | 10:33 |
| tbarron | marios: i dunno OOO well enough to tell who expects them to be there | 10:33 |
| marios | tbarron: well looks like it may be a client issue d0ugal do you have any idea what the issue is? tbarron deplyoed, deleted the stack, updated images and tries to deploy again but gets http://paste.fedoraproject.org/428757/14740209/ | 10:35 |
| *** thrash|g0ne is now known as thrash | 10:35 | |
| marios | d0ugal: is it stale plan data or something? looks like is getting /trying and failing to get something from swift? | 10:35 |
| * marios caffeinne brb | 10:36 | |
| tbarron | marios: dougal exactly, and note in the paste messages about removing current and uploading new plan files. I didn't see those on the first deploy atempt. | 10:36 |
| tbarron | dobson: marios the first deploy *did* have a 404 for overcloud-without-mergepy.yaml though | 10:37 |
| tbarron | s/dobson/dougal/ - sorry dobson | 10:37 |
| tbarron | ah, d0ugal ^^^^ didn't see the '0', sorry | 10:38 |
| tbarron | d0ugal: marios so on the first deploy attempt there wasn't the stuff about removiing current plan and updating new plan, which makes sense | 10:39 |
| tbarron | d0ugal: marios but on the first deploy there *was* a 404 for overcloud-without-mergepy.yaml but no 404 for overcloud.yaml | 10:40 |
| tbarron | d0ugal: marios it just got that one 404 and kept on chugging | 10:40 |
| d0ugal | tbarron: I think that error can actually be ignored. | 10:41 |
| tbarron | d0ugal: on the second deploy, as you see in http://paste.fedoraproject.org/428757/14740209/ there is a second series of 404s, for both of the old overcloud*.yaml files | 10:41 |
| d0ugal | tbarron: It's related to a change I think shardy made - it it looking for both the files in swift - we should change it to be more clearly debug info because everyone is asking about it :) | 10:41 |
| tbarron | d0ugal: well i did ignore it on the first deploy, on the second I paid attention because the deploy stopped instead of continuing on | 10:41 |
| d0ugal | therve: did it stop with an error? | 10:42 |
| d0ugal | tbarron: oh, I see | 10:42 |
| tbarron | d0ugal: line 32 in that paste was the last i saw | 10:42 |
| d0ugal | tbarron: That is odd. | 10:42 |
| tbarron | d0ugal: that's why i was inclined to blame the 404s | 10:42 |
| d0ugal | Yeah, I am more inclined to take them seriously now :) | 10:43 |
| * d0ugal looks at the code | 10:43 | |
| tbarron | d0ugal: and the fact that i didn't see the second set of 404s, for both the old overcloud*yaml fiiles, the firs time, only the single 404 for overcloud-without-mergepy.yaml | 10:43 |
| marios | d0ugal: thanks ... tbarron i wouldn't nuke my env yet.. i mean yes you should be able to do this (redeploy) fine ! hanve't come across the issue you're seeing here before though. | 10:45 |
| d0ugal | tbarron: yeah, so this is where it is happening: https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L438-L449 | 10:45 |
| marios | d0ugal: could/would manually deleting the plan help? would a new one just be created on next attempt? | 10:45 |
| d0ugal | marios: It might help and yes it would | 10:46 |
| d0ugal | marios, tbarron: openstack overcloud plan delete overcloud | 10:46 |
| *** adarazs is now known as adarazs_lunch | 10:47 | |
| tbarron | dobson: marios done - and 'openstack overcloud plan list' shows empty - now just re-deploy? | 10:49 |
| marios | tbarron: yeah see what happens.. also sanity check all the env files etc you are passing | 10:49 |
| marios | tbarron: ironic nodes all available and no heat stack right? | 10:50 |
| tbarron | d0ugal: ^^ (i'm a slow learner, did your nick wrong again) | 10:50 |
| d0ugal | tbarron: lol, sorry for being awkward :) | 10:50 |
| d0ugal | tbarron: Yeah, just redeploy | 10:50 |
| d0ugal | I think we need to change this plan management stuff, causing too many problems | 10:51 |
| d0ugal | I really need some help with it from somebody that understands Heat better | 10:51 |
| d0ugal | We have this bug which is related to it: https://bugs.launchpad.net/tripleo/+bug/1622683 | 10:51 |
| openstack | Launchpad bug 1622683 in tripleo "Updating plans breaks deployment" [Critical,In progress] - Assigned to Dougal Matthews (d0ugal) | 10:51 |
| tbarron | 'heat stack-list' is empty, 'ironic node-list' shows all 'baremetals' available | 10:52 |
| tbarron | k, here goes | 10:52 |
| tbarron | it's creatiing a new swift container for the plan, that looks better :) | 10:53 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Fix the default plan creation https://review.openstack.org/371347 | 10:54 |
| tbarron | 404 on overclous-without-mergepy.yaml but it is contiinuting on, like my first deploy | 10:54 |
| tbarron | d0ugal: marios thanks, over that hump and I didn't nuke everything! | 10:54 |
| d0ugal | tbarron: np, sorry for the plan related issues :( | 10:55 |
| marios | tbarron: AND its friday \o/ | 10:55 |
| marios | d0ugal: thanks :) | 10:55 |
| * tbarron observes that in OOO it feels so good to quit hitting ones head on wall | 10:55 | |
| d0ugal | lol | 10:55 |
| marios | haha | 10:55 |
| tbarron | ^^^ couldn't resist, that's true everywhere of course | 10:56 |
| d0ugal | tbarron: we require extra head hitting so that when you get beyong that it feels even better! | 10:56 |
| tbarron | d0ugal: rofl | 10:56 |
| marios | tbarron: sorry for the pain and it really is as bad as it gets, i mean with the puppet-tripleo so you need to inject the images too... i.e. not just grab templates and go | 10:57 |
| marios | tbarron: please keep fighting :) we are waiting for you to tell us it works | 10:58 |
| marios | tbarron: once we hear that i think we could land the series today or whenever we start landing things again | 10:58 |
| jaosorior | tbarron: or you could use swift to update the puppet manifest on the next overcloud deploy | 10:58 |
| marios | tbarron: we have at least one +2 everywhere i think now | 10:59 |
| jaosorior | tbarron: http://hardysteven.blogspot.fi/2016/08/tripleo-deploy-artifacts-and-puppet.html | 10:59 |
| tbarron | marios: i will; i know that there are a lot of changes in flight and big infra stuff in OOO has happened this cycle, so i understand | 10:59 |
| *** tobias_fiberdata has quit IRC | 11:00 | |
| tbarron | marios: i will declare victory just as soon as I can, minimum viable product is great with me. I know rc1 is imminent :) | 11:00 |
| tbarron | jaosorior: thanks,, something to read during the deploy attempt :) | 11:00 |
| marios | tbarron: ack... yeah as longs as it works, we can tidy up or add stuff easily once base is in. plus we are dealing with a general backend tidy up AND adding netapp and cephfs backends so is a lot there already. | 11:01 |
| tbarron | jaosorior: that looks cool, the next optimizations after virt-customize for faster workflow | 11:04 |
| marios | tbarron: regarding rc1 yes... i think we got a reprieve with the ci issues yesterday so would be awesome to get it in, otherwise we start to risk not landing (still couple more weeks but .... would be nice not to have to start backporting everything) | 11:05 |
| *** ccamacho is now known as ccamacho|lunch | 11:05 | |
| tbarron | marios: ack | 11:05 |
| gfidente | marios, https://review.openstack.org/#/c/370830/1 here you mean basically replace true with an echo? | 11:09 |
| *** lucasagomes is now known as lucas-hungry | 11:09 | |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1624274 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1624274 in tripleo "CI: OVB jobs consistently fail to get environments" [Critical,Triaged] | 11:10 |
| marios | gfidente: well only if we care... that was the question | 11:10 |
| marios | gfidente: if we don't care if it pass/fail then it is fine like that | 11:10 |
| gfidente | marios, thing is the directory might not exist at all | 11:12 |
| *** tobias_fiberdata has joined #tripleo | 11:12 | |
| gfidente | marios, let's use an echo, I'd have to recheck it anywa | 11:12 |
| gfidente | thanks :) | 11:12 |
| marios | jistr: can you see comment at https://review.openstack.org/#/c/357192/4 - ill revote on the others too if need be (I was +2 on all 3 of those reverts) | 11:14 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Fixes the Ceph upgrade scripts https://review.openstack.org/370830 | 11:14 |
| jaosorior | derekh: hey dude, could you check this commit out? https://review.openstack.org/#/c/370623/ | 11:16 |
| marios | gfidente: nice thanks voted | 11:16 |
| marios | tbarron: btw sounds like rc1 is bumped so you can go back to one spoon of coffee for now ;) thought you'd be please to hear http://lists.openstack.org/pipermail/openstack-dev/2016-September/103690.html | 11:23 |
| * tbarron pulls the syringe back out of his vein | 11:23 | |
| marios | lol intravenous caffeinne is serious dedication | 11:24 |
| jaosorior | marios: can you check this out? https://review.openstack.org/#/c/370029/ | 11:33 |
| *** paramite has joined #tripleo | 11:36 | |
| *** pgadiya has quit IRC | 11:38 | |
| *** fultonj has joined #tripleo | 11:38 | |
| jaosorior | shadower, jistr hey guys, if you have time can you check this commit out? https://review.openstack.org/#/c/370577/ | 11:38 |
| marios | jaosorior: ack | 11:39 |
| derekh | jolooking at it now, so is the ssl job broken? | 11:41 |
| d0ugal | tbarron: You'll be happy to hear I am hitting the swift 404 errors in CI :) | 11:42 |
| jistr | jaosorior: lgtm | 11:43 |
| jaosorior | derekh: So, the current stuff works. But when trying to test zaqar websocket's behind HAProxy, which actually takes proxies into account, it keeps failing. Hoping that does the trick, cause it works locally (on different deployments) | 11:43 |
| derekh | jaosorior: ok | 11:43 |
| tbarron | d0ugal: well, i guess it's good not to feel alone, but sorry about CI | 11:44 |
| EmilienM | hello | 11:44 |
| derekh | As if CI wasn't bad enough /me has just delete 19 random ports from neutron on the RH1 overcloud by accident | 11:44 |
| d0ugal | EmilienM: Morning | 11:44 |
| tbarron | marios: so now we hit a weird mongodb error, certainly unrelated to your changes: http://paste.fedoraproject.org/428796/14740257/ | 11:45 |
| *** tobias_fiberdata has quit IRC | 11:45 | |
| derekh | EmilienM: howdy, testenvs were failing to get created overnight, I've been cleaning thing up so I think they are in better shape now | 11:46 |
| tbarron | marios: Error: /Stage[main]/Tripleo::Profile::Base::Database::Mongodb/Mongodb_replset[tripleo]: Could not evaluate: rs.add() failed to add host to replicaset tripleo: replSetReconfig command must be sent to the current replica set primary.\u001b[0m\n" | 11:46 |
| EmilienM | derekh: thanks a lot | 11:46 |
| *** abregman has joined #tripleo | 11:46 | |
| tbarron | marios: unfort it's early enough that manila.conf isn't getting updated and no attempt to start manila services: http://paste.fedoraproject.org/428803/40261511/ so i can't confirm that your patches are working yet | 11:47 |
| marios | tbarron: wow that entire paste has only one instance of 'error'... not seen that before... yeah it would be before the cluster services are brought up | 11:47 |
| marios | tbarron: i mean i am not sure why you are seeing that... may be worth browsing the recent bugs for tripleo? | 11:48 |
| tbarron | marios: i wonder if anyone else is seeing that mongodb deploy error | 11:48 |
| derekh | EmilienM: at least I'm starting to seem some ovb jobs passing now, although I may have just cause some to fail by deleting some neutron ports I shouldn't have deleted | 11:48 |
| tbarron | marios: yeah, i'll browse | 11:48 |
| *** masco has quit IRC | 11:49 | |
| *** jpena is now known as jpena|lunch | 11:49 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Cleanup the previous plan when deploying https://review.openstack.org/371468 | 11:51 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Fix the default plan creation https://review.openstack.org/371347 | 11:51 |
| *** egafford has joined #tripleo | 11:51 | |
| derekh | I'm off for the weekend before things start to go downhill http://chunk.io/f/08744a6e6a8544d1a063c93c65420c71 | 11:52 |
| d0ugal | derekh: nice! | 11:52 |
| *** trown|outtypewww is now known as trown | 11:53 | |
| *** adarazs_lunch is now known as adarazs | 11:53 | |
| jaosorior | lol | 11:54 |
| jaosorior | awesome | 11:54 |
| EmilienM | derekh: can we close https://bugs.launchpad.net/tripleo/+bug/1624274 ? | 11:56 |
| openstack | Launchpad bug 1624274 in tripleo "CI: OVB jobs consistently fail to get environments" [Critical,Triaged] | 11:56 |
| derekh | EmilienM: we can downgrade it I think, will come up with a more permanent solution later | 11:57 |
| *** derekh changes topic to "TripleO : http://tripleo.org/ | https://wiki.openstack.org/wiki/TripleO | Meetings On Tuesdays at 14:00 UTC in #openstack-meeting-alt" | 11:57 | |
| EmilienM | ok | 11:57 |
| EmilienM | done | 11:58 |
| derekh | EmilienM: ack | 11:58 |
| d0ugal | gah, ipv6 is super annoying :) | 11:59 |
| derekh | bnemec: slagle bug 1624274 , I think we need something to consume the messages from ceilometer's queue, otherwise the whole thing just grows | 11:59 |
| openstack | bug 1624274 in tripleo "CI: OVB jobs consistently fail to get environments" [High,In progress] https://launchpad.net/bugs/1624274 | 11:59 |
| derekh | bnemec: slagle and I think causes use problems, as it must be slowing down the other queues | 11:59 |
| *** dprince has joined #tripleo | 12:00 | |
| slagle | derekh: i thought we'd shut down ceilometer? | 12:00 |
| slagle | why do we need it | 12:00 |
| derekh | bnemec: dprince slagle: yes thats the problem, things are writing to the queues that ceilometer normally consums, but nothing is consuming the messages | 12:00 |
| derekh | dprince: RE. https://bugs.launchpad.net/tripleo/+bug/1624274 | 12:01 |
| openstack | Launchpad bug 1624274 in tripleo "CI: OVB jobs consistently fail to get environments" [High,In progress] | 12:01 |
| EmilienM | derekh, slagle, dprince : fyi https://review.openstack.org/#/c/371157/ | 12:01 |
| EmilienM | derekh: ah, we should maybe stop to send events on this queue | 12:01 |
| slagle | derekh: k, i understand now | 12:02 |
| slagle | we had stopped it due to the high load on the controller | 12:02 |
| EmilienM | or set a low ttl to the messages | 12:02 |
| derekh | slagle: yup | 12:02 |
| dprince | derekh: so we need ceilometer then? | 12:02 |
| *** jayg|g0n3 is now known as jayg | 12:03 | |
| derekh | dprince: I think either turn back on ceilo or cron job to pruge the notify queues or the ttl EmilienM suggested | 12:04 |
| *** ccamacho|lunch is now known as ccamacho | 12:04 | |
| EmilienM | shardy: have you seen the failures on https://review.openstack.org/#/c/365796/ ? | 12:04 |
| dprince | derekh: we can leave it on then for now | 12:04 |
| EmilienM | shardy: sounds like transient | 12:04 |
| derekh | EmilienM: that tripleo-cd-admin file is now deprecated, I'm not sure if its used anywhere any longer | 12:05 |
| derekh | EmilienM: you should add it here too http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/tripleo-cd-admins | 12:05 |
| dprince | derekh: we disabled it as part of the services we thought we didn't use | 12:05 |
| EmilienM | derekh: ok I will | 12:05 |
| EmilienM | shardy: what is worries me is https://review.openstack.org/#/c/353506/ - failures on scenario003 look valid | 12:05 |
| shardy | EmilienM: looking now - http://logs.openstack.org/96/365796/14/check/gate-tripleo-ci-centos-7-nonha-multinode/6fbb84e/console.html is confusing as it actually looks like the job worked fine | 12:06 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Add Emilien Macchi ssh key to TripleO admins https://review.openstack.org/371484 | 12:07 |
| shardy | EmilienM: Yes, the fluentd patch failures do look real | 12:08 |
| shardy | I'll see if I can see where the problem is | 12:08 |
| slagle | derekh: which queues in rabbit? notifications.info? | 12:08 |
| shardy | there's a syntax error somewhere in the service template outputs | 12:08 |
| jistr | EmilienM, shardy: also the HA job failure might be valid -- 31mError: /Stage[main]/Tripleo::Profile::Base::Database::Mongodb/Mongodb_replset[tripleo]: Could not evaluate: Can't find master host for replicaset tripleo | 12:08 |
| EmilienM | shardy: right | 12:08 |
| jistr | http://logs.openstack.org/96/365796/14/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/8d79c0f/logs/overcloud-controller-0/var/log/messages | 12:08 |
| jistr | i just got an env up, i'll try to deploy with that patch locally | 12:09 |
| openstackgerrit | Martin André proposed openstack/puppet-tripleo: Manage tripleo-ui configuration files with puppet https://review.openstack.org/363167 | 12:11 |
| *** lucas-hungry is now known as lucasagomes | 12:11 | |
| slagle | EmilienM: did anything get added in our jobs to attempt to collect logs in post_test_hook? | 12:12 |
| slagle | EmilienM: b/c we have some jobs failing at that step, even though the pingtest succeeded: | 12:12 |
| slagle | http://logs.openstack.org/48/363748/6/check/gate-tripleo-ci-centos-7-nonha-multinode/998a939/console.html | 12:12 |
| d0ugal | jtomasek: If I have a plan in tripleo-ui and I want to update it, how do I do that? | 12:13 |
| d0ugal | jtomasek: Delete and re-create? | 12:13 |
| d0ugal | jtomasek: (update the templates etc.) | 12:13 |
| EmilienM | slagle: no we haven't anything added | 12:13 |
| *** abehl has quit IRC | 12:14 | |
| *** anshul_ has joined #tripleo | 12:14 | |
| EmilienM | i think we are unlucky, log collections was close to the timeout limit | 12:14 |
| EmilienM | hmm no, 1h13 is not too bad | 12:15 |
| *** mbound has quit IRC | 12:15 | |
| slagle | actually, i see the FAILURE earlier | 12:15 |
| slagle | maybe coming from postci? | 12:15 |
| jtomasek | d0ugal: we currently add/overwrite files in swift | 12:15 |
| *** mbound has joined #tripleo | 12:15 | |
| EmilienM | 2016-09-16 11:00:50.811542 | Job timeout set to: 95 minutes | 12:16 |
| tbarron | jistr: EmilienM shardy that mongodb replicaset error you cite looks like what I hit with deploy attempt with fresh packages and git clones this morning: http://paste.fedoraproject.org/428796/14740257/ | 12:16 |
| d0ugal | jtomasek: oh, fun :) | 12:16 |
| jistr | tbarron: hmm yea... the error message isn't exactly the same, but it's very similar | 12:17 |
| EmilienM | looking at http://logs.openstack.org/48/363748/6/check/gate-tripleo-ci-centos-7-nonha-multinode/998a939/logs/postci.log | 12:17 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Add CephRgw to roles_data.yaml https://review.openstack.org/370687 | 12:17 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Use osd_pool_default_* puppet parameters when creating the pools https://review.openstack.org/370270 | 12:18 |
| jistr | ok so i'll try to first deploy HA without any modifications | 12:18 |
| EmilienM | slagle: it could be a multinode/infra issue with networking | 12:18 |
| *** zoli|lunch is now known as zoli | 12:18 | |
| shardy | larsks: Hey, looks like an error crept into https://review.openstack.org/#/c/353506 somewhere | 12:19 |
| slagle | EmilienM: yea i see an error in http://logs.openstack.org/48/363748/6/check/gate-tripleo-ci-centos-7-nonha-multinode/998a939/_zuul_ansible/ansible_log.txt | 12:19 |
| jtomasek | d0ugal: when I try to deploy with latest tripleo-heat-templates I am getting Failed to validate nested template: Property error: resources[9].properties: Property KeystoneCredential0 not assigned | 12:19 |
| shardy | larsks: I also added a question re the snmp credentials | 12:19 |
| slagle | 2016-09-16 11:00:17,775 p=14535 u=zuul | fatal: [node]: FAILED! => {"failed": true, "msg": "Failed to connect to the host via ssh."} | 12:19 |
| EmilienM | yeah | 12:20 |
| EmilienM | probably a network issue with bluebox cloud | 12:20 |
| openstackgerrit | Dmitry Tantsur proposed openstack/instack-undercloud: Fix nova-related deprecation warnings https://review.openstack.org/371495 | 12:20 |
| d0ugal | jtomasek: I suspect you will hit similar plan updating problems if you have enough testing :( | 12:20 |
| slagle | EmilienM: let's go with that :) | 12:20 |
| EmilienM | jtomasek: I wrote that code | 12:20 |
| d0ugal | jtomasek: I've not seen that before, sounds like a parameter isn't set? | 12:20 |
| EmilienM | jtomasek: the property is generated in tripleoclient | 12:20 |
| EmilienM | https://review.openstack.org/#/q/status:merged+topic:keystone/credentials | 12:20 |
| d0ugal | jtomasek: it is a new one: https://github.com/openstack/python-tripleoclient/commit/5f0694a64efcf6527306e8507efa92d03d60a05a | 12:21 |
| EmilienM | it's even backported ! | 12:21 |
| d0ugal | EmilienM: :) | 12:21 |
| d0ugal | jtomasek: I'd check with rbrady to see how he is getting on with the password generation stuff | 12:22 |
| d0ugal | I am not sure that is going to make Newton at this point :/ | 12:22 |
| jtomasek | EmilienM, d0ugal : is it going to get into a Mistral action? is Ryan working on that? | 12:22 |
| d0ugal | jtomasek: Yeah, it should be part of his general password generation stuff | 12:22 |
| d0ugal | jtomasek: because that didn't exist yet we had to let people add more to tripleoclient. | 12:23 |
| jtomasek | d0ugal: I'd say it is a blocker for GUI then | 12:23 |
| *** pradk has joined #tripleo | 12:23 | |
| d0ugal | jtomasek: Sure, but a blocker doesn't make it any easier to do :) | 12:23 |
| d0ugal | jtomasek: https://review.openstack.org/#/c/368150/ | 12:23 |
| d0ugal | jtomasek: That is the start of it, but it doesn't update the CLI | 12:23 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Port password generation from tripleoclient to tripleo-common https://review.openstack.org/368150 | 12:24 |
| dprince | pabelanger: could we try this late this afternoon? https://review.openstack.org/#/c/357308/4 | 12:24 |
| dprince | pabelanger: if it breaks then revert over the weekend? | 12:25 |
| *** rhallisey has joined #tripleo | 12:25 | |
| EmilienM | dprince: I'm doing a recheck on it | 12:25 |
| EmilienM | to see if it pass now ;-) | 12:25 |
| d0ugal | EmilienM: chances are we are going to hit that mistral timeout error against soon | 12:25 |
| therve | d0ugal, The mistral patch seems to have worked, no? | 12:26 |
| EmilienM | Aug 25 is last year for me in CI world | 12:26 |
| EmilienM | d0ugal: why? | 12:26 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Add mongo config settings in collector service templates https://review.openstack.org/370426 | 12:26 |
| therve | At least we don't see that error anymore | 12:26 |
| d0ugal | therve: oh, I hadn't seen result yet | 12:26 |
| d0ugal | therve: right, yeah, it did | 12:26 |
| d0ugal | therve: but should they accept it as is? | 12:26 |
| d0ugal | therve: I don't really know. | 12:26 |
| therve | d0ugal, No idea :) | 12:27 |
| slagle | EmilienM: https://review.openstack.org/#/c/357308/ isnt tested by CI | 12:27 |
| d0ugal | EmilienM: https://bugs.launchpad.net/mistral/+bug/1624284 | 12:27 |
| openstack | Launchpad bug 1624284 in Mistral "MessagingTimeout when executing mistral actions" [Undecided,Confirmed] | 12:27 |
| dprince | EmilienM: I'm not 100 sure a recheck would test that actually | 12:27 |
| slagle | EmilienM: we'd have to deploy the script onto the te-broker | 12:27 |
| d0ugal | EmilienM: therve found how to reproduce it, we have not fixed anything. So I guess any user with the same setup could well hit it too | 12:27 |
| d0ugal | EmilienM: and the only reason we don't hit it is because something else is broken | 12:27 |
| dprince | exactly, we have to install it manually | 12:27 |
| d0ugal | EmilienM: CI was never really fixed, just broken quietly :) | 12:28 |
| * d0ugal -> lunch and dog walk | 12:29 | |
| EmilienM | dprince: ok, so we can maybe approve it. /me just making sure we don'tr break CI again | 12:29 |
| EmilienM | d0ugal: wait | 12:31 |
| EmilienM | d0ugal: where do you test https://review.openstack.org/#/c/371435/ in tripleO? | 12:31 |
| therve | EmilienM, https://review.openstack.org/#/c/371347/ | 12:31 |
| EmilienM | therve: thx | 12:32 |
| dprince | EmilienM: sure, we might even test it in place first | 12:32 |
| EmilienM | sighs at mistral :( please don't break us during a release | 12:32 |
| slagle | dprince: EmilienM : which ceileometer service should we start up on rh1 to clear the queue? openstack-ceilometer-collector? | 12:34 |
| dprince | slagle: maybe just restart all the ones that were running before | 12:34 |
| dprince | slagle: we stopped them to save some CPU power | 12:34 |
| slagle | dprince: i'm not sure how i would tell which ones were running before | 12:35 |
| tbarron | jistr: if you decide that the mongodb replset 'can't find master' and 'replSetReconfig command must be sent to the current replica set primary' are likely at root the same problem and want to look at my deployment, ping me as I'll likely leave my beaker machine in that state until i find a way to unblock | 12:36 |
| osp | hi, can anyone provide details on how i can get an ldap backend configured in heat during a tripleo deployment? In my parameters i can supply parameters to keystone.conf but can't seem to get a file create within /etc/keystone/domains | 12:36 |
| slagle | pradk: which ceilometer service consumes events from the notifications.info rabbit queue? | 12:36 |
| gfidente | therve, can you give a look at https://review.openstack.org/#/c/370127/ and see if you can spot anything wrong with my comments there? | 12:37 |
| gfidente | looks like we should be able to use batch_create and rolling_update updating the template | 12:37 |
| pradk | slagle, collector | 12:38 |
| therve | Looking | 12:38 |
| slagle | pradk: can i start just collector? or does it need other ceilometer services? | 12:39 |
| slagle | pradk: backstory is that we disabled ceilometer services in our cloud due to cpu usage, but now we are seeing the notifications.info queue fill up | 12:39 |
| slagle | and we think that's causing a bottleneck | 12:39 |
| therve | gfidente, So 1) Resources aren't tied to template versions | 12:39 |
| pradk | slagle, yea if the services are continue to publish it will fill up quickly | 12:39 |
| slagle | we don't actually care about the messages, just want to clear the queue | 12:40 |
| therve | gfidente, 2) batch_create is not a property, it's an update policy key | 12:40 |
| gfidente | therve, dah, that's why then | 12:40 |
| gfidente | so it goes | 12:40 |
| gfidente | update_policy: | 12:40 |
| gfidente | batch_create: | 12:40 |
| pradk | slagle, mongo and gnocchi still running ? | 12:40 |
| gfidente | max_batch_size: 1 | 12:40 |
| dprince | slagle: I just ran this: http://paste.openstack.org/show/579977/ | 12:40 |
| gfidente | ? | 12:40 |
| *** noslzzp has joined #tripleo | 12:40 | |
| shadower | jaosorior: could you have a look at https://review.openstack.org/#/c/362194/ ? +2 and the gate passes O_o | 12:40 |
| gfidente | therve, maybe you can comment there how to use it? | 12:41 |
| pradk | slagle, so collector will clear up that queue but that has to go somewhere.. which is by default to mongo and gnocchi for events and metrics respectively | 12:41 |
| therve | gfidente, Yep looks about right. | 12:41 |
| dprince | slagle: anything else you think we should enable? | 12:41 |
| gfidente | therve, thanks! | 12:41 |
| slagle | pradk: k, mongod is up | 12:41 |
| pradk | slagle, so long as they are up, it should do it | 12:41 |
| slagle | dprince: don't think so. i see the queue going down | 12:41 |
| *** noslzzp has quit IRC | 12:42 | |
| slagle | cpu load was high, but it was rabbitmq that was consuming a lot, so maybe once the queue is empty, it will settle some | 12:42 |
| *** flepied has quit IRC | 12:42 | |
| *** noslzzp has joined #tripleo | 12:42 | |
| slagle | actually, the load is high b/c of "mprime" | 12:43 |
| slagle | did we leave a benchmark running? :) | 12:43 |
| shadower | jaosorior: thanks! | 12:43 |
| pradk | slagle, can you check if the notification agent is running.. you probably will need that up too | 12:43 |
| slagle | pradk: openstack-ceilometer-notification? it'sup | 12:44 |
| slagle | it's up | 12:44 |
| *** rlandy has joined #tripleo | 12:44 | |
| pradk | slagle, cool | 12:44 |
| slagle | bnemec: do you have mprime running on the rh1 overcloud controller? | 12:45 |
| *** masco has joined #tripleo | 12:45 | |
| *** fultonj_ has joined #tripleo | 12:45 | |
| jaosorior | gfidente: hey dude, could you check this commit out? https://review.openstack.org/#/c/370577/ | 12:48 |
| EmilienM | matbu: do you have any progress on upgrade testing? | 12:48 |
| EmilienM | panda: do you have progress on ipv6 testing? | 12:48 |
| *** pkovar has joined #tripleo | 12:49 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Add ipv6 nic-configs https://review.openstack.org/364479 | 12:49 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Fix use of batch_create in CephMon major upgrade template https://review.openstack.org/370127 | 12:49 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Add IPv6 network configuration for ipv6 job types https://review.openstack.org/363674 | 12:49 |
| *** Goneri has joined #tripleo | 12:50 | |
| jistr | tbarron: hmm btw i didn't seem to hit any problem on stack-create... | 12:50 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Fix use of batch_create in CephMon major upgrade template https://review.openstack.org/370127 | 12:52 |
| tbarron | jistr: well, maybe i'll retry then. the only changes I have are for manila, in puppet-tripleo and THT, but naively I don't see how they would have anything to do with mongodb replset issues | 12:52 |
| gfidente | shadower, can you vote on https://review.openstack.org/#/c/370127/4 if we got it right ? | 12:52 |
| gfidente | therve, ^^ added you there as well | 12:53 |
| shadower | gfidente: did you really mean me? That's the first time I'm seeing that change | 12:53 |
| gfidente | shadower, yes I did :) | 12:53 |
| tbarron | jistr: and fresh packages and complete rebuild from vms on up this morniing ... | 12:54 |
| matbu | EmilienM: i'm working on upgrade bugs, but still blocking on underclod install hanging, any help would be welcome .. i think it's infra related | 12:54 |
| therve | gfidente, You can only have one of the 2 keys | 12:54 |
| matbu | EmilienM: but i have probably set something wrong that cause thishanging | 12:54 |
| gfidente | therve, ack -1 please | 12:54 |
| therve | Sure | 12:54 |
| EmilienM | matbu: do you have logs? have you reported a bug? | 12:54 |
| jistr | tbarron: hmm yea i've also rebuilt from scratch today (both undercloud and overcloud) | 12:54 |
| gfidente | therve, though seems problematic | 12:54 |
| gfidente | on the first upgrade we don't have that resource so we'll want batch_create | 12:55 |
| jistr | tbarron: could be that we have something intermittent perhaps, we'll see now that we have CI running if there are some jobs that will hit this problem still | 12:55 |
| gfidente | on a further update the rolling_update was meant to have same effect | 12:55 |
| matbu | EmilienM: yep, but no very useful (log i mean, i asked sshnaidm and slagle for help) | 12:55 |
| gfidente | therve, can we get it to do both create/update always on one at a time? | 12:55 |
| *** tzumainn has joined #tripleo | 12:56 | |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates: Add fluentd client service https://review.openstack.org/353506 | 12:56 |
| *** hjensas has quit IRC | 12:57 | |
| therve | gfidente, I don't know, I don't think so | 12:57 |
| gfidente | therve, auch! | 12:58 |
| gfidente | srsly? :) | 12:58 |
| therve | zaneb, Do you know? | 12:58 |
| EmilienM | larsks: it's weird scenario003 failed, scenario003 deploys sahara (scenario002 was working with cinder) | 12:58 |
| larsks | shardy, that was a rebase mistake; thanks for catching it. | 12:58 |
| gfidente | it's kind of a show stopper :) | 12:58 |
| *** jpena|lunch is now known as jpena | 12:58 | |
| *** jcoufal_ has joined #tripleo | 12:58 | |
| larsks | EmilienM, if only I could get an overcloud deploy not to fall over early on locally... :/ | 12:58 |
| larsks | d0ugal, do you know off the top of your head if those bugs I was hitting yesterday have a resolution yet? | 12:59 |
| therve | gfidente, Hum no I'm wrong, sorry | 12:59 |
| zaneb | rolling update only creates/updates up to batch_size resources at a time | 12:59 |
| zaneb | was that the question? | 12:59 |
| therve | You'd better test that, though. More than making sure that the template validates :) | 12:59 |
| therve | zaneb, So you can specify both batch_create and rolling_update in the policy? | 13:00 |
| zaneb | I believe so. batch_create affects only the original creation of the group | 13:00 |
| therve | (side note, that's an horrible interface, but don't mind me :)) | 13:01 |
| zaneb | rolling_updates affects only subsequent changes to the group | 13:01 |
| zaneb | therve: blah blah historical reasons... | 13:01 |
| therve | gfidente, So forget me, that fix looks good :) | 13:01 |
| gfidente | therve, zaneb yeah the expected behaviour we want is what zaneb described | 13:02 |
| gfidente | cause on the initial upgrade of tirpleo the resource doesn't exist, so it goes into create mode | 13:02 |
| gfidente | on further attempts it goes into update mode | 13:02 |
| gfidente | but we always want 1 by 1 | 13:02 |
| *** lblanchard has joined #tripleo | 13:03 | |
| *** jaosorior has quit IRC | 13:03 | |
| *** jaosorior has joined #tripleo | 13:04 | |
| zaneb | does SoftwareDeploymentGroup have both of those policies? | 13:04 |
| therve | Hopefully by inheritance | 13:05 |
| zaneb | apparently it does | 13:05 |
| * zaneb checks code | 13:05 | |
| zaneb | therve: it didn't used to have either, so it's not picking it up just from inheritance | 13:06 |
| therve | zaneb, Not sure what you mean | 13:06 |
| zaneb | in fact it can't, because you can't specify min_in_service on a SoftwareDeploymentGroup | 13:06 |
| gfidente | it inherits ResourceGroup | 13:08 |
| zaneb | https://review.openstack.org/gitweb?p=openstack/heat.git;a=commitdiff;h=5465579bdf378f8731737c07dc8832cc4466e776 | 13:08 |
| therve | Oh, it overrides the schema | 13:08 |
| zaneb | it didn't support it at all before that, despite inheriting from ResourceGroup | 13:09 |
| zaneb | interestingly, there is a bug there | 13:09 |
| therve | zaneb, Because update_policy_schema was overriden? | 13:09 |
| zaneb | because it actually *doesn't* override the schema when it should | 13:10 |
| zaneb | therve: yes | 13:10 |
| zaneb | so min_in_service is included and a lot of the code in that patch is dead http://docs.openstack.org/developer/heat/template_guide/openstack.html#OS::Heat::SoftwareDeploymentGroup-prop-rolling_update-min_in_service | 13:10 |
| therve | Why does it need to override it? | 13:11 |
| *** tobias_fiberdata has joined #tripleo | 13:11 | |
| *** ayoung_ has joined #tripleo | 13:11 | |
| therve | Oh of course | 13:11 |
| zaneb | therve: because it defines (but does not use) a different schema for rolling_update | 13:11 |
| therve | Because there is no way to use the proper copy of rolling_update_schema | 13:11 |
| therve | Sigh | 13:11 |
| * zaneb raises bug | 13:11 | |
| therve | Thanks | 13:11 |
| zaneb | at least it landed in Newton! | 13:12 |
| zaneb | we can fix in rc2 ;) | 13:12 |
| therve | :) | 13:12 |
| gfidente | zaneb, therve thanks guys :) | 13:12 |
| gfidente | we can probably vote on https://review.openstack.org/#/c/370127/4 anyway | 13:12 |
| gfidente | as it fixes the syntax error anyway | 13:13 |
| gfidente | ? | 13:13 |
| zaneb | gfidente: yeah, just reviewed it | 13:13 |
| zaneb | indentation is messed up, but otherwise its fine | 13:13 |
| therve | gfidente, So talking about template, https://review.openstack.org/#/c/307838/2/network/ports/external_from_pool_v6.yaml | 13:13 |
| therve | gfidente, Where does ExternalPort come from? | 13:13 |
| gfidente | zaneb, ack updating there | 13:13 |
| gfidente | therve, sec | 13:14 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Fix use of batch_create in CephMon major upgrade template https://review.openstack.org/370127 | 13:14 |
| tobias_fiberdata | overcloud plan list, what plan is this exactly? and what does it do? | 13:14 |
| derekh | slagle: notifications.error http://paste.openstack.org/show/579982/ , I guess there is an error somewhere populating that too | 13:16 |
| gfidente | therve, yeah seems wrong | 13:16 |
| tobias_fiberdata | dpeloying and getting stuck on this: Uploading new plan files | 13:17 |
| tobias_fiberdata | , could someone guide me? | 13:17 |
| therve | gfidente, There is a bunch of those in that patch | 13:17 |
| gfidente | therve, well external_v6 does create the ExternalPort resource | 13:18 |
| gfidente | therve, it's the _from_pool which need fixing :( | 13:18 |
| gfidente | therve, thanks | 13:18 |
| gfidente | therve, curious how you spotted it? | 13:19 |
| therve | gfidente, I validated all templates in the repo | 13:19 |
| *** Jokke_ has joined #tripleo | 13:19 | |
| therve | gfidente, http://paste.openstack.org/show/579984/ | 13:19 |
| jrist | random question | 13:22 |
| jrist | I filed a bug | 13:22 |
| jrist | it says | 13:22 |
| jrist | Please if you are a developer, self-triage the bug. We do not need to wait for another developer to confirm that this is a bug. | 13:22 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Set client protocol for glance registry client https://review.openstack.org/370029 | 13:23 |
| jrist | does that just mean setting the importance? | 13:23 |
| jrist | or more? | 13:23 |
| openstackgerrit | Merged openstack/tripleo-incubator: Allow Emilien Macchi to be root on TripleO Cloud https://review.openstack.org/371157 | 13:23 |
| zaneb | therve: https://review.openstack.org/#/c/371536/ | 13:24 |
| therve | zaneb, Should we try to test this... | 13:24 |
| *** akshai has joined #tripleo | 13:25 | |
| trown | jrist: importance, and also the "triaged" flag | 13:25 |
| jrist | oh | 13:25 |
| jrist | trown: see that's what I needed to know :) thanks | 13:25 |
| trown | jrist: targeting to a milestone is also good | 13:25 |
| jrist | did that | 13:25 |
| zaneb | therve: there were unit tests already | 13:25 |
| therve | zaneb, Not the useful kind apparently | 13:26 |
| jrist | trown: confirmed or triaged? | 13:26 |
| zaneb | therve: well, it supported too *much*. it's hard to test for that | 13:26 |
| trown | jrist: I set triaged for bugs I file, or confirmed on bugs I am looking at that someone else filed | 13:26 |
| trown | not sure it matters though | 13:26 |
| therve | zaneb, I guess :) | 13:26 |
| jrist | nice | 13:26 |
| zaneb | therve: because for any given feature there are an infinite number of things it isn't intended to do ;) | 13:27 |
| therve | zaneb, I'd be ok if it'd just tested min_in_services though :p | 13:27 |
| *** akshai_ has joined #tripleo | 13:27 | |
| *** mbound has quit IRC | 13:28 | |
| jpich | jrist: Seems the preference in TripleO is Triaged, according to https://wiki.openstack.org/wiki/TripleO#Bug_Triage | 13:29 |
| jrist | jpich: thanks for the clarification | 13:29 |
| jrist | might need a launchpad patch to point to that | 13:30 |
| jrist | :) | 13:30 |
| *** akshai has quit IRC | 13:30 | |
| jpich | heh | 13:30 |
| trown | oh TIL | 13:32 |
| trown | will stop using confirmed all together | 13:32 |
| *** flepied has joined #tripleo | 13:35 | |
| openstackgerrit | Marios Andreou proposed openstack/puppet-tripleo: Add manila-netapp backend to manila class and tidy up generic https://review.openstack.org/354014 | 13:37 |
| *** mgarciam has joined #tripleo | 13:37 | |
| shadower | Folks, can we merge this? https://review.openstack.org/#/c/368621/ It will generate better validations docs and exercise the publish-docs job :-) | 13:39 |
| shadower | jaosorior ^ ? | 13:39 |
| openstackgerrit | Marios Andreou proposed openstack/puppet-tripleo: Fixup manila-cephfs native backend defaults https://review.openstack.org/366760 | 13:40 |
| *** limao has joined #tripleo | 13:42 | |
| *** rodrigods has quit IRC | 13:45 | |
| *** rodrigods has joined #tripleo | 13:45 | |
| jrist | jtomasek: how do I test https://review.openstack.org/#/c/367993/5 in the UI? | 13:45 |
| jrist | jtomasek: I have it pulled down | 13:46 |
| jrist | but I'm not sure what changed | 13:46 |
| jrist | perhaps that's the point? :) | 13:46 |
| jrist | nevermind | 13:46 |
| jrist | the little pencil? | 13:46 |
| jtomasek | jrist: yep | 13:47 |
| jrist | I just get infinite spinner | 13:47 |
| jrist | https://paste.fedoraproject.org/428884/33658147/ | 13:47 |
| gfidente | d0ugal, can you point to a commit in tripleoclient older than the one calling mistral action to do the jinja compiling? | 13:50 |
| gfidente | a version which can be used without overcloud.j2.yaml | 13:50 |
| jtomasek | jrist: hmpf, no idea, I'll look into it when I get back | 13:51 |
| jrist | jtomasek: yeah sounds good. thanks | 13:51 |
| jrist | jtomasek: ping me when you're back | 13:51 |
| openstackgerrit | Merged openstack/tripleo-validations: Generate documentation for validations https://review.openstack.org/368621 | 13:52 |
| *** zephcom has left #tripleo | 13:52 | |
| gfidente | d0ugal, looks like up to 8th of sept? | 13:53 |
| gfidente | https://github.com/openstack/python-tripleoclient/commit/817f84d04337419b5697ebee7ab469dd3eca30c3 | 13:53 |
| tobias_fiberdata | slagle, shardy, could anyone of you help out? We are running newton undercloud and deploying our overcloud, and it says "Uploading new plan files | 13:54 |
| tobias_fiberdata | ", what exactly does that mean? | 13:54 |
| panda | EmilienM: I didn't go past the swift error, I was testing primarily with experimental jobs ... | 13:55 |
| *** saneax is now known as saneax-_-|AFK | 13:55 | |
| shardy | tobias_fiberdata: we copy the template files from the local filesystem into a swift container (same name as the overcloud you're deploying), and a mistral environmeent (again named e.g "overcloud") | 13:56 |
| *** anshul_ has quit IRC | 13:56 | |
| shardy | tobias_fiberdata: combined those two things are referred to as a "plan" | 13:56 |
| shardy | because it contains all the stuff needed to deploy an overcloud | 13:56 |
| tobias_fiberdata | shardy, ah okey, so basically it does that on the undercloud node? | 13:56 |
| tobias_fiberdata | that explains why it takes ages | 13:56 |
| shardy | tobias_fiberdata: Yeah | 13:56 |
| shardy | shouldn't take that long, few seconds perhaps | 13:57 |
| tobias_fiberdata | our undercloud aint the fastest thing in the world :P | 13:57 |
| shardy | aha | 13:57 |
| tobias_fiberdata | well then something is wrong | 13:57 |
| tobias_fiberdata | cause i've started this 10mins ago | 13:57 |
| shardy | Yeah, that's defintely wrong, it's just copying a few files via some API calls | 13:57 |
| tobias_fiberdata | do you have any clue if there's any logs about this? | 13:58 |
| jaosorior | thrash: I think this is needed for the zaqar websocket stuff to work in CI https://review.openstack.org/#/c/370623/ | 13:59 |
| shardy | tobias_fiberdata: I'd check the mistral logs /var/log/mistral/mistral-server.log | 13:59 |
| shardy | sounds like something went wrong but the error wasn't reported to the client | 13:59 |
| shardy | that happens via zaqar, so ensure the zaqar services are running OK | 14:00 |
| gfidente | tobias_fiberdata, I have seen sometimes errors in mistral trying to reach zaqar on the wrong endpoint | 14:00 |
| gfidente | defaulting to localhost:someport instead of the websockets endpoint | 14:00 |
| tobias_fiberdata | looks like ZaqarAction.queue_post failed: <class 'requests.exceptions.ConnectionError'>: HTTPConnectionPool(host='10.16.31.2', | 14:00 |
| tobias_fiberdata | okey gfidente | 14:00 |
| gfidente | tobias_fiberdata, though in your case looks like it's going to the right socket? | 14:01 |
| *** cdearborn has joined #tripleo | 14:01 | |
| tobias_fiberdata | [Errno 111] ECONNREFUSED',)) | 14:01 |
| tobias_fiberdata | it says this aswell | 14:01 |
| tobias_fiberdata | perhaps it's blocking something? | 14:01 |
| *** coolsvap has quit IRC | 14:02 | |
| gfidente | which port is it going to? | 14:02 |
| gfidente | can you compare that with endpoint list of undercloud? | 14:02 |
| tobias_fiberdata | sec | 14:02 |
| *** jcoufal__ has joined #tripleo | 14:03 | |
| tobias_fiberdata | do you want the websocket one? | 14:04 |
| derekh | bnemec: I went into sysctl.conf for persist the conntract settiong we set last friday and found this http://paste.openstack.org/show/580013/ | 14:04 |
| gfidente | tobias_fiberdata, well you just want to make sure mistral is trying to reach zaqar on the right ip:port so compare those | 14:05 |
| tobias_fiberdata | http://paste.openstack.org/show/580018/ | 14:05 |
| derekh | bnemec: so I've bumped up the timouts again (not quite as high as they were), will keep and eye on it and persist what ever we finish up on | 14:05 |
| *** jlinkes has quit IRC | 14:05 | |
| *** jcoufal_ has quit IRC | 14:06 | |
| *** jlinkes has joined #tripleo | 14:06 | |
| jistr | EmilienM, shardy, tbarron: FYI i was able to reproduce the MongoDB replicaset problem (same message as in CI, different than what tbarron posted). It's most probably some kind of intermittent issue / race condition, as running the same puppet replset resource for the 2nd time worked just fine. | 14:07 |
| gfidente | tobias_fiberdata, and to which one of the two is mistral going? | 14:07 |
| openstackgerrit | Merged openstack/instack-undercloud: Deploy validations SSH key in post config https://review.openstack.org/362194 | 14:07 |
| tobias_fiberdata | gonna check the logs | 14:07 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Fixes the Ceph upgrade scripts https://review.openstack.org/370830 | 14:07 |
| jistr | i'll at least report it now, so far i don't see what would be the cause | 14:07 |
| gfidente | tobias_fiberdata, I think we only launch the websocket one | 14:07 |
| gfidente | so you probably don't have anything on 8888 | 14:07 |
| tobias_fiberdata | ZaqarAction.queue_post failed: <class 'requests.exceptions.ConnectionError'>: HTTPConnectionPool(host='10.16.31.2', port=8888 | 14:07 |
| d0ugal | larsks: I don't think they do yet - not sure. | 14:07 |
| tobias_fiberdata | ah | 14:07 |
| *** [1]cdearborn has joined #tripleo | 14:07 | |
| gfidente | yeah | 14:08 |
| d0ugal | gfidente: Yeah, that sounds about right. | 14:08 |
| d0ugal | gfidente: The 8th, it was recent[ | 14:08 |
| gfidente | d0ugal, ack, thanks! | 14:08 |
| gfidente | tobias_fiberdata, so that's not good, I think mistral is supposed to look for the zaqar-websocket endpoint | 14:08 |
| jaosorior | d0ugal: is that so? Thought we used the HTTP API from zaqar. At least mistral should be using it to create queues from there. And subsequently use the websocket endpoint to actually use that queue | 14:09 |
| tobias_fiberdata | gfidente, we installed this undercloud machine yesterday | 14:09 |
| gfidente | from master? | 14:09 |
| tobias_fiberdata | gfidente, | 14:09 |
| tobias_fiberdata | gfidente, not sure | 14:10 |
| gfidente | using tripleo.sh ? | 14:10 |
| *** jaosorior has quit IRC | 14:10 | |
| tobias_fiberdata | we used the latest repos because mitaka was not working very well for us | 14:10 |
| tobias_fiberdata | not tripleo.sh no | 14:10 |
| tobias_fiberdata | tripleo.org/docs | 14:10 |
| tobias_fiberdata | sorry this | 14:11 |
| tobias_fiberdata | docs.openstack.org/developer/tripleo-docs/ | 14:11 |
| *** ramishra has quit IRC | 14:11 | |
| *** hjensas has joined #tripleo | 14:11 | |
| *** hjensas has joined #tripleo | 14:11 | |
| openstackgerrit | Miles Gould proposed openstack/instack-undercloud: Update undercloud.conf.sample https://review.openstack.org/371567 | 14:12 |
| openstackgerrit | Miles Gould proposed openstack/instack-undercloud: Enable introspection of UEFI nodes by default https://review.openstack.org/371568 | 14:12 |
| gfidente | tobias_fiberdata, so I think this could be an issue in tripleo-common | 14:12 |
| tobias_fiberdata | gfidente, can we do an upgrade on the undercloud machine and hope that is correcting it to those ports it's supposed to be? | 14:13 |
| *** ramishra has joined #tripleo | 14:13 | |
| tobias_fiberdata | i mean if there's any change from yesterday | 14:13 |
| tobias_fiberdata | hehe | 14:13 |
| gfidente | right I would remove the three delorean .repo files from /etc/yum.repos.d | 14:13 |
| gfidente | re-curl those (as per docs) | 14:13 |
| gfidente | and try a tripleo-common update | 14:13 |
| tobias_fiberdata | aye | 14:13 |
| tobias_fiberdata | we'll give it a shot | 14:13 |
| tobias_fiberdata | thanks alot gfidente | 14:14 |
| gfidente | good luck :) | 14:14 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Cleanup the previous plan when deploying https://review.openstack.org/371468 | 14:16 |
| slagle | bnemec: do we need the mprime process running on the rh1 overcloud? | 14:17 |
| *** tobias-fiberdata has joined #tripleo | 14:17 | |
| slagle | it's eating up cpu | 14:17 |
| jrist | florianf: did you pull down 367993 too or just visual review? | 14:17 |
| jrist | florianf: curious if you're having the issue I'm having http://paste.openstack.org/show/580022/ | 14:17 |
| *** [4]cdearborn has joined #tripleo | 14:18 | |
| bnemec | slagle: I shut it off. It was niced, so it shouldn't have been taking priority over anything else, but it sounds like we found the issue. | 14:18 |
| slagle | bnemec: ok | 14:19 |
| slagle | wasn't sure :). i just saw it using a lot of cpu | 14:19 |
| slagle | when i checked this morning | 14:19 |
| bnemec | That was my attempt to get the cpu to stop scaling down without being able to actually get into the bios and change the setting. | 14:19 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Fix _from_pool_v6.yaml str_split https://review.openstack.org/371576 | 14:20 |
| *** tobias_fiberdata has quit IRC | 14:20 | |
| EmilienM | jistr: ok, any solution? | 14:21 |
| *** cdearborn has quit IRC | 14:21 | |
| florianf | jrist: I pulled it down | 14:21 |
| jistr | EmilienM: no, i don't know yet why the puppet module fails on that. When re-run, it succeeds, it must be something temporary. | 14:21 |
| jrist | florianf: did it load ok for you? | 14:22 |
| jrist | like the panel had content? | 14:22 |
| florianf | jrist: yes, both panels have content | 14:23 |
| *** fultonj has quit IRC | 14:23 | |
| florianf | jrist: I don't see that error | 14:23 |
| tobias-fiberdata | gfidente, there was 2 newer packages | 14:23 |
| jrist | :( | 14:23 |
| tobias-fiberdata | for tripleo-common | 14:23 |
| *** fultonj_ is now known as fultonj | 14:23 | |
| tobias-fiberdata | so we'll try that | 14:23 |
| jrist | florianf: I wonder what I'm missing | 14:23 |
| *** hjensas has quit IRC | 14:23 | |
| florianf | jrist: updating changes (to the services for a role) works for me too | 14:24 |
| jrist | well I can't do that | 14:24 |
| jrist | because the panel fails | 14:24 |
| jrist | spinner then error in console | 14:24 |
| jrist | honza: merge conflict https://review.openstack.org/370775 | 14:25 |
| jrist | florianf: can I get a +2 on https://review.openstack.org/#/c/370537/ | 14:25 |
| jrist | and a +1 workflow plzzzzzz <3 | 14:25 |
| honza | jrist: yes | 14:25 |
| shardy | tobias-fiberdata: note that if tripleo-common changed, you may need to either re-run openstack undercloud install, or manually refresh the mistral actions/workflows (depending on what changed in the update) | 14:25 |
| shardy | http://paste.openstack.org/show/580026/ | 14:25 |
| florianf | jrist: oh, I thought I already did that... | 14:26 |
| shardy | that's the manual approach, which copies what the undercloud install does internally | 14:26 |
| florianf | jrist: +2'ed | 14:26 |
| jrist | thx flaper87 | 14:26 |
| jrist | asldkf | 14:26 |
| jrist | thanks florianf | 14:26 |
| jrist | florianf: it's bugging me because I test on two machines | 14:26 |
| bnemec | slagle: derekh: Did you re-enable ceilometer too? I had disabled all the stuff that we shut off. | 14:26 |
| jrist | florianf: thank youuuu | 14:26 |
| florianf | jrist: yeah, good idea to change that | 14:27 |
| derekh | bnemec: I havn't touched it at all, we could just call the rabbit command to clear the queue every hour | 14:27 |
| jrist | it's just for dev but | 14:27 |
| *** mbound has joined #tripleo | 14:27 | |
| jrist | it's useful | 14:27 |
| slagle | bnemec: probably not. dprince started them | 14:29 |
| *** mbound has quit IRC | 14:31 | |
| *** mbound has joined #tripleo | 14:31 | |
| *** [1]cdearborn has quit IRC | 14:32 | |
| openstackgerrit | Merged openstack/tripleo-ui: Have dev server listen everywhere instead of just local https://review.openstack.org/370537 | 14:34 |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 14:36 |
| *** bnemec has quit IRC | 14:37 | |
| openstackgerrit | John Trowbridge proposed openstack-infra/tripleo-ci: POC DO NOT MERGE: Add virt-setup option to tripleo.sh https://review.openstack.org/371587 | 14:39 |
| trown | panda: ^ | 14:39 |
| panda | trown: finally! :) looking | 14:40 |
| trown | lol | 14:40 |
| EmilienM | panda: my question was about experimental job, did you make progress on making it pass? Does it deploy ipv6 correctly? | 14:42 |
| openstackgerrit | Merged openstack/puppet-tripleo: Fix wrong flag name for VNC Proxy in HAProxy https://review.openstack.org/370552 | 14:44 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Use osd_pool_default_* puppet parameters when creating the pools https://review.openstack.org/370270 | 14:45 |
| panda | EmilienM: I released a patch for the swift problem, made it dependent on the ha-ipv6 patch for tripleo-ci, then was waiting for CI to stabilize again. I launched another check now on that patch. | 14:47 |
| *** jlinkes has quit IRC | 14:48 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/puppet-tripleo: Add swift proxy for ceilometer middleware https://review.openstack.org/371591 | 14:48 |
| *** jcoufal__ has quit IRC | 14:48 | |
| *** bnemec has joined #tripleo | 14:49 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Remove openstackclient imports in the new parameters command https://review.openstack.org/371594 | 14:50 |
| openstackgerrit | Jiri Stransky proposed openstack/puppet-tripleo: Wait for MongoDB connections before creating replset https://review.openstack.org/371596 | 14:50 |
| EmilienM | panda: here ? https://review.openstack.org/#/c/363674/ | 14:51 |
| EmilienM | panda: the depends-on is https://review.openstack.org/#/c/363204/ | 14:51 |
| jistr | EmilienM: ^^ here's an attempt for the MongoDB fix, but given that the issue is intermittent, only time can tell if it works or not | 14:51 |
| EmilienM | and it's not swift it's nova | 14:51 |
| EmilienM | jistr: nice!! | 14:51 |
| EmilienM | jistr: looking | 14:51 |
| openstackgerrit | Markos Chandras proposed openstack/diskimage-builder: elements: opensuse: Add support for openSUSE Leap https://review.openstack.org/371602 | 14:53 |
| panda | EmilienM: https://review.openstack.org/369976 | 14:53 |
| panda | EmilienM: and I have to remove that Depends-On on my first patch, since it is on the same project, it doesn't do anything | 14:54 |
| *** athomas has quit IRC | 14:54 | |
| EmilienM | panda: you should rather do the other way around imho | 14:55 |
| EmilienM | so you can have multiple depends-on in tripleo-ci patch | 14:55 |
| panda | EmilienM: ok, I would probably have arrived at this solution at the third patch on the experimental ipv6 job | 14:56 |
| panda | EmilienM: but I'll invert after these results | 14:56 |
| EmilienM | thanks | 14:57 |
| EmilienM | panda: we really need to make progress on this thing | 14:57 |
| EmilienM | panda: we can't releae newton without ipv6 support | 14:57 |
| bnemec | panda: EmilienM: Swift is broken in ipv6 though: https://bugs.launchpad.net/tripleo/+bug/1623672 | 14:57 |
| openstack | Launchpad bug 1623672 in tripleo "Swift failing to deploy in ipv6" [Critical,In progress] - Assigned to Ben Nemec (bnemec) | 14:57 |
| bnemec | My first patch resulted in broken hieradata, and I haven't been able to figure out the right puppet to fix it. | 14:58 |
| bnemec | Although I haven't had a lot of time to look into it the past couple of days either. | 14:58 |
| openstackgerrit | Miles Gould proposed openstack/instack-undercloud: Enable introspection of UEFI nodes by default https://review.openstack.org/371568 | 14:58 |
| panda | bnemec: that is the same error I'm trying to fix here https://review.openstack.org/369976 | 14:59 |
| bnemec | panda: Ah, cool. | 15:00 |
| *** jcoufal_ has joined #tripleo | 15:03 | |
| *** athomas has joined #tripleo | 15:05 | |
| panda | EmilienM: any suggestion on how to speed up this process even when CI is down ? | 15:06 |
| shardy | alembic.script.revision.RevisionError: Requested revision 4b47ea298795 overlaps with other requested revisions d6a12e637e28 | 15:06 |
| shardy | hrm, anyone seen that trying to upgrade neutron on the undercloud? | 15:07 |
| openstackgerrit | Markos Chandras proposed openstack/diskimage-builder: elements: opensuse: Add support for openSUSE Leap https://review.openstack.org/371602 | 15:13 |
| *** jkraj has quit IRC | 15:13 | |
| *** dtantsur is now known as dtantsur|pto | 15:13 | |
| *** rcernin has quit IRC | 15:15 | |
| therve | d0ugal, Is there a list of tripleoclient command that happens during a ci run? | 15:16 |
| *** jkraj has joined #tripleo | 15:19 | |
| shardy | therve: CI uses tripleo.sh, which calls tripleoclient: | 15:19 |
| shardy | https://github.com/openstack-infra/tripleo-ci/blob/master/scripts/tripleo.sh#L550 | 15:19 |
| shardy | so you can look at that and see how its called, does that help? | 15:19 |
| shardy | you can also run that script locally with same/similar inputs to CI | 15:19 |
| therve | shardy, And things like plan create? | 15:20 |
| shardy | therve: openstack overcloud deploy does a plan create internally, we don't yet explicitly test the seperated steps of create plan, deploy plan | 15:21 |
| shardy | we'll need to do that soon tho | 15:21 |
| therve | Hum okay | 15:21 |
| EmilienM | panda: which process? | 15:21 |
| *** bnemec has quit IRC | 15:22 | |
| jpich | therve: "undercloud install" should be creating a plan with the default templates as well | 15:22 |
| panda | EmilienM: moving forward with the experimental ipv6 job | 15:23 |
| therve | jpich, Ahah, that's the one I was missing, thanks | 15:23 |
| jpich | Yw! | 15:23 |
| therve | jpich, It doesn't wait for the plan to be created though :/ | 15:24 |
| *** aufi has quit IRC | 15:24 | |
| therve | That sounds like it may be an issue | 15:24 |
| EmilienM | panda: moving forward like how? | 15:24 |
| *** bnemec has joined #tripleo | 15:25 | |
| panda | EmilienM: make progress | 15:25 |
| *** ramishra has quit IRC | 15:25 | |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Update README about running a full end to end deployment https://review.openstack.org/369659 | 15:25 |
| EmilienM | panda: when CI is down, we can't make any progress | 15:25 |
| jpich | therve: It sounds like it could be yeah... and there's also an issue with creating the default plan atm that means it's not being created at all | 15:25 |
| jpich | (I think d0ugal has a patch up for that, just hit this on a brand new undercloud locally and about to test it) | 15:26 |
| therve | Cool, I'm making progress understanding this though | 15:27 |
| * therve bbl | 15:27 | |
| *** ramishra has joined #tripleo | 15:27 | |
| EmilienM | panda, bnemec also https://bugs.launchpad.net/tripleo/+bug/1605363 | 15:28 |
| openstack | Launchpad bug 1605363 in tripleo "[Newton] ipv6 HA deployments are currently broken" [Critical,Triaged] | 15:28 |
| EmilienM | bandini: have you an update about ^ ? | 15:28 |
| EmilienM | is it fixed ? | 15:28 |
| bandini | EmilienM: I put it on my todo to test it with master. won't get to it today though | 15:29 |
| EmilienM | bandini: ok, please let me know, it sounds quite critical | 15:30 |
| *** jkraj has quit IRC | 15:30 | |
| bandini | EmilienM: will do, thanks for checking | 15:30 |
| *** matbu is now known as matbu|brb | 15:38 | |
| jpich | therve: And thank you for that! ( https://review.openstack.org/#/c/371347/ does resolve the particular issue I hit fwiw) | 15:39 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Convert deploy steps to jinja2 loop https://review.openstack.org/365796 | 15:40 |
| *** [4]cdearborn has quit IRC | 15:42 | |
| *** ebalduf has joined #tripleo | 15:43 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Add mongo config settings in collector service templates https://review.openstack.org/370426 | 15:48 |
| EmilienM | heh, mitaka jobs are green again with https://review.openstack.org/#/c/371029/ | 15:48 |
| EmilienM | now liberty | 15:49 |
| *** jcoufal__ has joined #tripleo | 15:49 | |
| marios | EmilienM: gfidente revote please when you get a chance https://review.openstack.org/#/c/354014/ thanks | 15:49 |
| *** electrofelix has quit IRC | 15:50 | |
| *** jcoufal_ has quit IRC | 15:50 | |
| EmilienM | marios: -2 | 15:50 |
| gfidente | marios, I told you indetation would kill you | 15:50 |
| gfidente | that's why they invented python | 15:51 |
| gfidente | no? | 15:51 |
| marios | :( | 15:51 |
| gfidente | I like enforced indentation | 15:51 |
| gfidente | not the -lint jobs | 15:51 |
| marios | you know i had to fight to even get rake lint to pass locally | 15:51 |
| marios | i mean had to install stuff, not done it before | 15:52 |
| gfidente | yeah for me it wasn't installing a couple of gems | 15:52 |
| gfidente | the other day | 15:52 |
| gfidente | bundle install | 15:52 |
| marios | (or lately, gems in general haven't touched for loong time) | 15:52 |
| EmilienM | marios: wow, you installed lint locally? | 15:52 |
| EmilienM | you ok? :) | 15:52 |
| marios | EmilienM: no, i mean it wasn't passing 'rake lint' | 15:53 |
| marios | EmilienM: had to install the right dependencies (gems) to get it to work, bundler helped in the end | 15:53 |
| EmilienM | bundle install? | 15:53 |
| EmilienM | that all you need ;) | 15:53 |
| gfidente | yeah when it passes | 15:54 |
| marios | EmilienM: well i also had to manually install puppet for some reason | 15:54 |
| ayoung | dprince, BTW, EmilienM 's changes to get Credentials initialized have all landed. I confirmed they worked last night. We should be able to un-peg Keystone now | 15:54 |
| marios | EmilienM: there was a dependency issue so bundle install didn't pass | 15:54 |
| marios | EmilienM: perhaps these things are just easier for you ;) | 15:54 |
| EmilienM | marios: nothing is easy man | 15:54 |
| *** benoit has quit IRC | 15:54 | |
| marios | EmilienM: gfidente thanks guys | 15:55 |
| ayoung | also, the keystone upstream changed such that there is a null key used during the migration process to deal with keys, so the breakage was removed. I understand why they did what they did, but was able to convince them it was not something we could accept. | 15:55 |
| ayoung | either way, we should be able to unpeg Keystone from N3 | 15:55 |
| EmilienM | marios: anytime | 15:55 |
| marios | gfidente: please readd here too https://review.openstack.org/#/c/366760/ | 15:55 |
| marios | EmilienM: if you have time ^^^ related one | 15:56 |
| EmilienM | ayoung: it's already unpin ... | 15:56 |
| ayoung | EmilienM, excellent | 15:56 |
| EmilienM | ayoung: you're late | 15:56 |
| EmilienM | we unpinned like last week | 15:56 |
| EmilienM | 2016-09-07 14:22 Emilien Macchi o Revert "Pin Keystone to Newton milestone 3" | 15:57 |
| EmilienM | 9 days ago | 15:57 |
| ayoung | EmilienM, I can't track it all. Just wanted to make sure I was backing you guys up. | 15:57 |
| EmilienM | ayoung: https://review.rdoproject.org/r/#/c/2098/ | 15:57 |
| EmilienM | ayoung: well, hopefully we can track them all | 15:57 |
| EmilienM | otherwise our CI would be broken every day. | 15:58 |
| EmilienM | marios: sure, looking | 15:58 |
| EmilienM | marios: i'll trust you for testing it, as we don't any testing for manila | 15:58 |
| EmilienM | marios: puppet code looks ok | 15:58 |
| marios | EmilienM: right, yeah tbarron is testing that stuff he is commenting about things passing failing.. we are waiting to hear final ok but he was blocked today on unrelated issue 14:45 < tbarron> marios: so now we hit a weird mongodb error, certainly unrelated to your changes: http://paste.fedoraproject.org/428796/14740257/ | 16:00 |
| marios | EmilienM: thanks | 16:00 |
| EmilienM | cool yw | 16:00 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: [mitaka-only] mysql: never add brackets to mysql_bind_host https://review.openstack.org/371029 | 16:02 |
| *** zoli is now known as zoli|gone | 16:02 | |
| *** derekh has quit IRC | 16:02 | |
| beagles | dprince: I've been looking at https://bugs.launchpad.net/tripleo/+bug/1623155 - I'd like to try to find a reasonable way to fix this somehow in the templates, but I'm coming up with nada | 16:03 |
| openstack | Launchpad bug 1623155 in tripleo "Neutron L3 HA isn't apparently being enabled anywhere" [High,Confirmed] | 16:03 |
| zoli|gone | have a nice weekend | 16:03 |
| beagles | dprince: the alternative is to revert the change to the tripleoclient that removed enabling neutron L3 HA when the controller count > 1 | 16:04 |
| beagles | dprince: could use your insight here | 16:04 |
| *** panda is now known as panda|bbl | 16:04 | |
| *** zoli|gone is now known as zoli_gone-proxy | 16:04 | |
| beagles | shardy too if he's around ^^^ | 16:04 |
| shardy | beagles: can we use the equals function to convert e.g ControllerCount == 1 to a boolean? | 16:07 |
| shardy | http://docs.openstack.org/developer/heat/template_guide/hot_spec.html#equals | 16:07 |
| shardy | it's going to mean the service is still tied to the Controller Role, but at least it won't be hard-coded in tripleoclient | 16:07 |
| beagles | shardy: yup... is the ControllerCount available to where NeutronL3HA is set? | 16:07 |
| beagles | shardy: if so, that'd be awesomely easy | 16:08 |
| shardy | beagles: it should be, it's passed in via parameter_defaults like everything else | 16:08 |
| * shardy thinks for a role agnostic way to do it | 16:08 | |
| beagles | shardy: ooo... actually I have to be a bit more slick than that. I have to make a conditional I think, because we don't want to enable it if controllercount > 1 and dvr is enabled | 16:09 |
| shardy | beagles: perhaps nest two if functions? | 16:10 |
| shardy | http://docs.openstack.org/developer/heat/template_guide/hot_spec.html#if | 16:10 |
| beagles | shardy: ah of course, yeah | 16:10 |
| shardy | These are bleeding edge functions that just landed in Heat | 16:10 |
| shardy | what could possibly go wrong ;) | 16:10 |
| *** cdearborn has joined #tripleo | 16:10 | |
| shardy | http://docs.openstack.org/developer/heat/template_guide/hot_spec.html#or | 16:11 |
| beagles | shardy: I briefly considered suggesting a conditional resource for roles that resourcegroups with counts >1, but it got complicated pretty fast. | 16:11 |
| EmilienM | shardy: our CI looks pretty good now, if I propose rc1, stable/newton will be created and we'll have to backport all the things we want from master to stable/newton. Do we want that? | 16:11 |
| shardy | Actually I think the or example which combines or, equals and not gets you pretty close? | 16:11 |
| beagles | shardy: yeah. I'll give it as hot | 16:12 |
| beagles | I mean shot . | 16:12 |
| shardy | EmilienM: Lets look at the FFE blueprint status - It'd be nice to cut an RC1 so we can really focus on bugfixes for an RC2 | 16:13 |
| shardy | but ideally I'd prefer we didn't carry lots of FFEs into the RC2 (ideally none) | 16:13 |
| *** colonwq has quit IRC | 16:14 | |
| *** absubram has quit IRC | 16:14 | |
| *** masco has quit IRC | 16:15 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Swift add_devices.pp IPv6 handling https://review.openstack.org/369976 | 16:15 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Add IPv6 network configuration for ipv6 job types https://review.openstack.org/363674 | 16:16 |
| EmilienM | panda|bbl: done ^ | 16:16 |
| *** fragatina has joined #tripleo | 16:16 | |
| EmilienM | shardy: right | 16:16 |
| EmilienM | shardy: maybe could we create a gerrit topic with all patches we want in rc1 | 16:16 |
| gfidente | beagles, add me on review pls | 16:17 |
| beagles | gfidente: ack | 16:17 |
| shardy | EmilienM: It looks like there's going to be a few FFEs still to land, including the last custom-roles patches and the large fluentd client one | 16:17 |
| gfidente | the idea was that neutron/l3 would be disabled on dev environments | 16:17 |
| *** mcornea has quit IRC | 16:17 | |
| gfidente | and that the default would work fine for 3 controllers | 16:17 |
| EmilienM | shardy: ok for gerrit topic? it will help reviewers | 16:18 |
| gfidente | but having the logic in the template would be nicer | 16:18 |
| shardy | EmilienM: I'm fine with branching now tho, I guess it'd be good to align with all the other projects, and it will help us focus on what we really need to land for the final release | 16:18 |
| EmilienM | https://review.openstack.org/#/q/topic:tripleo/rc1 | 16:18 |
| shardy | EmilienM: Yes, that's a good idea, thanks | 16:18 |
| EmilienM | shardy: i'm starting something but I'm afraid to miss patches, I'll ask you to review | 16:18 |
| *** colonwq has joined #tripleo | 16:19 | |
| shardy | EmilienM: did you check with dhellmann that it's OK for us to miss the RC1 deadline given that most projects are cycle-trailing? | 16:19 |
| EmilienM | marios, shardy: is https://review.openstack.org/#/c/358525/ in FFE? | 16:20 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates: Move rabbit's clustering port away from the ephemeral port range https://review.openstack.org/345851 | 16:20 |
| EmilienM | shardy: yes I did | 16:20 |
| EmilienM | he said that's not too bad, as long as we release our final release on time | 16:20 |
| *** fragatina has quit IRC | 16:20 | |
| shardy | EmilienM: Ok, thats good | 16:20 |
| EmilienM | shardy: please add patches in https://review.openstack.org/#/q/status:open+project:openstack/tripleo-heat-templates+branch:master+topic:tripleo/rc1 | 16:21 |
| EmilienM | err sorry wrong ling | 16:21 |
| shardy | EmilienM: yeah that's an FFE https://blueprints.launchpad.net/tripleo/+spec/manila-cephfs-integration | 16:21 |
| EmilienM | https://review.openstack.org/#/q/topic:tripleo/rc1 | 16:21 |
| EmilienM | shardy: well the patch is in bad shape | 16:21 |
| shardy | marios, Jokke_: what's the status of https://review.openstack.org/#/c/358525/ ? | 16:21 |
| shardy | should we defer it to Ocata-1? | 16:21 |
| shardy | EmilienM: agreed | 16:22 |
| EmilienM | merge conflict, -1 from marios, CI not passing | 16:22 |
| EmilienM | I don't want to be pessimist but... | 16:22 |
| shardy | Yeah, we're going to have to start deferring things pretty soon as we've pushed things pretty far with FFEs already | 16:22 |
| EmilienM | shardy: I'm not putting bugs in rc1 topic | 16:23 |
| gfidente | shardy, EmilienM is the gerrit branch for bug fixes too? | 16:23 |
| EmilienM | just features | 16:23 |
| EmilienM | ideally, just features | 16:24 |
| gfidente | ah that just answers it :) | 16:24 |
| shardy | EmilienM: Yep, lets focus on the features, then we can groom the bugs for RC2 | 16:24 |
| EmilienM | bug fixes can still be backported | 16:24 |
| shardy | +1 | 16:24 |
| *** dprince has quit IRC | 16:24 | |
| openstackgerrit | Giulio Fidente proposed openstack/python-tripleoclient: Do not use selinux-permissive for the CentOS image https://review.openstack.org/360097 | 16:25 |
| openstackgerrit | Giulio Fidente proposed openstack-infra/tripleo-ci: Revert "Create overcloud images for liberty using EXT4" https://review.openstack.org/368953 | 16:25 |
| marios | shardy: i was waiting for Jokke_ to fix that up ... i think it is a really easy fix... | 16:26 |
| EmilienM | paramite, larsks: about ops tools, is https://review.openstack.org/353506 last blocker? | 16:26 |
| shardy | marios: yeah I nearly did it myself but wasn't sure if you were already on it | 16:26 |
| shardy | we're running out of time for Newton, can you push an update? | 16:27 |
| EmilienM | shardy: do we have all you need for custom roles in https://review.openstack.org/#/q/topic:tripleo/rc1 ? | 16:27 |
| *** osp has quit IRC | 16:27 | |
| marios | shardy: EmilienM that tht https://review.openstack.org/#/c/358525/ depends on the puppet-tripleo side which I asked EmilienM to vote on https://review.openstack.org/#/c/366760/ | 16:27 |
| EmilienM | akrivoka, jrist: hey, can you add patches related to "node tagging workflow" in https://review.openstack.org/#/q/topic:tripleo/rc1 please ? | 16:27 |
| EmilienM | marios: please add tripleo/rc1 gerrit topic to all manila cephfs patches | 16:28 |
| shardy | EmilienM: Yes, just those two remaining patches | 16:28 |
| marios | shardy: yeah will try update before i finish up today | 16:28 |
| jrist | I thought I did | 16:28 |
| shardy | we then need CI coverage and docs, but they can be done independent of the release | 16:28 |
| marios | Jokke_: you around? | 16:28 |
| marios | tbarron: any idea? ^ | 16:28 |
| EmilienM | shardy: ack | 16:29 |
| jrist | EmilienM: how do I add it? | 16:29 |
| jrist | https://review.openstack.org/#/c/332132 | 16:29 |
| jtomasek | jrist: your error happens probably because with latest tripleo-heat-templates the heat validate fails again | 16:29 |
| jrist | jtomasek: ah that is possible. this is latest master | 16:29 |
| jrist | jtomasek: what templates should I use? got a hash? | 16:29 |
| jtomasek | jrist: use my capabilities patch, that works. | 16:30 |
| jrist | figure. ha | 16:30 |
| EmilienM | jrist: there is a button in Gerrit | 16:30 |
| jrist | patch the patches | 16:30 |
| EmilienM | "Edit topic" | 16:30 |
| jtomasek | jrist: git review -d <mypatchid> | 16:30 |
| *** absubram has joined #tripleo | 16:30 | |
| jrist | ya | 16:30 |
| jrist | I know how to do that :) | 16:30 |
| jrist | thanks jtomasek | 16:30 |
| EmilienM | done | 16:30 |
| * EmilienM afk lunch | 16:30 | |
| jrist | EmilienM: change to tripleo/rc1 ? | 16:30 |
| EmilienM | yes | 16:31 |
| jrist | ha you did it | 16:31 |
| jrist | thanks | 16:31 |
| EmilienM | yup | 16:31 |
| * EmilienM afk_real | 16:31 | |
| jtomasek | jrist: this work is supposed to fix the heat validate hopefully https://review.openstack.org/#/c/368150/ | 16:31 |
| jrist | woot nice thanks | 16:31 |
| jrist | so I could pull that too :) | 16:31 |
| jrist | oh yheah | 16:31 |
| jrist | I saw this | 16:31 |
| jrist | thanks | 16:31 |
| gfidente | is it okay for me to tag a couple of patches tripleo-rc1 ? | 16:31 |
| *** rasca has quit IRC | 16:32 | |
| jrist | d0ugal: on above patch 368150 should we recheck again? | 16:32 |
| EmilienM | if they are related to the blueprints in https://launchpad.net/tripleo/+milestone/newton-rc1 yes | 16:32 |
| shardy | gfidente: Yes, but please only tag FFE feature patches, or bugfixes that should block the release and can't wait for RC2 | 16:32 |
| EmilienM | otherwise no. | 16:32 |
| jrist | or does it happen after a rebase automatically | 16:32 |
| *** tbonds has joined #tripleo | 16:32 | |
| gfidente | shardy, wait we said no bugs | 16:32 |
| shardy | gfidente: critical release blockers only | 16:32 |
| tbarron | marios: i haven't talked to Jokke_ today | 16:33 |
| EmilienM | marios, gfidente: please do the same for manila cephfs :-) | 16:33 |
| shardy | like, we land that list, then we tag the release | 16:33 |
| *** ebalduf has quit IRC | 16:33 | |
| *** jpich has quit IRC | 16:33 | |
| openstackgerrit | Adriano Petrich proposed openstack/tripleo-heat-templates: GATE TEST, please ignore https://review.openstack.org/360618 | 16:33 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Add integration with Manila CephFS Native driver https://review.openstack.org/358525 | 16:35 |
| marios | tbarron: ack thanks | 16:35 |
| marios | shardy: done https://review.openstack.org/#/c/358525/ EmilienM will do but maybe monday are you guys cuttin rc1 today? I thought 26th? | 16:36 |
| gfidente | shardy, honestly I am not sure how come not everybody hits https://bugs.launchpad.net/tripleo/+bug/1623552 | 16:36 |
| openstack | Launchpad bug 1623552 in tripleo "heatclient resolves paths for types and get_file calls that then don't make sense in swift" [Critical,Confirmed] | 16:36 |
| EmilienM | marios: no we don't cut today | 16:36 |
| EmilienM | we cut as soon as we have critical features merged | 16:37 |
| shardy | marios: that's the deadline for final RC's, we're trying to cut an RC1 before then (same as other projects), but we're going to miss the RC1 window (which closes today) by a few days due to all the FFEs we had | 16:38 |
| shardy | (combined with CI issues, as normal) | 16:38 |
| marios | shardy: ack thanks | 16:38 |
| shardy | marios: we'll do an RC2 with a bunch of bugfixes, but we're trying to get the FFEs cleared away before RC1 | 16:38 |
| *** masco has joined #tripleo | 16:39 | |
| shardy | gfidente: ack, we definitely need to fix that, I've not yet reproduced locally but will try, and see if I can help with the fix | 16:41 |
| gfidente | shardy, I am sorry I wanted to look into it today | 16:42 |
| gfidente | but had more ceph stuff :( | 16:42 |
| gfidente | shardy, though if after an upload you try | 16:43 |
| gfidente | swift download container | 16:43 |
| gfidente | in the downloaded files you get the file:// references | 16:43 |
| *** jaosorior has joined #tripleo | 16:44 | |
| shardy | gfidente: Yeah, that should actually be OK provided they match the keys in the files map we pass to heat | 16:44 |
| gfidente | the paths are okay | 16:45 |
| shardy | but it sounds like we're failing to get those files because we add the file:// prefix before resolving the file contents and adding it to the map | 16:45 |
| gfidente | yes exactly | 16:45 |
| jaosorior | EmilienM: hey dude, read your comment on the vnc proxy CR. if you have time, could you make the puppet-nova change? Else I'll check that out on monday. | 16:45 |
| *** bfournie has quit IRC | 16:45 | |
| gfidente | requests/sessions fails with https://bugs.launchpad.net/tripleo/+bug/1623552 | 16:45 |
| openstack | Launchpad bug 1623552 in tripleo "heatclient resolves paths for types and get_file calls that then don't make sense in swift" [Critical,Confirmed] | 16:45 |
| gfidente | sorry "No connection adapters were found" | 16:45 |
| gfidente | so either we strip it out and make those relatives, or we point to swift | 16:46 |
| gfidente | it seems | 16:46 |
| EmilienM | jaosorior: puppet-tripleo you mean? | 16:46 |
| *** bfournie has joined #tripleo | 16:46 | |
| shardy | gfidente: yeah, I'm not sure which of those will work, but in theory heatclient should do all this for us | 16:46 |
| *** fragatina has joined #tripleo | 16:46 | |
| shardy | I had to fight with heatclient a bit previously to get the template_object stuff to work, so will take a look | 16:47 |
| *** lucasagomes is now known as lucas-dinner | 16:47 | |
| *** thrash is now known as thrash|f00dz | 16:47 | |
| jaosorior | EmilienM: aaah, i thought you meant to do that change directly in puppet-nova | 16:47 |
| *** abregman is now known as abregman|afk | 16:48 | |
| EmilienM | jaosorior: no | 16:49 |
| *** fragatina has quit IRC | 16:49 | |
| *** fragatina has joined #tripleo | 16:49 | |
| jaosorior | EmilienM: alright. That could work. | 16:49 |
| gfidente | thanks EmilienM for checking those as well | 16:50 |
| gfidente | me going afk | 16:50 |
| *** jprovazn has quit IRC | 16:54 | |
| *** gfidente has quit IRC | 16:55 | |
| *** ohamada_ has quit IRC | 16:55 | |
| Jokke_ | marios, shardy, tbarron: I will have the fix up still today | 17:00 |
| *** bana_k has joined #tripleo | 17:00 | |
| Jokke_ | just haven't pushed it out yet | 17:00 |
| *** rajinir has joined #tripleo | 17:04 | |
| *** fragatina has quit IRC | 17:05 | |
| *** fragatina has joined #tripleo | 17:06 | |
| *** florianf is now known as florianf|afk | 17:08 | |
| *** jcoufal_ has joined #tripleo | 17:08 | |
| *** jcoufal__ has quit IRC | 17:11 | |
| *** tosky has quit IRC | 17:13 | |
| EmilienM | larsks, shardy: so like I said, something is wrong with scenario003, and scenario003 only adds sahara iirc | 17:13 |
| EmilienM | https://review.openstack.org/#/c/353506/ | 17:13 |
| EmilienM | http://logs.openstack.org/06/353506/53/check/gate-tripleo-ci-centos-7-scenario003-multinode/b4e2dd2/console.html#_2016-09-16_13_29_10_762448 | 17:14 |
| EmilienM | honza: i'm not going to -1 because we don't have time but please write commit messages, eg https://review.openstack.org/#/c/332132/10 | 17:17 |
| EmilienM | akrivoka: you're also commiter on this patch ^ | 17:17 |
| honza | EmilienM: thanks, you're totally right, i need to be pay more attention to those | 17:18 |
| EmilienM | cool np | 17:18 |
| marios | honza: i updated it | 17:18 |
| marios | sorry Jokke_ | 17:18 |
| marios | honza: apologies, it is long day for me, was a mistake was meant for Jokke_ | 17:19 |
| honza | marios: :) | 17:19 |
| marios | Jokke_: i updated https://review.openstack.org/#/c/358525/5 | 17:19 |
| *** jpena is now known as jpena|away | 17:19 | |
| marios | Jokke_: was a quick fix | 17:19 |
| marios | Jokke_: puppet-tripleo side was also updated today so should be good. testing is what we need righ tnow, for this and manila (both the THT change and the puppet-tripleo change have the netapp as parent since it does the tidy up for the backends etc) | 17:20 |
| *** rhallisey has quit IRC | 17:21 | |
| *** rhallisey has joined #tripleo | 17:23 | |
| tbarron | Jokke_: notes on my testing are at https://etherpad.openstack.org/p/manila-overcloud-deploy-with-netapp-notes | 17:29 |
| tbarron | Jokke_: obviously your patches will be slightly difft than mine, but you can see how we're doing it | 17:30 |
| tbarron | Jokke_: when a deploy fails i run 'heat resource-list --nested-depth 5 overcloud | grep FAILED' and 'heat deployment show <uuid>' to see why and update the review. | 17:31 |
| bnemec | tbarron: Jokke_: If you're on master, it's way easier to use "openstack stack failures list overcloud" to find out what failed. | 17:35 |
| tbarron | also 'ssh heat-admin@<controller-ip> 'sudo grep enabled_ /etc/manila/manila.conf; sudo ls -a /var/log/manila' to see if we got lucky and the deploy is far enough along to update manila.conf and attempt to start up services | 17:40 |
| tbarron | bnemec: thanks for the tip! i'm sure i'll get opportunity to try that soon :) | 17:40 |
| *** kjw3 has quit IRC | 17:44 | |
| *** trown is now known as trown|lunch | 17:48 | |
| tbarron | bnemec: that is not only more convenient, but it's showing me an error (parameter w/o value from Hiera data file and no default supplied in puppet-tripleo module) that I didn't see before. Thanks! | 17:48 |
| *** thrash|f00dz is now known as thrash | 17:51 | |
| *** _milan_ has quit IRC | 17:51 | |
| openstackgerrit | Adriano Petrich proposed openstack/tripleo-heat-templates: GATE TEST, please ignore https://review.openstack.org/365449 | 17:52 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-heat-templates: WIP Add a function to upgrade from full HA to NG HA https://review.openstack.org/358626 | 17:53 |
| *** akshai_ has quit IRC | 18:01 | |
| *** jcoufal_ has quit IRC | 18:05 | |
| *** kjw3 has joined #tripleo | 18:05 | |
| *** jcoufal_ has joined #tripleo | 18:06 | |
| *** paramite has quit IRC | 18:13 | |
| *** akrivoka has quit IRC | 18:18 | |
| *** jcoufal_ has quit IRC | 18:20 | |
| *** jcoufal_ has joined #tripleo | 18:20 | |
| *** chlong_ has quit IRC | 18:27 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Fix _from_pool_v6.yaml str_split https://review.openstack.org/371576 | 18:29 |
| *** tbonds has quit IRC | 18:31 | |
| EmilienM | slagle: this one can be merged also https://review.openstack.org/#/c/371495/ | 18:31 |
| slagle | ok | 18:32 |
| slagle | for that one, i wasn't actually sure if we needed a bug for that | 18:32 |
| slagle | it's not a feature, i guess it's fine | 18:32 |
| EmilienM | slagle: yeah, it's just helping to remove warnings in puppet catalog | 18:34 |
| EmilienM | (we already have a ton because of deprecations things) | 18:35 |
| EmilienM | but they should disappear in ocata | 18:35 |
| *** yamahata has joined #tripleo | 18:37 | |
| *** cwolferh has quit IRC | 18:37 | |
| slagle | roger | 18:38 |
| *** [1]cdearborn has joined #tripleo | 18:38 | |
| *** lhinds has joined #tripleo | 18:41 | |
| *** cdearborn has quit IRC | 18:42 | |
| beagles | woohoo undercloud upgrade no issues | 18:42 |
| * beagles does a little dance | 18:43 | |
| beagles | it's the little things | 18:43 |
| *** pkovar has quit IRC | 18:44 | |
| *** bnemec is now known as beekneemech | 18:45 | |
| *** saneax-_-|AFK is now known as saneax | 18:46 | |
| *** athomas has quit IRC | 18:46 | |
| *** abregman|afk is now known as abregman | 18:48 | |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-ui: When deploy finishes, show overcloud info https://review.openstack.org/370765 | 18:49 |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-ui: Update tripleo-ui-deps RPM https://review.openstack.org/370775 | 18:54 |
| honza | jrist: merge conflict resolved https://review.openstack.org/#/c/370775 | 18:55 |
| *** trown|lunch is now known as trown | 19:02 | |
| *** jaosorior has quit IRC | 19:05 | |
| *** cwolferh has joined #tripleo | 19:07 | |
| beagles | EmilienM: re: https://bugs.launchpad.net/tripleo/+bug/1612786 - I added a comment proposing we bump to ocata. We have a partial fix in that will probably do for now. | 19:07 |
| openstack | Launchpad bug 1612786 in tripleo "Add validation to disallow OVS round-robin bonding" [Medium,In progress] - Assigned to Brent Eagles (beagles) | 19:07 |
| EmilienM | beagles: rc2 or ocata, up to you | 19:07 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Add hyperconverged-ceph environment to include CephOSD on computes https://review.openstack.org/338113 | 19:08 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Fix use of batch_create in CephMon major upgrade template https://review.openstack.org/370127 | 19:08 |
| *** mburned is now known as mburned_out | 19:11 | |
| beagles | EmilienM: mmm... re-reading bnemec's comment there are some examples and doc changes that should be done, so RC-2 probably is appropriate. I'll update | 19:11 |
| EmilienM | ok | 19:12 |
| EmilienM | roger! | 19:12 |
| *** absubram has quit IRC | 19:13 | |
| EmilienM | bandini: do you have a patch for https://bugs.launchpad.net/tripleo/+bug/1623818 ? | 19:15 |
| openstack | Launchpad bug 1623818 in tripleo "RabbitMQ should use predefined ports below ephemeral ports range " [High,In progress] - Assigned to Michele Baldessari (michele) | 19:15 |
| *** r-mibu has quit IRC | 19:16 | |
| *** r-mibu has joined #tripleo | 19:17 | |
| openstackgerrit | Merged openstack/instack-undercloud: Update undercloud.conf.sample https://review.openstack.org/371567 | 19:17 |
| *** mgarciam has quit IRC | 19:22 | |
| EmilienM | shardy: I updated https://launchpad.net/tripleo/+milestone/newton-rc1 | 19:22 |
| EmilienM | all patches that landed or are going to land today are rc1 | 19:23 |
| EmilienM | all patches that do not pass CI or negative review are rc2 | 19:23 |
| EmilienM | all bugs without patches, and not high or critical are ocata-1 | 19:23 |
| EmilienM | all bugs without patches, critical or high are rc2 | 19:23 |
| EmilienM | so in rc1, we still have 3 bugs in progress and these patches https://review.openstack.org/#/q/topic:tripleo/rc1 | 19:24 |
| EmilienM | in RC2, we have 4 Confirmed, 13 Triaged, 26 In Progress | 19:24 |
| slagle | EmilienM: are you +2 on https://review.openstack.org/#/c/332132/ ? | 19:26 |
| slagle | agree the commit message is bad | 19:26 |
| EmilienM | slagle: yeah I told to honza that commit messages are important. | 19:27 |
| EmilienM | slagle: and no, I won't +2 until ovb jobs are green | 19:27 |
| slagle | yea, i just meant with the commit message as-is | 19:27 |
| slagle | if it passes CI, i'm ok to merge it | 19:28 |
| slagle | i'll +2 | 19:28 |
| EmilienM | well, usually I would -1 but since we are close to the release, I'm not blocking it | 19:28 |
| EmilienM | slagle: ok | 19:28 |
| EmilienM | slagle: if CI pass i'll review it and maybe approve it | 19:28 |
| slagle | ok | 19:28 |
| EmilienM | slagle: https://review.openstack.org/#/c/353506/ worries me | 19:29 |
| EmilienM | slagle: gate-tripleo-ci-centos-7-scenario003-multinode fails and that's not normal | 19:29 |
| slagle | yea some response or investigation about that failure on the patch is needed | 19:30 |
| *** david-lyle has quit IRC | 19:30 | |
| *** david-lyle has joined #tripleo | 19:30 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Convert UpdateWorkflow to support composable roles https://review.openstack.org/367282 | 19:31 |
| *** sarathk has joined #tripleo | 19:31 | |
| slagle | EmilienM: scenario001 is also failing | 19:32 |
| slagle | is that expected? | 19:32 |
| pradk | can we merge this https://review.openstack.org/#/c/363748/ ? | 19:33 |
| EmilienM | slagle: yes scenario001 is failing I think because of ceph | 19:35 |
| EmilienM | slagle: I'll look in a few min | 19:35 |
| *** dprince has joined #tripleo | 19:37 | |
| EmilienM | larsks: are you around? | 19:37 |
| *** larsks has left #tripleo | 19:38 | |
| *** larsks has joined #tripleo | 19:38 | |
| larsks | EmilienM, sort of :). | 19:38 |
| *** dprince has quit IRC | 19:39 | |
| *** akshai has joined #tripleo | 19:39 | |
| *** akshai_ has joined #tripleo | 19:40 | |
| EmilienM | larsks: you'll have to help us a bit if you want https://review.openstack.org/#/c/353506/ merged | 19:42 |
| EmilienM | we're debugging why it doesn't pass functional tests | 19:42 |
| *** david-lyle has quit IRC | 19:43 | |
| larsks | EmilienM, I have been trying to look at that but I've bene stymied by the fact that I can't get an overcloud deploy to start, at all, due to all the bugs in tripleo master. | 19:43 |
| larsks | I'm going to spend some more time with it this evening. | 19:43 |
| larsks | I could use some help from someone! | 19:43 |
| EmilienM | what bugs? | 19:43 |
| EmilienM | slagle: for the record, scenario001 is failing because of a gnocchi bug: http://logs.openstack.org/06/353506/53/check/gate-tripleo-ci-centos-7-scenario001-multinode-nv/dd8ea5f/console.html#_2016-09-16_14_14_39_051833 | 19:44 |
| *** akshai has quit IRC | 19:44 | |
| EmilienM | slagle: really low prio imho, checked with pradk and gnocchi works for him so I'll continue to debug after the release | 19:44 |
| *** jpena|away is now known as jpena|off | 19:44 | |
| EmilienM | slagle: scenario003 error is important though, it sounds like a bad format of heat template in larsks's patch. | 19:45 |
| bandini | EmilienM: yes I do, not sure why LP did not pick it up | 19:45 |
| slagle | EmilienM: sounds good, as long as we know | 19:45 |
| EmilienM | bandini: please put the patch in the lp manually, so at least we know you work on it. | 19:46 |
| larsks | EmilienM, last I checked there still bugs that mean (a) deploying with custom envrionments doens't work and (b) error reporting is problematic. Do you know if these have been fixed? | 19:47 |
| larsks | EmilienM, I have to perform some kid transport, but will check in a bit. | 19:47 |
| slagle | what the launchpad bugs? | 19:48 |
| slagle | *are | 19:48 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Add CephRgw to roles_data.yaml https://review.openstack.org/370687 | 19:48 |
| openstackgerrit | Merged openstack/instack-undercloud: Fix nova-related deprecation warnings https://review.openstack.org/371495 | 19:48 |
| EmilienM | I'm trying the patch locally, there is something wrong in the template i think | 19:48 |
| slagle | tripleo-ci has a lot of green and it uses custom environments all over the place | 19:48 |
| beekneemech | slagle: larsks may be referring to the broken --templates parameter | 19:49 |
| beekneemech | I don't actually know if that's fixed because I just haven't tried lately. :-/ | 19:50 |
| slagle | maybe. but that doesnt block this patch | 19:51 |
| beekneemech | Yeah. It does make it a bit of a pita to work on template changes though. | 19:52 |
| EmilienM | pradk: +A lgtm | 19:53 |
| *** absubram has joined #tripleo | 19:57 | |
| *** openstackstatus has quit IRC | 19:58 | |
| *** openstackstatus has joined #tripleo | 20:01 | |
| *** ChanServ sets mode: +v openstackstatus | 20:01 | |
| *** kjw3 has quit IRC | 20:08 | |
| *** rcarrillocruz has joined #tripleo | 20:08 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Refactor upgrade checks. https://review.openstack.org/357750 | 20:11 |
| *** kjw3 has joined #tripleo | 20:11 | |
| *** jayg is now known as jayg|g0n3 | 20:18 | |
| *** panda|bbl is now known as panda | 20:18 | |
| *** jcoufal has quit IRC | 20:18 | |
| *** jcoufal_ has quit IRC | 20:19 | |
| larsks | EmilienM, in the logs for scenario0003, what don't I see "Uploading new plan files" in the console output? | 20:19 |
| larsks | s/what/why | 20:19 |
| larsks | Or slagle or really anybody who is around.... | 20:20 |
| EmilienM | larsks: any url handy? | 20:22 |
| larsks | http://logs.openstack.org/06/353506/53/check/gate-tripleo-ci-centos-7-scenario003-multinode/b4e2dd2/console.html | 20:22 |
| larsks | Mostly I am concerned that I am using locally a different version of things than CI is using... | 20:23 |
| EmilienM | http://logs.openstack.org/06/353506/53/check/gate-tripleo-ci-centos-7-scenario003-multinode/b4e2dd2/console.html#_2016-09-16_13_28_19_260695 | 20:24 |
| EmilienM | plan created | 20:24 |
| larsks | Yes, I see that. | 20:24 |
| EmilienM | there is no problem in mistral | 20:24 |
| EmilienM | the prob is 2016-09-16 13:29:10.762448 | 2016-09-16 13:29:05Z [overcloud]: CREATE_FAILED Resource CREATE failed: The Referenced Attribute (ControllerServiceChain role_data) is incorrect. | 20:24 |
| larsks | But when I run a deploy, I first see "uploading new plan files". | 20:24 |
| larsks | Yes, I see that error, too :) | 20:24 |
| larsks | But that is not my question right now. | 20:25 |
| EmilienM | let's look at http://logs.openstack.org/06/353506/53/check/gate-tripleo-ci-centos-7-scenario003-multinode/b4e2dd2/logs/postci.txt.gz | 20:25 |
| EmilienM | because that error is the reason of overcloud failure | 20:25 |
| larsks | That error from heat (re; controllerservicechain) often indicates a typo or bad parameter reference in a deeply nested stack. Unfortunately, heat doens't actually log the source of the error anywhere. | 20:26 |
| larsks | My first question, if we could just pause, is why the output I see when starting a deploy is different than what I see in CI. That would help me make sure I am testing in an appropriate environment. | 20:26 |
| larsks | I understand that is not the cause of the error. | 20:27 |
| EmilienM | somewhere in http://logs.openstack.org/06/353506/53/check/gate-tripleo-ci-centos-7-scenario003-multinode/b4e2dd2/logs/postci.txt.gz#_2016-09-16_13_29_20_000 | 20:27 |
| EmilienM | I'm digging | 20:27 |
| EmilienM | let's see in heat engine logs | 20:27 |
| openstackgerrit | Merged openstack/tripleo-common: Clearer error when the Mistral env already exists https://review.openstack.org/367379 | 20:27 |
| openstackgerrit | Merged openstack/puppet-tripleo: Add manila-netapp backend to manila class and tidy up generic https://review.openstack.org/354014 | 20:27 |
| larsks | EmilienM, never mind. We appear to be having different conversations right now. I will see if you have some time later. | 20:28 |
| EmilienM | larsks: i'm looking for the root cause of why your patch fails | 20:28 |
| larsks | EmilienM, yes, and I was asking a different set of questions to try to get a local testing environment that mataches what ci is using. | 20:29 |
| EmilienM | if you want to reproduce the problem, you can use the same environment as tripleo CI scenario003 | 20:29 |
| EmilienM | let me find you the link | 20:29 |
| EmilienM | https://github.com/openstack-infra/tripleo-ci/blob/master/test-environments/scenario003-multinode.yaml | 20:30 |
| larsks | Since I can't even *start* a deploy locally, debugging this has been difficult. | 20:30 |
| *** mburned_out is now known as mburned | 20:31 | |
| therve | larsks, You don't see "uploading new plan files" in the CI because it's a fresh deployment | 20:34 |
| larsks | therve, thanks. So, with my local environment, all attemtps to start a deploy currently fail with "Exception updating plan: The environment is not a valid YAML mapping data type." | 20:35 |
| larsks | I think I am going to kill it all and start fresh. | 20:35 |
| therve | larsks, https://review.openstack.org/#/c/371027/ maybe | 20:36 |
| therve | But restarting from scratch would fix that particular issue | 20:36 |
| larsks | This has been frustrating enough that I think starting fresh is probably the best idea. | 20:37 |
| EmilienM | what I don't understand is why other scenarios are working | 20:38 |
| EmilienM | the only diff between scenario003 and other is that we have SaharaApi and SaharaEngine services | 20:38 |
| therve | Ah, good idea :) | 20:41 |
| therve | larsks, EmilienM: typo here: https://review.openstack.org/#/c/353506/53/puppet/services/sahara-engine.yaml | 20:41 |
| therve | Why it doesn't give an error here is a good question | 20:43 |
| larsks | therve, there are (were?) open heat bugs about validation problems. | 20:43 |
| therve | I bet there are :/ | 20:44 |
| EmilienM | therve: I've been reading this file 10 times | 20:44 |
| EmilienM | therve: thank you := | 20:44 |
| therve | No pb | 20:45 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Add fluentd client service https://review.openstack.org/353506 | 20:45 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates: Add fluentd client service https://review.openstack.org/353506 | 20:46 |
| EmilienM | larsks: why did you rebase on master yesterday? | 20:46 |
| EmilienM | I rebased it on shardy's patches to avoid merge conflict.. | 20:46 |
| larsks | EmilienM, there was a rebase yesterday because I introduced a typo when rebasing on the overcloud.j2.yaml changes. | 20:46 |
| EmilienM | ok, I just hope it will pass this time | 20:47 |
| EmilienM | I'm +2'ing it | 20:47 |
| EmilienM | and will approve it tonight if CI is full green (we already had +2 before) | 20:47 |
| larsks | I will keep my fingers crossed. | 20:47 |
| *** sarathk has quit IRC | 20:48 | |
| therve | EmilienM, Is the change in swift-storage ok? | 20:49 |
| *** noslzzp_ has joined #tripleo | 20:49 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common: Add node tagging workflow https://review.openstack.org/332132 | 20:50 |
| *** noslzzp has quit IRC | 20:50 | |
| *** ccamacho has quit IRC | 20:50 | |
| EmilienM | we're hitting https://launchpad.net/bugs/1624420 a lot | 20:50 |
| openstack | Launchpad bug 1624420 in tripleo "MongoDB Can't find master host for replicaset tripleo." [High,In progress] - Assigned to Jiří Stránský (jistr) | 20:50 |
| EmilienM | larsks: why do you add SwiftRawDisks ? | 20:51 |
| *** ccamacho has joined #tripleo | 20:51 | |
| larsks | EmilienM, sounds like another rebase error from rebasing on shardy's big change. | 20:51 |
| EmilienM | setting up the alert on the mongodb bug | 20:51 |
| EmilienM | it breaks HA job very often | 20:51 |
| EmilienM | larsks: ok, please double check the patch and submit it again; | 20:51 |
| *** trown is now known as trown|outtypewww | 20:53 | |
| openstackgerrit | Gabriele Cerami proposed openstack/puppet-tripleo: Swift add_devices.pp IPv6 handling https://review.openstack.org/369976 | 20:53 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates: Add fluentd client service https://review.openstack.org/353506 | 20:53 |
| openstackgerrit | Alex Schultz proposed openstack/instack-undercloud: Catch runtime exceptions during validation https://review.openstack.org/371802 | 20:54 |
| *** ebalduf has joined #tripleo | 20:55 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Wait for MongoDB connections before creating replset https://review.openstack.org/371596 | 20:55 |
| EmilienM | jistr: I fixed your patch, there was a typo ^ | 20:56 |
| EmilienM | if anyone can +2 it, I think it will make ha job more stable | 20:56 |
| panda | but ipv6 job did not even start :( | 20:58 |
| EmilienM | you need to run "check experimental on tripleo-ci patch" | 20:59 |
| EmilienM | panda: didn't it work? | 20:59 |
| EmilienM | I saw it in zuul this morning | 20:59 |
| beekneemech | panda: telnet://66.187.229.51:19885 | 21:00 |
| *** rhallisey has quit IRC | 21:02 | |
| panda | EmilienM: that's what I do after every patch, but I started a check experimental almost 6 hours ago, and did not receive any result. | 21:04 |
| panda | EmilienM: maybe I should have waited longer. Now I pushed another patch, probably any older check was canceled | 21:04 |
| panda | beekneemech: weeeee | 21:04 |
| beekneemech | panda: Yeah, new patch sets cancel any previous jobs for that change. | 21:07 |
| panda | beekneemech: even funnier than telnet towel.blinkenlights.nl | 21:07 |
| beekneemech | panda: It looks like there were a lot of jobs in the queue earlier, and I believe experimental jobs get lowest priority, so it may just not have started. | 21:07 |
| *** Goneri has quit IRC | 21:08 | |
| openstackgerrit | Merged openstack/tripleo-specs: Spec for TripleO validations https://review.openstack.org/255792 | 21:09 |
| panda | beekneemech: I have to invert the order of my daily tasks, checks in the morning (Europe) so the queue is smaller. | 21:09 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Expose parameter to enable combination alarms https://review.openstack.org/363748 | 21:09 |
| beekneemech | panda: That is a good plan. :-) | 21:09 |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1624420 | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1624420 in tripleo "MongoDB Can't find master host for replicaset tripleo." [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 21:10 |
| beekneemech | Okay, wtf? Now every time I try to open a new terminal it telnets to that review. | 21:11 |
| *** abregman has quit IRC | 21:15 | |
| *** sshnaidm is now known as sshnaidm|away | 21:16 | |
| *** akshai_ has quit IRC | 21:18 | |
| *** [1]cdearborn has quit IRC | 21:19 | |
| *** limao has quit IRC | 21:21 | |
| *** akshai has joined #tripleo | 21:21 | |
| *** ebalduf has quit IRC | 21:23 | |
| *** rlandy has quit IRC | 21:23 | |
| *** akshai has quit IRC | 21:29 | |
| *** fragatin_ has joined #tripleo | 21:33 | |
| pradk | EmilienM, i'm trying to add the ceilomiddleware to swift proxy .. https://review.openstack.org/#/c/371591/ | 21:36 |
| pradk | but ci seems to fail with .. Error: Could not find dependency Class[Ceilometer] for Concat::Fragment[swift_ceilometer] at /etc/puppet/modules/swift/manifests/proxy/ceilometer.pp:107\u001b[0m\n" | 21:36 |
| pradk | seems like a bug in puppet-swift? | 21:36 |
| *** fragatina has quit IRC | 21:36 | |
| openstackgerrit | Merged openstack/python-tripleoclient: Remove excessive output when configuring nodes https://review.openstack.org/366051 | 21:37 |
| *** mburned is now known as mburned_out | 21:38 | |
| *** akshai has joined #tripleo | 21:38 | |
| EmilienM | pradk: yes | 21:39 |
| EmilienM | https://github.com/openstack/puppet-swift/blob/master/manifests/proxy/ceilometer.pp#L106 | 21:39 |
| EmilienM | let me git blame | 21:39 |
| pradk | wow thats quite old | 21:40 |
| *** saneax is now known as saneax-_-|AFK | 21:40 | |
| EmilienM | https://review.openstack.org/#/c/27686/ | 21:40 |
| EmilienM | 3 years and 5 months | 21:41 |
| EmilienM | I guess you can submit a patch ;-) | 21:41 |
| pradk | lol | 21:41 |
| EmilienM | puppet-swift has some interesting things sometimes | 21:41 |
| pradk | sure i'll look into it first thing monday | 21:41 |
| pradk | surprised no one ran into it | 21:42 |
| pradk | i guess its never been used | 21:42 |
| EmilienM | pradk: or ceilometer was installed on same node as swift proxy | 21:45 |
| EmilienM | pradk: at enovance, we built an installer based on puppet and ansible, that used this class | 21:45 |
| EmilienM | and it worked fine because had ceilometer api running on swift proxy nodes | 21:45 |
| pradk | EmilienM, hmm so this is a side effect of composable roles as they dont run on same node any more by default? | 21:47 |
| EmilienM | pradk: yep | 21:47 |
| EmilienM | you found a good bug | 21:47 |
| EmilienM | just submit a patch in puppet-swift, one line and we're good | 21:47 |
| EmilienM | maybe tests needs to be updated | 21:47 |
| EmilienM | pradk: we're releasing puppet modules next week, better to do it asap but it could be backported worst case | 21:48 |
| *** fragatin_ has quit IRC | 21:48 | |
| pradk | understood | 21:49 |
| *** fragatina has joined #tripleo | 21:50 | |
| *** kjw3 has quit IRC | 21:50 | |
| *** ccamacho has quit IRC | 21:52 | |
| *** myoung is now known as myoung|gone | 21:54 | |
| *** lblanchard has quit IRC | 22:03 | |
| *** akshai has quit IRC | 22:04 | |
| *** ooolpbot has joined #tripleo | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1624420 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1624420 in tripleo "MongoDB Can't find master host for replicaset tripleo." [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 22:10 |
| *** fragatina has quit IRC | 22:10 | |
| *** fragatina has joined #tripleo | 22:10 | |
| *** ayoung_ has quit IRC | 22:24 | |
| *** yamahata has quit IRC | 22:25 | |
| EmilienM | it seems like https://review.openstack.org/#/c/371596/ is passing the ovb jobs | 22:33 |
| EmilienM | pingtest worked on both jobs | 22:33 |
| EmilienM | I'll approve it | 22:33 |
| *** yamahata has joined #tripleo | 22:37 | |
| *** panda is now known as panda|Zz | 22:38 | |
| EmilienM | removing alert on https://bugs.launchpad.net/tripleo/+bug/1624420 as it will merge in a few | 22:39 |
| openstack | Launchpad bug 1624420 in tripleo "MongoDB Can't find master host for replicaset tripleo." [Critical,In progress] - Assigned to Jiří Stránský (jistr) | 22:39 |
| EmilienM | beekneemech: if you still around https://review.openstack.org/#/c/369976/ please | 22:42 |
| *** dtrainor has quit IRC | 22:43 | |
| *** david-lyle has joined #tripleo | 22:48 | |
| *** absubram has quit IRC | 22:52 | |
| *** noslzzp_ has quit IRC | 23:25 | |
| openstackgerrit | Monty Taylor proposed openstack/diskimage-builder: Add libselinux-python to yum-minimal https://review.openstack.org/371834 | 23:26 |
| ayoung | deploying the overcloud twice in quick succession gives me an error, but no update in state. http://paste.openstack.org/show/580295/ Is this expected? | 23:30 |
| ayoung | the stack state is CREATE_COMPLETE | 23:31 |
| *** cwolferh has quit IRC | 23:49 | |
| *** mburned_out is now known as mburned | 23:51 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!