| *** lucasagomes has quit IRC | 00:00 | |
| *** lucasagomes has joined #tripleo | 00:02 | |
| *** egafford has quit IRC | 00:04 | |
| *** lucasagomes has quit IRC | 00:07 | |
| *** penick has quit IRC | 00:08 | |
| *** lucasagomes has joined #tripleo | 00:10 | |
| *** lucasagomes has quit IRC | 00:14 | |
| *** lucasagomes has joined #tripleo | 00:17 | |
| *** sai has quit IRC | 00:19 | |
| *** sai has joined #tripleo | 00:19 | |
| *** akuznetsov has joined #tripleo | 00:25 | |
| *** sai has quit IRC | 00:27 | |
| *** lucasagomes has quit IRC | 00:27 | |
| *** akuznetsov has quit IRC | 00:29 | |
| *** sai has joined #tripleo | 00:30 | |
| *** lucasagomes has joined #tripleo | 00:30 | |
| *** thrash is now known as thrash|g0ne | 00:31 | |
| *** lucasagomes has quit IRC | 00:34 | |
| *** lucasagomes has joined #tripleo | 00:38 | |
| *** itlinux has quit IRC | 00:40 | |
| *** itlinux has joined #tripleo | 00:41 | |
| *** limao has joined #tripleo | 00:43 | |
| *** lucasagomes has quit IRC | 00:46 | |
| *** sai has quit IRC | 00:47 | |
| *** lucasagomes has joined #tripleo | 00:48 | |
| *** Guest734 has quit IRC | 00:50 | |
| *** itlinux has quit IRC | 00:56 | |
| *** lucasagomes has quit IRC | 00:57 | |
| *** lucasagomes has joined #tripleo | 01:00 | |
| *** itlinux has joined #tripleo | 01:00 | |
| *** bana_k has quit IRC | 01:03 | |
| *** lucasagomes has quit IRC | 01:04 | |
| *** lucasagomes has joined #tripleo | 01:07 | |
| *** rook has joined #tripleo | 01:07 | |
| *** rook is now known as Guest88826 | 01:08 | |
| *** sai has joined #tripleo | 01:08 | |
| *** lucasagomes has quit IRC | 01:12 | |
| *** lucasagomes has joined #tripleo | 01:14 | |
| *** lucasagomes has quit IRC | 01:19 | |
| *** lucasagomes has joined #tripleo | 01:21 | |
| *** akuznetsov has joined #tripleo | 01:26 | |
| *** lucasagomes has quit IRC | 01:30 | |
| *** akuznetsov has quit IRC | 01:30 | |
| *** sai has quit IRC | 01:32 | |
| *** lucasagomes has joined #tripleo | 01:33 | |
| *** sai has joined #tripleo | 01:34 | |
| *** itlinux has quit IRC | 01:39 | |
| *** skramaja has joined #tripleo | 01:39 | |
| *** sai has quit IRC | 01:42 | |
| *** sai has joined #tripleo | 01:44 | |
| *** penick has joined #tripleo | 01:44 | |
| *** itlinux has joined #tripleo | 01:46 | |
| *** yamahata has quit IRC | 01:55 | |
| *** itlinux has quit IRC | 01:56 | |
| *** yolanda has quit IRC | 01:57 | |
| *** dmacpher has joined #tripleo | 01:58 | |
| *** sai has quit IRC | 01:59 | |
| *** Guest88826 has quit IRC | 01:59 | |
| larsks | EmilienM, around? | 02:00 |
|---|---|---|
| *** sai has joined #tripleo | 02:00 | |
| *** rook_ has joined #tripleo | 02:02 | |
| *** lucasagomes has quit IRC | 02:06 | |
| *** lucasagomes has joined #tripleo | 02:07 | |
| *** apetrich has quit IRC | 02:11 | |
| *** rook_ has quit IRC | 02:13 | |
| *** sai has quit IRC | 02:13 | |
| larsks | Actually...is *anybody* around who is familiar with the brave new world of overcloud.j2.yaml? | 02:14 |
| *** lucasagomes has quit IRC | 02:15 | |
| *** apetrich has joined #tripleo | 02:15 | |
| *** sai has joined #tripleo | 02:15 | |
| *** rook has joined #tripleo | 02:16 | |
| *** rook is now known as Guest16412 | 02:16 | |
| *** lucasagomes has joined #tripleo | 02:17 | |
| *** lucasagomes has quit IRC | 02:22 | |
| *** jlinkes has joined #tripleo | 02:22 | |
| *** ayoung has joined #tripleo | 02:23 | |
| *** lucasagomes has joined #tripleo | 02:23 | |
| *** jlinkes_ has quit IRC | 02:25 | |
| *** akuznetsov has joined #tripleo | 02:26 | |
| *** lucasagomes has quit IRC | 02:28 | |
| *** lucasagomes has joined #tripleo | 02:29 | |
| *** akuznetsov has quit IRC | 02:31 | |
| *** Ryjedo has quit IRC | 02:31 | |
| EmilienM | larsks: yes | 02:31 |
| EmilienM | larsks: i'm trying to rebase it now | 02:32 |
| larsks | EmilienM, hold up. | 02:32 |
| larsks | I rebased patchset 37 on the current master, and it seemed easy. | 02:32 |
| larsks | I'm just running a test locally now. | 02:32 |
| larsks | I do have a question, though: | 02:32 |
| EmilienM | yeah? | 02:32 |
| larsks | With the new overcloud.j2.yaml, node counts (e.g. ControllerCount) can only be set on the overcloud deploy command line, is that correct? | 02:33 |
| larsks | E.g., that can no longer be set in an environment file. | 02:33 |
| EmilienM | larsks: I'm not sure actually | 02:33 |
| larsks | Okay. Not important for what I am doing right now, but impacts something else I was working on. | 02:33 |
| *** lucasagomes has quit IRC | 02:34 | |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-heat-templates: Add fluentd client service https://review.openstack.org/353506 | 02:34 |
| larsks | I'm going to push my rebased changes to gerrit right now, if you want to look at them and see if I missed anything obvious. | 02:34 |
| larsks | I assume I will need to fix something once my local test completes :/ | 02:34 |
| larsks | ...but first I think I need to update python-tripleoclient. | 02:35 |
| EmilienM | ok I'm looking | 02:35 |
| *** lucasagomes has joined #tripleo | 02:37 | |
| EmilienM | larsks: the rebase lgtm | 02:38 |
| EmilienM | nice work | 02:38 |
| larsks | Awesome. We will see what ci says :) | 02:38 |
| EmilienM | larsks: +2'ing it so if CI pass someone will just approve it | 02:40 |
| larsks | Thanks! | 02:40 |
| EmilienM | larsks: though I saw some failures before on scenario003 not sure why | 02:40 |
| * larsks crosses fingers | 02:40 | |
| EmilienM | I hope they won't come back | 02:40 |
| EmilienM | they looked valid though, will see | 02:40 |
| larsks | Yeah, from the ci perspective this should be a no-op, because we're not activating the logging config. | 02:41 |
| *** lucasagomes has quit IRC | 02:41 | |
| EmilienM | larsks: it's frustrating to see your patch in merge conflict | 02:42 |
| EmilienM | though we are adding the last pieces for composable services | 02:42 |
| EmilienM | I hope you understand | 02:42 |
| *** lucasagomes has joined #tripleo | 02:43 | |
| EmilienM | larsks: but we'll make it :) | 02:43 |
| EmilienM | logging in tripleo! | 02:44 |
| larsks | No worries; I appreciate your help. Bummer about ci breaking on a friday, though. | 02:44 |
| larsks | So, what does this mean: Workflow not found [workflow_identifier=tripleo.plan_management.v1.update_deployment_plan] | 02:44 |
| larsks | I assume it means I need to upgrade something, but I'm not sure what. | 02:44 |
| larsks | I've just installed python-tripleoclient from master. | 02:45 |
| *** sai has quit IRC | 02:45 | |
| *** Guest16412 has quit IRC | 02:46 | |
| *** sai has joined #tripleo | 02:47 | |
| *** akshai has joined #tripleo | 02:48 | |
| EmilienM | larsks: I'm not sure | 02:48 |
| *** lucasagomes has quit IRC | 02:48 | |
| EmilienM | larsks: tripleo common? | 02:48 |
| larsks | Worth a shot. | 02:48 |
| *** rook_ has joined #tripleo | 02:48 | |
| EmilienM | :132 | 02:48 |
| *** akshai_ has joined #tripleo | 02:49 | |
| EmilienM | :179 | 02:49 |
| EmilienM | err | 02:49 |
| EmilienM | i'm tired | 02:49 |
| larsks | Yeah, me too. I would just like to know that the patch actually still functions, which ci won't test right now. | 02:49 |
| *** lucasagomes has joined #tripleo | 02:49 | |
| larsks | I guess I will bug shardy in the morning. | 02:49 |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient: Do not include epel element when building overcloud images https://review.openstack.org/369559 | 02:50 |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient: Do not include epel element when building overcloud images https://review.openstack.org/369559 | 02:51 |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient: Do not include epel element when building overcloud images https://review.openstack.org/369559 | 02:52 |
| EmilienM | ok this time is good | 02:52 |
| *** snecklifter has quit IRC | 02:53 | |
| *** akshai has quit IRC | 02:53 | |
| *** sai has quit IRC | 02:54 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: remove DIB workaround https://review.openstack.org/369792 | 02:55 |
| *** rook_ has quit IRC | 02:55 | |
| *** fultonj has quit IRC | 02:55 | |
| *** sai has joined #tripleo | 02:55 | |
| *** snecklifter has joined #tripleo | 02:55 | |
| EmilienM | larsks: /me goes to bed, ttyl | 02:56 |
| *** rook_ has joined #tripleo | 02:56 | |
| *** lucasagomes has quit IRC | 02:57 | |
| *** itlinux has joined #tripleo | 02:57 | |
| *** lucasagomes has joined #tripleo | 02:58 | |
| *** lucasagomes has quit IRC | 03:03 | |
| *** lucasagomes has joined #tripleo | 03:05 | |
| *** snecklifter has quit IRC | 03:05 | |
| *** snecklifter has joined #tripleo | 03:07 | |
| *** itlinux has quit IRC | 03:07 | |
| *** sai has quit IRC | 03:08 | |
| *** rook_ has quit IRC | 03:08 | |
| *** rook_ has joined #tripleo | 03:09 | |
| *** sai has joined #tripleo | 03:10 | |
| *** lucasagomes has quit IRC | 03:10 | |
| *** lucasagomes has joined #tripleo | 03:13 | |
| openstackgerrit | Merged openstack/python-tripleoclient: Migrate to using osc-lib https://review.openstack.org/335460 | 03:13 |
| *** jlinkes_ has joined #tripleo | 03:17 | |
| *** jlinkes has quit IRC | 03:19 | |
| *** akshai_ has quit IRC | 03:19 | |
| *** lucasagomes has quit IRC | 03:21 | |
| *** lucasagomes has joined #tripleo | 03:23 | |
| *** sai has quit IRC | 03:26 | |
| *** rook_ has quit IRC | 03:27 | |
| *** rook has joined #tripleo | 03:27 | |
| *** rook is now known as Guest41113 | 03:27 | |
| *** akuznetsov has joined #tripleo | 03:27 | |
| *** sai has joined #tripleo | 03:27 | |
| *** lucasagomes has quit IRC | 03:28 | |
| *** lucasagomes has joined #tripleo | 03:29 | |
| *** akuznetsov has quit IRC | 03:31 | |
| *** lucasagomes has quit IRC | 03:41 | |
| *** lucasagomes has joined #tripleo | 03:43 | |
| *** weshay is now known as weshay_pto | 03:45 | |
| *** itlinux has joined #tripleo | 03:46 | |
| *** sai has quit IRC | 03:47 | |
| *** lucasagomes has quit IRC | 03:48 | |
| *** Guest41113 has quit IRC | 03:48 | |
| *** rook_ has joined #tripleo | 03:49 | |
| *** sai has joined #tripleo | 03:49 | |
| *** lucasagomes has joined #tripleo | 03:50 | |
| *** lucasagomes has quit IRC | 03:54 | |
| *** itlinux has quit IRC | 03:56 | |
| *** lucasagomes has joined #tripleo | 03:56 | |
| *** sai has quit IRC | 03:58 | |
| *** rook_ has quit IRC | 03:58 | |
| *** rook_ has joined #tripleo | 03:59 | |
| *** sai has joined #tripleo | 03:59 | |
| *** padkrish has joined #tripleo | 04:00 | |
| *** lucasagomes has quit IRC | 04:01 | |
| *** lucasagomes has joined #tripleo | 04:04 | |
| *** sai has quit IRC | 04:05 | |
| *** sai_ has joined #tripleo | 04:08 | |
| *** lucasagomes has quit IRC | 04:12 | |
| *** dmacpher is now known as dmacpher-lunch | 04:12 | |
| *** lucasagomes has joined #tripleo | 04:13 | |
| *** lucasagomes has quit IRC | 04:17 | |
| *** rlandy has quit IRC | 04:19 | |
| *** lucasagomes has joined #tripleo | 04:19 | |
| *** lucasagomes has quit IRC | 04:24 | |
| *** lucasagomes has joined #tripleo | 04:26 | |
| *** akuznetsov has joined #tripleo | 04:28 | |
| *** akuznetsov has quit IRC | 04:33 | |
| *** lucasagomes has quit IRC | 04:33 | |
| *** oshvartz has quit IRC | 04:35 | |
| *** padkrish has quit IRC | 04:35 | |
| *** lucasagomes has joined #tripleo | 04:35 | |
| *** padkrish has joined #tripleo | 04:36 | |
| *** limao has quit IRC | 04:36 | |
| *** itlinux has joined #tripleo | 04:37 | |
| *** limao has joined #tripleo | 04:37 | |
| *** limao has quit IRC | 04:41 | |
| *** yamahata has joined #tripleo | 04:43 | |
| *** jaosorior has joined #tripleo | 04:44 | |
| *** sai_ has quit IRC | 04:44 | |
| *** lucasagomes has quit IRC | 04:44 | |
| *** rook_ has quit IRC | 04:44 | |
| *** sai_ has joined #tripleo | 04:45 | |
| *** rook has joined #tripleo | 04:46 | |
| *** itlinux has quit IRC | 04:47 | |
| *** rook is now known as Guest90114 | 04:47 | |
| *** lucasagomes has joined #tripleo | 04:47 | |
| *** lucasagomes has quit IRC | 04:51 | |
| *** lucasagomes has joined #tripleo | 04:53 | |
| *** nyechiel has joined #tripleo | 04:54 | |
| *** lucasagomes has quit IRC | 04:58 | |
| *** lucasagomes has joined #tripleo | 04:58 | |
| *** limao has joined #tripleo | 04:59 | |
| *** sai_ has quit IRC | 04:59 | |
| *** padkrish has quit IRC | 05:00 | |
| *** jaosorior has quit IRC | 05:00 | |
| *** padkrish has joined #tripleo | 05:01 | |
| *** jaosorior has joined #tripleo | 05:01 | |
| *** padkrish has quit IRC | 05:02 | |
| *** penick_ has joined #tripleo | 05:02 | |
| *** sai has joined #tripleo | 05:03 | |
| *** lucasagomes has quit IRC | 05:03 | |
| *** lucasagomes has joined #tripleo | 05:04 | |
| *** penick has quit IRC | 05:04 | |
| *** penick_ is now known as penick | 05:04 | |
| *** bana_k has joined #tripleo | 05:05 | |
| *** sai has quit IRC | 05:07 | |
| *** lucasagomes has quit IRC | 05:09 | |
| *** sai_ has joined #tripleo | 05:09 | |
| *** lucasagomes has joined #tripleo | 05:11 | |
| *** dmacpher-lunch is now known as dmacpher | 05:12 | |
| *** pgadiya has joined #tripleo | 05:15 | |
| *** sai_ has quit IRC | 05:17 | |
| *** Guest90114 has quit IRC | 05:18 | |
| *** rook_ has joined #tripleo | 05:18 | |
| *** sai_ has joined #tripleo | 05:19 | |
| *** lucasagomes has quit IRC | 05:19 | |
| *** lucasagomes has joined #tripleo | 05:23 | |
| *** sai_ has quit IRC | 05:25 | |
| *** sai_ has joined #tripleo | 05:27 | |
| *** lucasagomes has quit IRC | 05:27 | |
| *** lucasagomes has joined #tripleo | 05:28 | |
| *** lucasagomes has quit IRC | 05:33 | |
| *** rcernin has joined #tripleo | 05:34 | |
| *** lucasagomes has joined #tripleo | 05:35 | |
| *** sai_ has quit IRC | 05:35 | |
| *** rook_ has quit IRC | 05:36 | |
| *** sai has joined #tripleo | 05:37 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Hook internal TLS flag to apache-based services https://review.openstack.org/366075 | 05:38 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add flag for internal TLS https://review.openstack.org/365942 | 05:38 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add HAProxy TLS handled by certmonger as composable service https://review.openstack.org/356430 | 05:38 |
| *** rook has joined #tripleo | 05:38 | |
| *** rook is now known as Guest22457 | 05:38 | |
| *** karthiks has joined #tripleo | 05:40 | |
| *** sai has quit IRC | 05:44 | |
| *** lucasagomes has quit IRC | 05:45 | |
| *** sai has joined #tripleo | 05:45 | |
| *** lucasagomes has joined #tripleo | 05:47 | |
| *** itlinux has joined #tripleo | 05:47 | |
| *** oshvartz has joined #tripleo | 05:48 | |
| *** lucasagomes has quit IRC | 05:51 | |
| *** sai has quit IRC | 05:52 | |
| *** apetrich has quit IRC | 05:54 | |
| *** sai_ has joined #tripleo | 05:55 | |
| *** lucasagomes has joined #tripleo | 05:55 | |
| *** tzumainn has quit IRC | 05:55 | |
| *** apetrich has joined #tripleo | 05:56 | |
| *** pkovar has joined #tripleo | 05:59 | |
| *** lucasagomes has quit IRC | 05:59 | |
| *** aufi has joined #tripleo | 06:01 | |
| *** itlinux has quit IRC | 06:01 | |
| *** lucasagomes has joined #tripleo | 06:02 | |
| *** sai_ has quit IRC | 06:02 | |
| *** sai_ has joined #tripleo | 06:03 | |
| *** bana_k has quit IRC | 06:06 | |
| *** akuznetsov has joined #tripleo | 06:06 | |
| *** lucasagomes has quit IRC | 06:09 | |
| *** lucasagomes has joined #tripleo | 06:10 | |
| jaosorior | anybody else seeing this while trying to deploy the latest t-h-t? resources.StorageMgmtVirtualIP.properties: : Unknown Property PortName | 06:11 |
| *** akuznetsov has quit IRC | 06:12 | |
| *** rasca has joined #tripleo | 06:12 | |
| *** sai_ has quit IRC | 06:16 | |
| *** sai_ has joined #tripleo | 06:18 | |
| *** lucasagomes has quit IRC | 06:18 | |
| *** lucasagomes has joined #tripleo | 06:20 | |
| jaosorior | d0ugal: so, that commit worked for me | 06:20 |
| jaosorior | d0ugal: buuuut now I have (I think) t-h-t replated problems | 06:20 |
| jaosorior | d0ugal: uhm... well... what in use-environment.yaml looks kinda strange http://paste.openstack.org/show/576020/ is that ok? | 06:25 |
| *** saneax-_-|AFK is now known as saneax | 06:27 | |
| *** jaosorior has quit IRC | 06:29 | |
| *** jaosorior has joined #tripleo | 06:30 | |
| *** jprovazn has joined #tripleo | 06:32 | |
| *** pcaruana has joined #tripleo | 06:32 | |
| *** jlinkes_ has quit IRC | 06:33 | |
| *** liverpooler has joined #tripleo | 06:35 | |
| *** itlinux has joined #tripleo | 06:35 | |
| *** panda|Zz is now known as panda | 06:38 | |
| jaosorior | d0ugal: duuuude.... I think I found another issue X_x | 06:38 |
| *** sai_ has quit IRC | 06:40 | |
| *** Guest22457 has quit IRC | 06:41 | |
| *** sai has joined #tripleo | 06:42 | |
| *** rook has joined #tripleo | 06:43 | |
| *** jlinkes has joined #tripleo | 06:43 | |
| *** rook is now known as Guest99205 | 06:43 | |
| *** lucasagomes has quit IRC | 06:44 | |
| *** itlinux has quit IRC | 06:46 | |
| *** lucasagomes has joined #tripleo | 06:46 | |
| *** lucasagomes has quit IRC | 06:51 | |
| *** sai has quit IRC | 06:52 | |
| *** lucasagomes has joined #tripleo | 06:53 | |
| *** sai has joined #tripleo | 06:53 | |
| *** tremble has joined #tripleo | 06:54 | |
| *** tremble has quit IRC | 06:54 | |
| *** tremble has joined #tripleo | 06:54 | |
| *** hjensas has joined #tripleo | 06:55 | |
| *** hjensas has quit IRC | 06:55 | |
| *** hjensas has joined #tripleo | 06:55 | |
| openstackgerrit | Merged openstack/tripleo-common: Replace agent elements with package python-heat-agent-puppet https://review.openstack.org/369557 | 06:56 |
| d0ugal | jaosorior: it looks fine to me. | 06:57 |
| d0ugal | jaosorior: What's the new issue? | 06:57 |
| *** tesseract- has joined #tripleo | 06:57 | |
| *** bana_k has joined #tripleo | 06:57 | |
| jaosorior | d0ugal: submitting a bug report | 06:58 |
| jaosorior | the issue is not in t-h-t | 06:58 |
| jaosorior | but in tripleoclient actually :( | 06:58 |
| d0ugal | jaosorior: k, let me know when the bug is up | 06:58 |
| jaosorior | d0ugal: https://bugs.launchpad.net/tripleo/+bug/1623346 | 06:59 |
| openstack | Launchpad bug 1623346 in tripleo "Setting the user-environment for a plan results wrongly made resource-registry" [Undecided,New] | 06:59 |
| *** lucasagomes has quit IRC | 07:00 | |
| *** lucasagomes has joined #tripleo | 07:00 | |
| jaosorior | d0ugal: the issue is this function https://github.com/openstack/python-tripleoclient/blob/master/tripleoclient/v1/overcloud_deploy.py#L387 | 07:00 |
| *** sai has quit IRC | 07:01 | |
| d0ugal | jaosorior: right | 07:01 |
| d0ugal | jaosorior: hrm, I thought the checksum would solve this problem - what did I do wrong? | 07:01 |
| d0ugal | jaosorior: They don't all point to the same file | 07:02 |
| d0ugal | jaosorior: in your example I see one that is different? | 07:03 |
| *** sai has joined #tripleo | 07:03 | |
| jaosorior | d0ugal: which? | 07:03 |
| d0ugal | jaosorior: 'OS::TripleO::Network::StorageMgmt': user-files/10c5e3d056d3fe1f2f8c4ca83fb47f16-storage_mgmt.yaml, | 07:03 |
| d0ugal | oh wait | 07:03 |
| d0ugal | the formatting is weird. | 07:03 |
| jaosorior | yep | 07:03 |
| jaosorior | they're all user-files/10c5e3d056d3fe1f2f8c4ca83fb47f16-storage_mgmt.yaml | 07:03 |
| d0ugal | Odd. how is that happening? | 07:04 |
| jaosorior | d0ugal: the problem is that os.path.basename(path) only takes storage_mgmt.yaml | 07:04 |
| d0ugal | jaosorior: oh, it should be the hexdigest of 'path' not 'filename' - gah | 07:05 |
| *** lucasagomes has quit IRC | 07:05 | |
| jaosorior | d0ugal: so network/ports/storage_mgmt.yaml and network/storage_mgmt.yaml end up as storage_mgmt.yaml which ends up being checksumed | 07:05 |
| jaosorior | yep | 07:05 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Use the hexdigest of the path to make the filename unique in swift https://review.openstack.org/369859 | 07:06 |
| d0ugal | jaosorior: So, this should fix it ^ | 07:06 |
| d0ugal | jaosorior: I am going to work on some unit tests | 07:06 |
| jaosorior | awesome | 07:06 |
| jaosorior | I'll test that out meanwhile | 07:06 |
| *** shardy has joined #tripleo | 07:06 | |
| *** lucasagomes has joined #tripleo | 07:06 | |
| d0ugal | first I need some cofee | 07:06 |
| d0ugal | coffee even | 07:06 |
| jaosorior | d0ugal: how are we not hitting this in CI? | 07:06 |
| d0ugal | jaosorior: good question, that worries me! | 07:07 |
| jaosorior | pretty much with the stuff we've been seeing | 07:07 |
| jaosorior | nothing should work | 07:07 |
| jaosorior | it makes no sense | 07:07 |
| d0ugal | jaosorior: Indeed :/ | 07:07 |
| *** akuznetsov has joined #tripleo | 07:07 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Use the hexdigest of the path to make the filename unique in swift https://review.openstack.org/369859 | 07:07 |
| jaosorior | we supposedly are testing net-iso, but we can clearly see that that bug it shouldn't work at all X_x | 07:08 |
| *** athomas has joined #tripleo | 07:08 | |
| jaosorior | shardy: dude, is CI using pinned packages or something? | 07:08 |
| *** jistr is now known as jistr|mtgs | 07:09 | |
| *** jistr|mtgs is now known as jistr|mtg | 07:09 | |
| shardy | jaosorior: Yes, we use the current-tripleo pin which is promoted via the periodic job? | 07:10 |
| jaosorior | shardy: for python-tripleoclient too? | 07:10 |
| jaosorior | and tripleo-common? | 07:10 |
| *** sai has quit IRC | 07:10 | |
| shardy | jaosorior: No, those should use the currrent repo | 07:10 |
| jaosorior | shardy: those are REALLY broken, and it's really strange that's not showing up in CI | 07:10 |
| shardy | there was an issue because tripleoclient branched stable/newton | 07:11 |
| shardy | EmilienM fixed it AFAIK, sec | 07:11 |
| *** bana_k has quit IRC | 07:11 | |
| shardy | https://review.rdoproject.org/r/#/c/2186 | 07:12 |
| d0ugal | shardy: We have missed a whole bunch of issues with mistral and the jinja2 stuff - jaosorior is doing a great job at running into them. | 07:12 |
| *** florianf has joined #tripleo | 07:12 | |
| *** sai has joined #tripleo | 07:12 | |
| *** pmannidi has quit IRC | 07:13 | |
| jaosorior | I thought I was going crazy... since nobody seemed to be complaining about them | 07:13 |
| jaosorior | good thing d0ugal can assert they are actual issues | 07:13 |
| *** lucasagomes has quit IRC | 07:13 | |
| jaosorior | but yeah, in theory CI should be quite red | 07:14 |
| shardy | http://lists.openstack.org/pipermail/openstack-dev/2016-September/103432.html | 07:14 |
| jaosorior | so it seems to me that something is still not fetching the latest python-tripleoclient and tripleo-common | 07:14 |
| *** lucasagomes has joined #tripleo | 07:15 | |
| jaosorior | shardy: well, there's that and more bug reports | 07:15 |
| shardy | cistatus looks OK to me | 07:15 |
| *** dsneddon has joined #tripleo | 07:15 | |
| jaosorior | shardy: which is the problem, it shouldn't | 07:15 |
| d0ugal | shardy: That is the worrying thing :) | 07:15 |
| shardy | I'm in a meeting atm but will follow up later, hopefully folks can figure it out in the meanntime | 07:16 |
| *** shardy is now known as shardy_mtg | 07:16 | |
| jaosorior | d0ugal: is gerrit working for you? | 07:16 |
| d0ugal | jaosorior: A bunch of the issues are to do with re-deploying and updating the plan - CI doesn't have good coverage for that. | 07:16 |
| d0ugal | jaosorior: Yeah, it's fine | 07:16 |
| d0ugal | jaosorior: it actually doesn't even feel slow today | 07:16 |
| jaosorior | d0ugal: however, the checksum issue should break net-iso | 07:16 |
| d0ugal | jaosorior: I agree, that is bad :( | 07:17 |
| jaosorior | d0ugal: it completely froze to me :( can't download patchsets or access the UI | 07:17 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Remove the environments from Mistral when removing from Swift https://review.openstack.org/369486 | 07:17 |
| openstackgerrit | Dougal Matthews proposed openstack/instack-undercloud: Verify that the Deployment Plan creation was successful https://review.openstack.org/369247 | 07:19 |
| *** lucasagomes has quit IRC | 07:20 | |
| *** lucasagomes has joined #tripleo | 07:21 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Remove Ceph from the ha job https://review.openstack.org/362990 | 07:21 |
| openstackgerrit | Merged openstack/python-tripleoclient: Add `overcloud parameters set` to set Heat params in a plan https://review.openstack.org/360391 | 07:21 |
| jaosorior | d0ugal: can you pass a diff of the fix? | 07:22 |
| jaosorior | I'm trying to test it out | 07:22 |
| jaosorior | buit gerrit isn't working :( | 07:22 |
| jaosorior | so I'm gonna apply the patch and test it like that | 07:22 |
| d0ugal | jaosorior: Sure, 1 min | 07:23 |
| jaosorior | the nice thing about this is that at least I now have a better idea of what's going on | 07:23 |
| jaosorior | before I had very little clue how stuff was gonna end up working | 07:24 |
| *** ccamacho|afk has quit IRC | 07:24 | |
| d0ugal | jaosorior: http://paste.openstack.org/show/576035/ | 07:24 |
| d0ugal | jaosorior: That's why I broke everything, so now I'm not the only one :-P | 07:24 |
| jaosorior | hahaha | 07:24 |
| jaosorior | good trick | 07:25 |
| jaosorior | d0ugal: can you use pastebin or something else? seems that paste.openstack.org is also timing out (basically all openstack sites) | 07:25 |
| d0ugal | huh | 07:25 |
| *** jtomasek has joined #tripleo | 07:25 | |
| d0ugal | jaosorior: https://paste.fedoraproject.org/427856/38379681/ | 07:26 |
| *** akuznetsov has quit IRC | 07:26 | |
| d0ugal | jaosorior: it is a tiny change, might be easier just to tell you :) | 07:26 |
| *** ebarrera has joined #tripleo | 07:26 | |
| jaosorior | d0ugal: wait up | 07:27 |
| *** jaosorior has quit IRC | 07:27 | |
| *** jaosorior has joined #tripleo | 07:27 | |
| jaosorior | d0ugal: lol that was an easy change | 07:27 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Print the execution ID when starting Mistral workflows https://review.openstack.org/340711 | 07:27 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Add `openstack overcloud plan deploy` https://review.openstack.org/360305 | 07:27 |
| jaosorior | alright, I'll test it out | 07:27 |
| d0ugal | jaosorior: Thanks, hopefully it does the trick. I'm about to test it too | 07:28 |
| *** ccamacho has joined #tripleo | 07:28 | |
| *** jaosorior has quit IRC | 07:29 | |
| *** mcornea has joined #tripleo | 07:29 | |
| *** sai has quit IRC | 07:30 | |
| *** jaosorior has joined #tripleo | 07:30 | |
| *** akuznetsov has joined #tripleo | 07:31 | |
| *** lucasagomes has quit IRC | 07:31 | |
| openstackgerrit | Raoul Scarazzini proposed openstack/tripleo-quickstart: Removing locking hash on undercloud post role https://review.openstack.org/369870 | 07:32 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Check the result of the parent action when subclassing https://review.openstack.org/368081 | 07:32 |
| *** lucasagomes has joined #tripleo | 07:34 | |
| openstackgerrit | Raoul Scarazzini proposed openstack/tripleo-quickstart: Removing locking hash on undercloud post role https://review.openstack.org/369870 | 07:34 |
| jaosorior | d0ugal: that fix worked for me | 07:35 |
| *** Guest99205 has quit IRC | 07:35 | |
| jaosorior | hopefully gerrit starts working for me again soon so I can +2 it | 07:35 |
| *** fragatina has quit IRC | 07:37 | |
| *** zoli_gone-proxy is now known as zoliXXL | 07:37 | |
| *** abehl has joined #tripleo | 07:38 | |
| *** lucasagomes has quit IRC | 07:38 | |
| mcornea | jaosorior: good morning. did you manage to fix the undercloud issue you had yesterday? | 07:38 |
| *** sai has joined #tripleo | 07:39 | |
| *** rook_ has joined #tripleo | 07:39 | |
| *** zoliXXL is now known as zoli|trng | 07:39 | |
| *** lucasagomes has joined #tripleo | 07:39 | |
| *** jpena|off is now known as jpena | 07:42 | |
| jaosorior | mcornea: hey dude, first of all, thanks a lot for testing the BZ :D | 07:43 |
| mcornea | jaosorior: np, it was an easy one :) | 07:43 |
| jaosorior | mcornea: now, answering your question, my undercloud is pretty modified but stuff is working it seems | 07:43 |
| *** jmiu has quit IRC | 07:43 | |
| jaosorior | mcornea: you need a fix for zaqar, one for tripleo-common, and two for python-tripleoclient. That's the stuff that I can remember from the top of my head | 07:43 |
| mcornea | jaosorior: hm, do you know which one are for python-tripleoclient? | 07:44 |
| *** lucasagomes has quit IRC | 07:44 | |
| jaosorior | mcornea: gerrit is not working for me at the moment :( | 07:44 |
| jaosorior | d0ugal: ^^ | 07:44 |
| *** jmiu has joined #tripleo | 07:45 | |
| jaosorior | yay! it works now | 07:45 |
| jaosorior | alright | 07:45 |
| jaosorior | mcornea: this one https://review.openstack.org/#/c/369859/ | 07:45 |
| jaosorior | mcornea: and this https://review.openstack.org/#/c/369486/ | 07:46 |
| jaosorior | mcornea: then for tripleo-common you need this https://review.openstack.org/#/c/368760/ | 07:46 |
| *** lucasagomes has joined #tripleo | 07:47 | |
| mcornea | jaosorior: excellent, trying those. thanks | 07:47 |
| *** rook_ has quit IRC | 07:47 | |
| jaosorior | mcornea: and for zaqar this https://review.openstack.org/#/c/369285/ | 07:47 |
| *** sai has quit IRC | 07:47 | |
| *** sai has joined #tripleo | 07:48 | |
| *** rook has joined #tripleo | 07:48 | |
| *** rook is now known as Guest41006 | 07:49 | |
| *** zoli|trng is now known as zoli_gone-proxy | 07:49 | |
| *** ohamada has joined #tripleo | 07:50 | |
| *** onovy has quit IRC | 07:51 | |
| d0ugal | jaosorior: Thanks for testing! | 07:52 |
| *** lucasagomes has quit IRC | 07:52 | |
| jaosorior | mcornea: soo... why did rbrady downvote this? https://review.openstack.org/#/c/369486/ | 07:52 |
| jaosorior | d0ugal: ^^ | 07:52 |
| *** lucasagomes has joined #tripleo | 07:52 | |
| jaosorior | mcornea: sorry, that one wasn't directed to you | 07:52 |
| d0ugal | jaosorior: did you see his comment? | 07:52 |
| jaosorior | d0ugal: yeah, and then you explained what's up, and then there was another -1 | 07:53 |
| d0ugal | jaosorior: it was before I replied, gerrit just kept the -1 when I rebased | 07:54 |
| jaosorior | ah, true that | 07:54 |
| jaosorior | d0ugal: what's his timezone? | 07:54 |
| d0ugal | jaosorior: I'll speak with him about it today | 07:54 |
| d0ugal | jaosorior: he is US east | 07:54 |
| *** dsariel has joined #tripleo | 07:55 | |
| *** itlinux has joined #tripleo | 07:55 | |
| *** sai has quit IRC | 07:55 | |
| *** stendulker has joined #tripleo | 07:56 | |
| *** zoli_gone-proxy is now known as zoliXXL | 07:56 | |
| *** zoliXXL is now known as zoli|trng | 07:56 | |
| *** akuznetsov has quit IRC | 07:57 | |
| *** lucasagomes has quit IRC | 07:57 | |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Add Role Detail https://review.openstack.org/367993 | 07:58 |
| *** sai has joined #tripleo | 07:58 | |
| *** akuznetsov has joined #tripleo | 07:58 | |
| *** lucasagomes has joined #tripleo | 07:59 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Fetch internal certificates for HAProxy based on network https://review.openstack.org/366548 | 08:01 |
| *** sshnaidm|afk is now known as sshnaidm | 08:01 | |
| *** dtantsur is now known as dtantsur|brb | 08:03 | |
| *** lucasagomes has quit IRC | 08:03 | |
| *** Guest41006 has quit IRC | 08:05 | |
| *** sai has quit IRC | 08:05 | |
| *** itlinux has quit IRC | 08:05 | |
| *** gchamoul is now known as gchamoul|afk | 08:06 | |
| *** gchamoul|afk is now known as gchamoul | 08:06 | |
| *** rook_ has joined #tripleo | 08:06 | |
| *** sai has joined #tripleo | 08:07 | |
| *** lucasagomes has joined #tripleo | 08:08 | |
| *** gfidente has joined #tripleo | 08:09 | |
| *** jcoufal has joined #tripleo | 08:09 | |
| b00tcat | I am getting an error when building the overcloud-full image (with no additions) - anybody seen this RPM dependency error before? https://paste.fedoraproject.org/427862/47384063/ | 08:11 |
| *** sai has quit IRC | 08:11 | |
| *** rook_ has quit IRC | 08:11 | |
| *** sai has joined #tripleo | 08:12 | |
| *** derekh has joined #tripleo | 08:12 | |
| b00tcat | this package is defined in the `overcloud-controller` element | 08:13 |
| *** rook_ has joined #tripleo | 08:14 | |
| b00tcat | (in tripleo-puppet-elements) | 08:14 |
| jaosorior | b00tcat: seems to me like a package issue, have you tried asking in #rdo? | 08:16 |
| b00tcat | jaosorior: mmm no, but it's actually not a bad idea thanks ;-) | 08:18 |
| *** mbound has joined #tripleo | 08:18 | |
| *** rook_ has quit IRC | 08:20 | |
| *** rook_ has joined #tripleo | 08:21 | |
| *** mbound has quit IRC | 08:22 | |
| *** mbound has joined #tripleo | 08:22 | |
| *** liverpoo1er has joined #tripleo | 08:25 | |
| *** liverpooler has quit IRC | 08:26 | |
| *** lucasagomes has quit IRC | 08:27 | |
| *** sai has quit IRC | 08:27 | |
| gfidente | mwhahaha, you around? | 08:28 |
| *** rook_ has quit IRC | 08:29 | |
| panda | whois mwhahaha | 08:30 |
| panda | oops :) | 08:30 |
| *** akuznetsov has quit IRC | 08:30 | |
| *** sai has joined #tripleo | 08:30 | |
| panda | too nice of a nick | 08:30 |
| *** onovy has joined #tripleo | 08:30 | |
| jaosorior | gfidente: I think he's in the US | 08:30 |
| *** yamahata has quit IRC | 08:31 | |
| *** lucasagomes has joined #tripleo | 08:32 | |
| *** rook_ has joined #tripleo | 08:38 | |
| *** dsneddon has quit IRC | 08:40 | |
| *** dsneddon has joined #tripleo | 08:40 | |
| *** sai has quit IRC | 08:41 | |
| *** rook_ has quit IRC | 08:43 | |
| *** lucasagomes has quit IRC | 08:46 | |
| *** lucasagomes has joined #tripleo | 08:46 | |
| shadower | any puppet person able to +2/A this? https://review.openstack.org/#/c/368927/ | 08:46 |
| shadower | it's a small change | 08:47 |
| *** gchamoul is now known as gchamoul|afk | 08:47 | |
| *** gchamoul|afk is now known as gchamoul | 08:47 | |
| jaosorior | shadower: done | 08:49 |
| shadower | jaosorior: thanks! | 08:49 |
| *** akuznetsov has joined #tripleo | 08:56 | |
| EmilienM | Good morning | 08:56 |
| jaosorior | EmilienM: hey man, how's it going? | 08:57 |
| EmilienM | jaosorior: good and you? | 08:57 |
| EmilienM | Can someone + 2 https://review.openstack.org/#/c/367295/ ? | 08:59 |
| *** oneswig has joined #tripleo | 08:59 | |
| panda | periodic jobs passed except mitaka and liberty, but the error is not the same as yesterday, but still galera problems. replset, should be known | 08:59 |
| jaosorior | pretty good | 08:59 |
| jaosorior | EmilienM: seems like a pretty big change, had you ahd time to try it o ut? | 09:00 |
| EmilienM | I'm very worried about https://review.openstack.org/#/c/353506/ | 09:00 |
| *** paramite has joined #tripleo | 09:00 | |
| EmilienM | jaosorior: no but CI works fine | 09:00 |
| jaosorior | EmilienM: CI is not very reliable at the moment | 09:01 |
| EmilienM | Why that? | 09:01 |
| jaosorior | EmilienM: For instance, net-iso doesn't work. And somehow that's not being caught | 09:01 |
| jaosorior | as if we're not using the latest python-tripleoclient or tripleo-common | 09:01 |
| panda | jaosorior: links/examples ? | 09:02 |
| EmilienM | For tripleoclient, are we using stable/newton? | 09:02 |
| jaosorior | panda: this bug breaks net-iso https://bugs.launchpad.net/tripleo/+bug/1623346 | 09:03 |
| openstack | Launchpad bug 1623346 in tripleo "Setting the user-environment for a plan results wrongly made resource-registry" [Critical,In progress] - Assigned to Dougal Matthews (d0ugal) | 09:03 |
| jaosorior | panda, EmilienM: And then, for instance, updating the overcloud is broken too, due to other two bugs. | 09:04 |
| EmilienM | Where are launchpads? | 09:05 |
| EmilienM | Are they tagged CI? | 09:05 |
| *** lucasagomes has quit IRC | 09:05 | |
| jaosorior | EmilienM: tagged in CI? | 09:06 |
| EmilienM | Launchpad tags | 09:06 |
| *** itlinux has joined #tripleo | 09:06 | |
| jaosorior | EmilienM: https://bugs.launchpad.net/tripleo/+bug/1622683 | 09:07 |
| openstack | Launchpad bug 1622683 in tripleo "Updating plans breaks deployment" [Critical,In progress] - Assigned to Dougal Matthews (d0ugal) | 09:07 |
| EmilienM | panda mentioned two other bugs | 09:07 |
| jaosorior | https://bugs.launchpad.net/tripleo/+bug/1622720 | 09:07 |
| openstack | Launchpad bug 1622683 in tripleo "duplicate for #1622720 Updating plans breaks deployment" [Critical,In progress] - Assigned to Dougal Matthews (d0ugal) | 09:07 |
| jaosorior | and I'm not sure if there's something else | 09:07 |
| EmilienM | Ok | 09:07 |
| EmilienM | Is there some WIP? | 09:07 |
| jaosorior | EmilienM: yeah, they all have patches | 09:08 |
| EmilienM | Good. I'll look once on my computer in a few min | 09:08 |
| *** lucasagomes has joined #tripleo | 09:08 | |
| *** sai has joined #tripleo | 09:09 | |
| EmilienM | Brb | 09:09 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Add support for deploying Ceph RGW role https://review.openstack.org/289027 | 09:09 |
| *** coolsvap_ is now known as coolsvap | 09:09 | |
| openstackgerrit | Merged openstack/os-collect-config: Treat ec2 collector data as immutable https://review.openstack.org/364021 | 09:10 |
| *** sai has quit IRC | 09:13 | |
| *** lucasagomes has quit IRC | 09:14 | |
| jaosorior | panda: if you would like to help testing things out I can tell you how to reproduce it | 09:15 |
| *** chem has joined #tripleo | 09:15 | |
| *** itlinux has quit IRC | 09:16 | |
| *** lucasagomes has joined #tripleo | 09:17 | |
| jaosorior | shadower: hey dude, can you check this commit out? https://review.openstack.org/#/c/368643/ | 09:18 |
| shadower | jaosorior: looking | 09:19 |
| *** lucasagomes has quit IRC | 09:21 | |
| panda | jaosorior: ok, but I didn't understand why that bug is a reason why CI is not reliable | 09:23 |
| *** lucasagomes_ has joined #tripleo | 09:23 | |
| *** lucasagomes_ has quit IRC | 09:23 | |
| jaosorior | panda: cause net-iso shouldn't be working | 09:23 |
| jaosorior | panda: because of that bug, trying to deploy net-iso fails. And we clearly are not catching that in CI | 09:23 |
| jaosorior | and net-iso is supposedly one of the things we test... so what's going on? | 09:24 |
| *** akrivoka has joined #tripleo | 09:28 | |
| *** onovy has quit IRC | 09:29 | |
| *** mbound has quit IRC | 09:31 | |
| *** lucasagomes has joined #tripleo | 09:32 | |
| panda | jaosorior: ok, "CI not reliable" makes me think of inconsistent failures, very long queues for the tests, and result unavailability. Not catching a very subtle error, to me is not "not reliable". if using 10c5e3d056d3fe1f2f8c4ca83fb47f16-storage_mgmt.yaml did not break enything, it probably means that it contains a consistent (yet wrong) configuration. We'll have to add a test to give CI more coverage, I | 09:33 |
| panda | don't think I'd use "reliability" in this case | 09:33 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Deployment Plan page wizard styling https://review.openstack.org/369974 | 09:34 |
| tobias-fiberdata | Hi, | 09:35 |
| tobias-fiberdata | the installationguide for TripleO on tripleO.org is broken | 09:35 |
| tobias-fiberdata | there's dependency problems | 09:35 |
| tobias-fiberdata | there's packages that aint in the repo's | 09:35 |
| openstackgerrit | Merged openstack/puppet-tripleo: Add validations profile for tripleo https://review.openstack.org/368927 | 09:36 |
| ccamacho | tobias-fiberdata I followed it yesterday, can you put some logs? | 09:36 |
| jaosorior | panda: what do you mean a subtle error? It simply won't deploy | 09:36 |
| jaosorior | panda: and nit-picking what I said is really beside the point | 09:37 |
| lucasagomes | apparently creating an fedora 24 image on fedora 24 host is broken: http://paste.openstack.org/show/576069/ | 09:37 |
| lucasagomes | at least when the same kernel is used | 09:37 |
| ansiwen | chem: round? | 09:37 |
| *** sai has joined #tripleo | 09:37 | |
| tobias-fiberdata | we'll try again with a fresh install ccamacho, so nvm. | 09:38 |
| chem | ansiwen: hi | 09:38 |
| tobias-fiberdata | we'll see how it goes | 09:38 |
| jaosorior | panda: it doesn't end up in a "consistent" configuration. It actually not possible to deploy it because it will "consistently" result in an error. So that's pretty broken. | 09:38 |
| *** rook has joined #tripleo | 09:38 | |
| openstackgerrit | Gabriele Cerami proposed openstack/puppet-tripleo: Swift add_devices.pp IPv6 handling https://review.openstack.org/369976 | 09:38 |
| *** rook is now known as Guest4072 | 09:38 | |
| ansiwen | chem: my ec2 credentials request fails with "wrong number of arguments". Can you give me a hint how to do it correctly: http://logs.openstack.org/42/356442/3/check/gate-puppet-openstack-integration-3-scenario003-tempest-centos-7/4b76398/console.html#_2016-09-14_08_22_09_992146 | 09:39 |
| panda | jaosorior: the configuration you're trying that leads to deploy failure, is one of the tested configurations in CI ? are you using default parameters and you hit a failure while CI works perfectly with the same parameters ? | 09:41 |
| *** tosky has joined #tripleo | 09:41 | |
| jaosorior | panda: using same parameters as in the CI | 09:42 |
| shadower | jaosorior: now that the puppet dependency is merged, could you have a look at: https://review.openstack.org/#/c/322893/ pls? | 09:42 |
| jaosorior | shadower: sure man | 09:42 |
| shadower | cheers mate | 09:42 |
| jaosorior | shadower: hey, is it contemplated to add validations to CI? Would be pretty cool | 09:43 |
| ccamacho | EmilienM good morning! man, there are lot of puppet modules with lint issues, Im using this etherpad: https://etherpad.openstack.org/p/puppet-lint_upgrade to log all the submissions related | 09:44 |
| shadower | jaosorior: yeah absolutely. But we've focused on having all the necessary bits in place first | 09:46 |
| shadower | I think we'll have to optionally integrate the validations into the deployment workflow or something | 09:46 |
| *** sai has quit IRC | 09:46 | |
| jaosorior | shadower: makes sense | 09:46 |
| jaosorior | shadower: hey dude, did you get time to look at that failure in the undercloud upgrade job? | 09:46 |
| jaosorior | shadower: seems quite unrelated (neutron-db-sync issue). But still... any idea what's up? | 09:47 |
| shadower | jaosorior: not sure. The patch shouldn't actually affect it so I dunno :-( | 09:47 |
| *** sai has joined #tripleo | 09:47 | |
| chem | ansiwen: looking, sorry got distracted ... | 09:47 |
| jaosorior | shadower: yeah, it seemed unrelated indeed. I rechecked it to see if that happens again | 09:48 |
| ansiwen | chem: no problem | 09:48 |
| ansiwen | chem: and thanks for looking! | 09:48 |
| chem | ansiwen: I know this error :) | 09:48 |
| *** dtantsur|brb is now known as dtantsur | 09:48 | |
| shadower | jaosorior: yea, thanks | 09:48 |
| jaosorior | shadower: other than that funky thing. LGTM | 09:49 |
| panda | jaosorior: you have a link to the failed deploy ? | 09:50 |
| chem | ansiwen: the issue is in the provider | 09:52 |
| jaosorior | panda: as I mentioned earlier; I don't understand why CI is not hitting it (as if it's not using the latest python-tripleoclient or something of the sort). So no, I don't have a link failed deploy. Only pretty standard steps to reproduce it. | 09:52 |
| chem | ansiwen: which got merged :) | 09:53 |
| chem | ansiwen: I'll show you where it is | 09:53 |
| ansiwen | chem: I know the problem is the provider | 09:53 |
| ansiwen | chem: but what about the provider is the problem? ;-) | 09:53 |
| ansiwen | chem: https://review.openstack.org/gitweb?p=openstack/puppet-tempest.git;a=blob;f=lib/puppet/provider/tempest_ec2_credentials/openstack.rb;h=fab8d5d6789904ac6017d124545b8e164320195c;hb=5d81f40301f100b99fe37e6fea2453598b5cd844#l38 | 09:54 |
| *** sai has quit IRC | 09:54 | |
| *** jaosorior is now known as jaosorior_lunch | 09:54 | |
| chem | ansiwen: hold on, I know excatly what is failing, I'm just preparing my answer | 09:54 |
| ansiwen | chem: this is the line with the problem... but how can I add the additional parameters? just one string for all of them? | 09:55 |
| chem | ansiwen: hum ... It will be faster for me to do the review actualy, do you mind ? | 09:55 |
| ansiwen | chem: of course, as it is best for you! thanks! | 09:55 |
| *** pgadiya has quit IRC | 09:57 | |
| *** sai has joined #tripleo | 09:57 | |
| *** pgadiya has joined #tripleo | 09:58 | |
| chem | ansiwen: oups forgot to fix the rspec ... | 10:00 |
| ansiwen | chem: ? | 10:01 |
| chem | ansiwen: ha there is no unit test of this provider, never mind. | 10:01 |
| ansiwen | chem: yes, I wanted to write one, but suddenly it was already merged :-) | 10:02 |
| chem | ansiwen: we should got to #puppet-openstack for this discussion https://review.openstack.org/369993 | 10:02 |
| *** kbyrne has quit IRC | 10:02 | |
| EmilienM | ccamacho: I'll look a bit later, puppet lint is really low in priorities this week | 10:04 |
| EmilienM | ccamacho: trying to get newton release and CI working | 10:04 |
| ccamacho | np | 10:05 |
| *** kbyrne has joined #tripleo | 10:06 | |
| chem | ansiwen: did you get my last message on puppet-openstack ? | 10:06 |
| *** yolanda has joined #tripleo | 10:06 | |
| *** oneswig has quit IRC | 10:06 | |
| b00tcat | I'm facing another issue when building overcloud-full (again with no additions) | 10:07 |
| b00tcat | has anybody seen this? https://paste.fedoraproject.org/427894/73847534/ | 10:08 |
| b00tcat | I dont know where the line `os-svc-enable -n os-collect-config` that makes it fail comes from | 10:08 |
| EmilienM | shardy_mtg, jaosorior_lunch: do we have launchpads about "not testing netiso in CI"? | 10:08 |
| mwhahaha | gfidente: whats up? | 10:08 |
| gfidente | mwhahaha, I was trying to understand the service name/title story | 10:09 |
| gfidente | commented on the submission | 10:09 |
| gfidente | I get that they can be different | 10:09 |
| mwhahaha | k let me take a look | 10:09 |
| gfidente | also thanks for helping with that | 10:09 |
| gfidente | :) | 10:10 |
| mwhahaha | no problem | 10:10 |
| b00tcat | ah, it comes from tripleo-image-elements, but have no clue what that line's supposed to do | 10:10 |
| gfidente | b00tcat, how are you building the image? os-svc-enable should not be installed at all | 10:11 |
| gfidente | on which distro are you doing the build? | 10:11 |
| b00tcat | gfidente: building on a CentOS 7 box with `openstack overcloud image build --type overcloud-full --node-dist centos7` | 10:12 |
| *** zoli|trng is now known as zoli|lunch | 10:13 | |
| gfidente | b00tcat, also are you trying to build from master or a stable branch? | 10:15 |
| openstackgerrit | Merged openstack/tripleo-quickstart: Set default TERM for tput https://review.openstack.org/369455 | 10:15 |
| *** fzdarsky|afk has joined #tripleo | 10:16 | |
| *** matbu is now known as matbu|lunch | 10:17 | |
| *** itlinux has joined #tripleo | 10:17 | |
| *** liverpooler has joined #tripleo | 10:17 | |
| b00tcat | gfidente: from mitaka | 10:18 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: mysql: never add brackets to mysql_bind_host https://review.openstack.org/369369 | 10:18 |
| *** liverpoo1er has quit IRC | 10:19 | |
| EmilienM | dciabrin: ^ | 10:20 |
| dciabrin | EmilienM, saw that! | 10:22 |
| gfidente | EmilienM, I forgot why we can't use normalize_ip in that submission? | 10:25 |
| EmilienM | gfidente: we don't want brackets | 10:25 |
| EmilienM | gfidente: see dciabrin's comment | 10:26 |
| *** akuznetsov has quit IRC | 10:26 | |
| gfidente | so the brackets actually hurt in ipv6 ? | 10:26 |
| dciabrin | gfidente, apparently in recent version of galera yes | 10:26 |
| *** jaosorior_lunch is now known as jaosorior | 10:27 | |
| gfidente | okay so I can see why this fixes it but it's weird to have an ipv6:port couple without brackets | 10:27 |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Fix and improve the ansible-lint gate job https://review.openstack.org/366758 | 10:27 |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Fix ansible-lint errors in all playbooks and roles https://review.openstack.org/366837 | 10:27 |
| *** itlinux has quit IRC | 10:28 | |
| openstackgerrit | Saravanan KR proposed openstack/os-net-config: Add mac address to the DPDK mapping file https://review.openstack.org/370012 | 10:28 |
| *** hjensas has quit IRC | 10:31 | |
| *** milan has joined #tripleo | 10:31 | |
| *** ramishra has quit IRC | 10:31 | |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Use relative paths in ci-scripts for quickstart.sh https://review.openstack.org/360649 | 10:31 |
| *** limao has quit IRC | 10:31 | |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Document Third Party CI and developer usage https://review.openstack.org/360007 | 10:32 |
| *** ramishra has joined #tripleo | 10:33 | |
| *** jbadiapa has quit IRC | 10:34 | |
| *** mbound has joined #tripleo | 10:34 | |
| EmilienM | jaosorior: what is the netiso issue | 10:37 |
| *** thrash|g0ne is now known as thrash | 10:37 | |
| jaosorior | EmilienM: With the latest tripleoclient. Trying to deploy net-iso will fail, due to wrong manipulation of the environments | 10:40 |
| EmilienM | jaosorior: something was not backported? | 10:40 |
| EmilienM | or is it a new bug? | 10:40 |
| jaosorior | EmilienM: no, it's a bug. d0ugal has a fix and I tested it | 10:40 |
| jaosorior | seems to work | 10:40 |
| EmilienM | jaosorior: where? | 10:41 |
| EmilienM | the 2 patches that are reviewed already? | 10:41 |
| jaosorior | EmilienM: https://review.openstack.org/#/c/369859/ | 10:41 |
| EmilienM | jaosorior: +A | 10:41 |
| EmilienM | we need to backport it | 10:41 |
| EmilienM | I'll take care of it | 10:42 |
| jaosorior | EmilienM: what we don't understand is how that's not blowing up in CI. Which is why we got really concerned | 10:42 |
| jaosorior | EmilienM: thanks for checking it out dude | 10:42 |
| EmilienM | yes this is concerning | 10:42 |
| d0ugal | EmilienM: Thanks | 10:42 |
| EmilienM | I'll look after | 10:42 |
| EmilienM | do we have any other outstanding reviews today? | 10:42 |
| EmilienM | except the undercloud that waits for the dependency to merge? | 10:42 |
| EmilienM | d0ugal: thanks to you for the quick fixes | 10:43 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Add fluentd client service https://review.openstack.org/353506 | 10:44 |
| jaosorior | EmilienM: there are two other fixes by d0ugal that are pending to be merged. But one needs further discussing with rbrady | 10:44 |
| EmilienM | ok | 10:44 |
| jaosorior | EmilienM: got any reviews yourself? | 10:45 |
| EmilienM | now I'm looking why we didn't catch the netiso problem | 10:45 |
| EmilienM | jaosorior: nothing outstanding | 10:45 |
| EmilienM | I have some patches but not super urgent | 10:45 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Remove the environments from Mistral when removing from Swift https://review.openstack.org/369486 | 10:45 |
| d0ugal | jaosorior, EmilienM: I just updated 200~https://review.openstack.org/#/c/369486/ | 10:45 |
| d0ugal | with a link to a bug so we improve it later | 10:46 |
| d0ugal | I think rbrady will be happy with that now | 10:46 |
| EmilienM | jaosorior: this one is interested for debug (re EPEL issue) https://review.openstack.org/#/c/369305/ | 10:46 |
| jaosorior | d0ugal: thanks dude. I'll test it out again just to make sure. Great job man! | 10:46 |
| d0ugal | jaosorior: cheers. | 10:46 |
| d0ugal | EmilienM: One of the errors we missed that partially broke things was because we didn't check the plan creation in instack-undercloud. I added that here: https://review.openstack.org/#/c/369247/ | 10:47 |
| d0ugal | EmilienM: It's less urgent, but could help catch some errors in the future. | 10:47 |
| EmilienM | d0ugal: indeeed | 10:47 |
| EmilienM | yes | 10:47 |
| EmilienM | I'll make sure it lands asap | 10:47 |
| d0ugal | Thanks | 10:48 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: get_host_info: get repos list https://review.openstack.org/369305 | 10:48 |
| EmilienM | jaosorior: rebasing to check if we have epel now :) ^ | 10:48 |
| *** hjensas has joined #tripleo | 10:50 | |
| *** shardy_mtg is now known as shardy | 10:51 | |
| *** chem has quit IRC | 10:51 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Hook internal TLS flag to apache-based services https://review.openstack.org/366075 | 10:53 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add flag for internal TLS https://review.openstack.org/365942 | 10:53 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add HAProxy TLS handled by certmonger as composable service https://review.openstack.org/356430 | 10:53 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Set client protocol for glance registry client https://review.openstack.org/370029 | 10:53 |
| *** social has quit IRC | 10:54 | |
| *** skramaja has quit IRC | 10:55 | |
| *** skramaja has joined #tripleo | 10:58 | |
| *** itlinux has joined #tripleo | 10:58 | |
| *** akrivoka has quit IRC | 11:02 | |
| *** fzdarsky|afk has quit IRC | 11:03 | |
| *** chem has joined #tripleo | 11:03 | |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: Customize undercloud and overcloud with virt-customize https://review.openstack.org/361112 | 11:03 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: Customize undercloud and overcloud with virt-customize https://review.openstack.org/361112 | 11:04 |
| *** electrofelix has joined #tripleo | 11:04 | |
| jaosorior | ccamacho: gonna go to this? http://www.meetup.com/MAD-for-OpenStack/events/233790123/ | 11:06 |
| ccamacho | jaosorior, yahp | 11:06 |
| ccamacho | are you in Madrid? | 11:06 |
| jaosorior | ccamacho: nah | 11:06 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: Customize undercloud and overcloud with virt-customize https://review.openstack.org/361112 | 11:06 |
| jaosorior | just saw it while searching for openstack events | 11:06 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/tripleo-quickstart: Add Undercloud reboot https://review.openstack.org/367253 | 11:07 |
| ccamacho | yeahp, probably Ill ask all of you for some advice as now im organizing http://www.meetup.com/es-ES/OpenStack-Spain/ and https://twitter.com/openstackspain | 11:07 |
| *** shardy is now known as shardy_mtg | 11:07 | |
| jaosorior | duuude nice! | 11:08 |
| ccamacho | Im thinking in something like to do some "OpenStack happy Thursdays" | 11:08 |
| *** pgadiya has quit IRC | 11:08 | |
| *** sai has quit IRC | 11:08 | |
| ccamacho | 1 tech discussion and another off-topic one (Each month) | 11:08 |
| *** itlinux has quit IRC | 11:09 | |
| *** Guest4072 has quit IRC | 11:09 | |
| *** pgadiya has joined #tripleo | 11:09 | |
| *** pgadiya has quit IRC | 11:09 | |
| *** sai has joined #tripleo | 11:10 | |
| *** pgadiya has joined #tripleo | 11:10 | |
| jaosorior | ccamacho: gonna bring tripleo there? :P | 11:11 |
| EmilienM | I did last night in Montreal meetup | 11:11 |
| EmilienM | people was interested | 11:11 |
| ccamacho | yeeeah!! | 11:11 |
| jaosorior | EmilienM: are there some people that have tried it already? | 11:12 |
| EmilienM | jaosorior: not in the room, but it was a really small meetup | 11:12 |
| *** rook has joined #tripleo | 11:12 | |
| EmilienM | jaosorior: most of people here use manual deployments driven by puppet+ansible or ansible | 11:12 |
| *** bvandenh has quit IRC | 11:12 | |
| *** rook is now known as Guest63439 | 11:12 | |
| EmilienM | though I found some folks using our puppet modules! ahah | 11:13 |
| jaosorior | haha well that's pretty cool too | 11:13 |
| openstackgerrit | Merged openstack/tripleo-common: Add template processing to the update plan workflow. https://review.openstack.org/368760 | 11:14 |
| *** akrivoka has joined #tripleo | 11:14 | |
| *** stendulker has quit IRC | 11:16 | |
| *** jpena is now known as jpena|lunch | 11:17 | |
| colonwq | EmilienM, Thanks for the answer last night. I was trying to see if there was a way to run a subset of tests instead of all. | 11:19 |
| colonwq | which puppet modules were they using? | 11:20 |
| *** bvandenh has joined #tripleo | 11:28 | |
| *** chem has quit IRC | 11:29 | |
| openstackgerrit | Brad P. Crochet proposed openstack/python-tripleoclient: Revert "Upgrades: Add initialization stage to 'overcloud upgrade'" https://review.openstack.org/357192 | 11:30 |
| openstackgerrit | Brad P. Crochet proposed openstack/python-tripleoclient: Revert "Upgrades: Add finish stage to 'overcloud upgrade'" https://review.openstack.org/357194 | 11:30 |
| openstackgerrit | Brad P. Crochet proposed openstack/python-tripleoclient: Revert "Upgrades: Add 'stack upgrade' command" https://review.openstack.org/357195 | 11:30 |
| *** chem has joined #tripleo | 11:30 | |
| *** zoli|lunch is now known as zoli|trng | 11:30 | |
| *** chem has quit IRC | 11:30 | |
| *** chem has joined #tripleo | 11:30 | |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common: Revert "Upgrades: Add initialization to StackUpgradeManager" https://review.openstack.org/357219 | 11:31 |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common: Revert "Upgrades: Add post-upgrade stack update" https://review.openstack.org/357220 | 11:31 |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common: Revert "Upgrades: Add StackUpgradeManager" https://review.openstack.org/357221 | 11:31 |
| *** sai has quit IRC | 11:32 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Convert allNodesConfig properties to composable jinja2 https://review.openstack.org/365794 | 11:32 |
| *** sai has joined #tripleo | 11:33 | |
| *** openstackgerrit has quit IRC | 11:34 | |
| *** openstackgerrit has joined #tripleo | 11:34 | |
| *** hjensas has quit IRC | 11:37 | |
| *** bvandenh has quit IRC | 11:37 | |
| gfidente | d0ugal, I think I need help troubleshooting why deploystackaction fails | 11:37 |
| gfidente | have a min? | 11:37 |
| gfidente | it fails before the heat stack is created and after the plan is loaded in swift | 11:38 |
| jaosorior | gfidente: I've been dealing with some of those issues lately. What's up? | 11:38 |
| *** mbound has quit IRC | 11:38 | |
| gfidente | jaosorior, hey thanks | 11:38 |
| gfidente | message is u'message': u"Failed to run action [action_ex_id=0018c1a7-0443-4a52-8cbf-aeb3671079be, action_cls='<class 'mistral.actions.action_factory.DeployStackAction'>', attributes='{}', params='{u'container': u'overcloud', u'timeout': 240}']\n 'Result' object has no attribute 'copy'", | 11:39 |
| jaosorior | nwo that's a new one | 11:39 |
| jaosorior | is that the only error you're seeing? | 11:39 |
| jaosorior | gfidente: is it stable/newton or master? | 11:39 |
| d0ugal | gfidente, jaosorior: it's old :-D | 11:39 |
| d0ugal | I have a fix... | 11:39 |
| gfidente | d0ugal, it's master | 11:40 |
| d0ugal | gfidente: Sure, I mean it's not new to me :) | 11:40 |
| gfidente | though the reason I am sad is not the error itself but rather that I am unable to troubleshoot the | 11:40 |
| d0ugal | gfidente: You are getting an error, but it is being hidden. | 11:40 |
| gfidente | yeah | 11:40 |
| d0ugal | gfidente: https://review.openstack.org/#/c/368081/ | 11:40 |
| d0ugal | That will reveal that actual error. | 11:40 |
| *** bfournie has quit IRC | 11:41 | |
| d0ugal | jaosorior, EmilienM - so this would probably be good to add, I only done it in theory I didn't know anyone was hitting it but it seems gfidente is. https://review.openstack.org/#/c/368081/ | 11:41 |
| gfidente | ok trying it out | 11:42 |
| d0ugal | gfidente: Thanks - you might find the real error in the Mistral log | 11:42 |
| d0ugal | I mean, you should find it there. | 11:42 |
| *** sai has quit IRC | 11:43 | |
| *** Guest63439 has quit IRC | 11:43 | |
| gfidente | InvalidSchema: No connection adapters were found for 'file:///home/stack/deploy/tripleo-heat-templates/puppet/manifests/overcloud_controller_pacemaker.pp | 11:43 |
| *** sai has joined #tripleo | 11:44 | |
| *** matbu|lunch is now known as matbu | 11:46 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Provide: return task result in case of failure https://review.openstack.org/367552 | 11:46 |
| *** rook_ has joined #tripleo | 11:47 | |
| gfidente | d0ugal, looks like it didn't like file:// ? | 11:48 |
| d0ugal | gfidente: hrm, interesting. Do you have the full error? | 11:48 |
| d0ugal | I think I can guess roughly where it would be | 11:49 |
| *** lucasagomes is now known as lucas-hungry | 11:50 | |
| *** DrBacchus has joined #tripleo | 11:55 | |
| tobias-fiberdata | ccamacho, we reinstalled the machine running the undercloud, and installed with newton instead, and that seems to work. we used mitaka earlier and that failed on some package dependencies. So it's perhaps worth to check that out. we used the guide and followed the mitaka steps for repos. | 11:55 |
| ccamacho | tobias-fiberdata, thanks for re-checking, Ill add it to my queue to deploy mitaka (today afternoon/tomorrow morning) | 11:57 |
| *** ohamada has quit IRC | 11:58 | |
| *** ohamada_ has joined #tripleo | 11:58 | |
| *** rbowen has quit IRC | 11:59 | |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-heat-templates: Fix nested list for enabled_services https://review.openstack.org/370069 | 12:00 |
| tobias-fiberdata | ccamacho, great! from what we can see the official release of newton is october 6th? how much does the code of newton actually change in that time? As we are up to deploying a new overcloud for production. It for me feels scary to deploy something that hasnt been tested sharply in production yet. but i dont know, is that a bad idea? | 12:00 |
| *** bvandenh has joined #tripleo | 12:00 | |
| *** jbadiapa has joined #tripleo | 12:01 | |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-heat-templates: Fix nested list for enabled_services https://review.openstack.org/370069 | 12:01 |
| tobias-fiberdata | ccamacho, went through the guide and getting this, even with newton | 12:03 |
| tobias-fiberdata | http://paste.openstack.org/show/576119/ | 12:03 |
| tobias-fiberdata | so sadly i was to optimistic XD | 12:03 |
| *** morazi has joined #tripleo | 12:04 | |
| ccamacho | tobias-fiberdata give me a few mins to check it, I have installed master yesterday and it worked the undercloud install, let me provision and run it for mitaka to check some logs there | 12:06 |
| tobias-fiberdata | this error was with newton | 12:07 |
| tobias-fiberdata | but it might be related to ip6tables | 12:07 |
| *** jayg|g0n3 is now known as jayg | 12:09 | |
| *** itlinux has joined #tripleo | 12:09 | |
| *** rook_ has quit IRC | 12:10 | |
| *** sai has quit IRC | 12:10 | |
| *** rook has joined #tripleo | 12:10 | |
| *** rook is now known as Guest79926 | 12:10 | |
| *** sai has joined #tripleo | 12:11 | |
| jaosorior | EmilienM: hey dude, would really use your feedback on this commit https://review.openstack.org/#/c/366548/ | 12:11 |
| beagles | bnemec: EmilienM: I pinged on this yesterday, but I think it was around the time of a netsplit event and you may have missed it... | 12:13 |
| beagles | bnemec: EmilienM: I've marked it as RC-2 for the time being - mainly because I'm not sure what the best way to fix it yet | 12:14 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Expose parameter to enable combination alarms https://review.openstack.org/363748 | 12:15 |
| *** bfournie has joined #tripleo | 12:15 | |
| *** itlinux has quit IRC | 12:20 | |
| jaosorior | easy one liner: https://review.openstack.org/#/c/370029/ :D | 12:23 |
| jaosorior | if someone can check it out | 12:23 |
| slagle | jaosorior: oh was that deferred until ocata? | 12:27 |
| slagle | jaosorior: was just looking at the blueprint | 12:28 |
| *** pgadiya has quit IRC | 12:28 | |
| tobias-fiberdata | ccamacho, dont bother, updated the kernel and newton worked | 12:29 |
| ccamacho | tobias-fiberdata, thanks for the updat | 12:29 |
| jaosorior | slagle: well, that commit you just +2ed is required for the code of the blueprint to work. However, it pretty much fixes a bug (which people are not hitting cause nobody but me is configuring TLS everywhere) | 12:31 |
| slagle | gfidente: is this complete? https://blueprints.launchpad.net/tripleo/+spec/ceph-rgw-integration | 12:31 |
| gfidente | slagle, hey! yes! | 12:31 |
| jaosorior | slagle: but yeah, the certmonger blueprint was deferred | 12:31 |
| jaosorior | any idea when we branch? | 12:32 |
| *** chem has quit IRC | 12:32 | |
| slagle | jaosorior: ok, so we can't merge it due to FF | 12:33 |
| slagle | not sure about the branches, but probably soon'ish | 12:33 |
| jaosorior | slagle: that CR it's not a feature, but a bugfix | 12:33 |
| d0ugal | shardy_mtg: ping, if you get a spare bit of time today | 12:33 |
| slagle | jaosorior: yea, understood, but it's marked as implementing a bp and not associated with a bug :) | 12:34 |
| jaosorior | good point | 12:34 |
| jaosorior | let me fix that | 12:34 |
| slagle | and we've never supported tls everywhere....so it could go either way | 12:34 |
| jaosorior | slagle: well, it will always be set to 'http' until we support it in ocata | 12:34 |
| jaosorior | slagle: and having that bugfix is pretty harmless in that sense. | 12:35 |
| slagle | jaosorior: yes, i understand what you're saying. but we are in FF. and it's harmless not have it right? | 12:35 |
| slagle | just saying :) | 12:35 |
| slagle | i guess if someone feels strongly about pushing it through... | 12:36 |
| *** chem has joined #tripleo | 12:36 | |
| slagle | i'm just personally taking a different view at this stage in the cycle | 12:36 |
| jaosorior | slagle: unless our consultants actually pull off doing TLS everywhere through extraconfig (which they are attempting), then it'll actually benefit those folks | 12:36 |
| *** shardy_mtg is now known as shardy | 12:39 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Hook internal TLS flag to apache-based services https://review.openstack.org/366075 | 12:39 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add flag for internal TLS https://review.openstack.org/365942 | 12:39 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add HAProxy TLS handled by certmonger as composable service https://review.openstack.org/356430 | 12:39 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add option to specify Certmonger CA https://review.openstack.org/368643 | 12:39 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Set client protocol for glance registry client https://review.openstack.org/370029 | 12:39 |
| shardy | d0ugal: Hi | 12:39 |
| d0ugal | shardy: Hey - so, I have found a problem with gfidente and I'm not sure how easy it will be to solve. | 12:40 |
| d0ugal | shardy: we use heatclient to find all the files to upload to swift, but in the process of it doing that it resolves the paths in type: and get_file: | 12:41 |
| jaosorior | slagle: sure, the CA one I'm actually fine with the -2 | 12:41 |
| d0ugal | shardy: which then means the yaml files have file:// everywhere, which make no sense once they are in swift | 12:41 |
| jaosorior | makes sense | 12:41 |
| d0ugal | shardy: are you familiar with this functionality in heatclient? I assume that is where it is. or do you know where I can get help? | 12:41 |
| d0ugal | I need to be able to hook into it I guess somehow :/ | 12:42 |
| shardy | jaosorior, slagle: we were planning to branch for RC1 tomorrow, but Emilien isn't planning to branch the puppet modules yet, so we may delay by a week or so to align with that | 12:42 |
| shardy | which will also reduce some of the backports required for the remaining FFE's & bugfixes | 12:42 |
| slagle | shardy: yea, that's fine, i just want to make sure we are honoring FF in the meantime | 12:43 |
| jaosorior | shardy: what's missing for the puppet-modules? | 12:43 |
| shardy | slagle: ack, agreed, thanks! | 12:43 |
| *** pgadiya has joined #tripleo | 12:43 | |
| slagle | we ought to be focused on newton regardless, but, i understand some have moved on | 12:43 |
| *** pgadiya has quit IRC | 12:43 | |
| shardy | jaosorior: No idea, EmilienM can probably provide details on why | 12:43 |
| *** pgadiya has joined #tripleo | 12:43 | |
| shardy | d0ugal: Yeah that's from heatclient, if it's not what you want then you can just rewrite the prefix in the files map passed to heat | 12:44 |
| shardy | d0ugal: the keys can be anything, but they must match the resource_registry entries, and any direct file references in the heat templates | 12:44 |
| shardy | d0ugal: since we're resolving the files via heat, the file:// prefix sounds like a heat bug | 12:45 |
| shardy | e.g we are resolving them via swift now since https://review.openstack.org/#/c/365735/ | 12:46 |
| *** sai has quit IRC | 12:46 | |
| gfidente | d0ugal, shardy from heatclient looks like that is because we pass a path | 12:46 |
| *** jtomasek has quit IRC | 12:46 | |
| gfidente | and not a url, if we were to pass a url it wouldn't use file:// | 12:46 |
| shardy | d0ugal: I'm not sure how much of a problem it is - provided the keys are consistent it should work, but I agree the prefix may be misleading | 12:46 |
| gfidente | but that means uploading before the heatclient call? | 12:46 |
| d0ugal | shardy: so, for example, this type: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/cinder-volume.yaml#L66 | 12:47 |
| *** Guest79926 has quit IRC | 12:47 | |
| shardy | gfidente: Yeah, we must create the plan in swift before the call to get_template_contents: | 12:47 |
| shardy | https://review.openstack.org/#/c/365735/9/tripleoclient/v1/overcloud_deploy.py | 12:47 |
| gfidente | ah right we pass template_object sorry | 12:47 |
| d0ugal | shardy: it becomes something like: {"type": "file:///home/stack/deploy/tripleo-heat-templates/puppet/services/ceph-base.yaml" | 12:47 |
| shardy | gfidente: Yeah, I changed that quite recently | 12:47 |
| d0ugal | shardy: (I picked a differnt type, but you should get the idea) | 12:48 |
| shardy | d0ugal: Yeah, that's expected | 12:48 |
| larsks | shardy, if you have a moment, I'd like to bug you w/ some questions about the brave new world of overcloud.j2.yaml. | 12:48 |
| shardy | that type will match a key in the files map | 12:48 |
| jaosorior | slagle: commits for documentation are OK still, right? | 12:48 |
| gfidente | and resolve_template_get_files is doing the same for get_file: | 12:48 |
| shardy | jaosorior: yes | 12:48 |
| shardy | larsks: Hi, sure | 12:49 |
| *** dsariel has quit IRC | 12:49 | |
| jaosorior | Can someone check these two doc commits out? https://review.openstack.org/#/c/348038/ https://review.openstack.org/#/c/368601/ | 12:49 |
| *** proffalken has joined #tripleo | 12:49 | |
| larsks | shardy, first, what are the requirements for running an overcloud deploy w/ tht master? I upgraded python-tripleoclient and tripleo-common, but was getting a "Workflow not found" error. | 12:50 |
| *** hjensas has joined #tripleo | 12:50 | |
| *** hjensas has quit IRC | 12:50 | |
| *** hjensas has joined #tripleo | 12:50 | |
| jaosorior | larsks: you might need to update your undercloud | 12:50 |
| shardy | larsks: You need the latest master tripleoclient and tripleo-common, and you need to either manually update the mistral actions/workflows or re-run openstack undercloud install (which does this) | 12:51 |
| larsks | shardy, jaosorior, thanks. I'll give that a try. | 12:51 |
| larsks | shardy, second question, is the only way to set things like ControllerCount via the cli now (i.e., is there a way to set that via an environment file)? | 12:51 |
| larsks | Looking at overcloud.j2.yaml and roles_data.yml I wasn't sure. | 12:52 |
| shardy | larsks: mistral-db-manage populate populates any new actions (then restart mistral-engine and possibly mistral-executor) | 12:52 |
| shardy | larsks: https://github.com/openstack/instack-undercloud/blob/master/elements/undercloud-install/os-refresh-config/post-configure.d/98-undercloud-setup#L99 | 12:53 |
| *** sai has joined #tripleo | 12:53 | |
| shardy | larsks: that shows how to update the workflows | 12:53 |
| larsks | shardy, thanks, I will take a look. | 12:53 |
| shardy | larsks: No you can still set ControllerCount via the environment exactly the same as before | 12:54 |
| shardy | larsks: we just generate the parameter definition on the fly now | 12:54 |
| shardy | larsks: you can check by swift download overcloud/overcloud.yaml after triggering the deployment or creating the plan | 12:54 |
| shardy | that'll show your rendered overcloud template | 12:54 |
| larsks | shardy, ah, I see that now that I am looking at overcloud.j2.yaml after a full night's sleep. | 12:55 |
| larsks | Thanks! | 12:55 |
| shardy | larsks: np - I need to write some docs/blog about this but been travelling this week | 12:55 |
| *** rook_ has joined #tripleo | 12:55 | |
| *** jpena|lunch is now known as jpena | 12:55 | |
| *** pgadiya has quit IRC | 12:57 | |
| *** tzumainn has joined #tripleo | 12:57 | |
| *** lucas-hungry is now known as lucasagomes | 12:57 | |
| *** larsks has left #tripleo | 12:58 | |
| *** larsks has joined #tripleo | 12:59 | |
| *** jaosorior has quit IRC | 13:01 | |
| *** dprince has joined #tripleo | 13:01 | |
| *** dprince has quit IRC | 13:01 | |
| *** jaosorior has joined #tripleo | 13:02 | |
| *** pradk has quit IRC | 13:03 | |
| *** ccamacho is now known as ccamacho|lunch | 13:04 | |
| *** fzdarsky|afk has joined #tripleo | 13:05 | |
| *** sai has quit IRC | 13:05 | |
| *** lblanchard has joined #tripleo | 13:06 | |
| *** mgarciam has joined #tripleo | 13:06 | |
| *** chem has quit IRC | 13:07 | |
| *** sai has joined #tripleo | 13:07 | |
| *** Goneri has joined #tripleo | 13:08 | |
| *** fzdarsky|afk is now known as fzdarsky | 13:08 | |
| *** proffalken has quit IRC | 13:09 | |
| *** fultonj has joined #tripleo | 13:09 | |
| *** chem has joined #tripleo | 13:10 | |
| *** onovy has joined #tripleo | 13:12 | |
| *** cwolferh has quit IRC | 13:15 | |
| *** rlandy has joined #tripleo | 13:17 | |
| *** jprovazn has quit IRC | 13:18 | |
| *** myoung|bbl is now known as myoung | 13:20 | |
| *** itlinux has joined #tripleo | 13:20 | |
| ayoung | so just reran quickstart, and looking at the RPMs in the image from master, some of them are quite old. For example python-os-client-config-1.18.0-0.20160802185324.51b4cbe.el7.centos.src.rpm2016-08-02 18:53 63K | 13:21 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: Customize undercloud and overcloud with virt-customize https://review.openstack.org/370114 | 13:22 |
| ayoung | the Yum directory was update 2016-08-23 15:52 | 13:23 |
| ayoung | is there a reason we locked the RPMs to a specific version? | 13:23 |
| *** shardy has quit IRC | 13:23 | |
| slagle | rbrady: d0ugal : can we call this one implemented now? https://blueprints.launchpad.net/tripleo/+spec/mistral-deployment-library | 13:23 |
| slagle | seems like the last patches are covered by the other blueprint | 13:24 |
| slagle | and i dont see any other patches needing review on https://etherpad.openstack.org/p/tripleo-mistral-api | 13:24 |
| *** jistr|mtg is now known as jistr | 13:24 | |
| *** rhallisey has joined #tripleo | 13:24 | |
| *** dsneddon has quit IRC | 13:25 | |
| larsks | I am trying to run a deploy and I get: Exception creating plan: Error occurred creating plan: Duplicate entry for Environment: ['name'] | 13:25 |
| larsks | What is that trying to tell me? | 13:25 |
| *** dsariel has joined #tripleo | 13:26 | |
| larsks | jaosorior, any idea? | 13:26 |
| mcornea | shardy: hi! how does the node tagging work in the context of custom roles? we usually pass the flavors via the cli e.g --compute-flavor. how ar we going to specify those for the custom roles? | 13:26 |
| *** athomas has quit IRC | 13:26 | |
| larsks | mcornea, I think shardy is not here right now. | 13:27 |
| mcornea | larsks: oops, missed him | 13:27 |
| jaosorior | larsks: hey dude, you need some patches | 13:27 |
| jaosorior | wait up | 13:27 |
| jaosorior | let me fetch them | 13:28 |
| jaosorior | larsks: but yeah, that one is a known issue | 13:28 |
| jaosorior | larsks: you need these for tripleoclient https://review.openstack.org/369486 https://review.openstack.org/369859 | 13:28 |
| larsks | jaosorior, thanks, taking a look... | 13:29 |
| jaosorior | larsks: you need this for tripleo-common https://review.openstack.org/#/c/368760/ | 13:29 |
| jaosorior | larsks: and this from zaqar https://review.openstack.org/#/c/369285/ | 13:29 |
| larsks | My day is going to be a lot more complicated than I had anticipated. | 13:29 |
| jaosorior | larsks: haha, yep, been having these issues from yesterday, but thankfully there are now fixes to all of them | 13:30 |
| jaosorior | with those I've been able to continue my development workflow | 13:30 |
| larsks | jaosorior, that last patch was for zaqar? | 13:30 |
| jaosorior | yes | 13:30 |
| larsks | Where is that used? | 13:30 |
| jaosorior | larsks: so mistral uses zaqar, and the deal is that we pass around environments and such, which might contain periods. now zaqar uses mongodb, so if it finds a period in a key it doesn't allow it | 13:31 |
| larsks | Got it. | 13:31 |
| jaosorior | larsks: so that commit gets rid of that limitation | 13:31 |
| rbrady | slagle: I believe so. We're just fixing bugs at this point | 13:32 |
| openstackgerrit | Gabriele Cerami proposed openstack/puppet-tripleo: Swift add_devices.pp IPv6 handling https://review.openstack.org/369976 | 13:32 |
| *** dsariel has quit IRC | 13:34 | |
| *** itlinux has quit IRC | 13:34 | |
| *** jcoufal has quit IRC | 13:34 | |
| *** cwolferh has joined #tripleo | 13:34 | |
| slagle | rbrady: great, thanks | 13:39 |
| *** dsariel has joined #tripleo | 13:40 | |
| openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart: Collapse devmode definitions on release file https://review.openstack.org/369571 | 13:41 |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-heat-templates: Remove batch_create property from the CephMon SFdeploymentGroup https://review.openstack.org/370127 | 13:41 |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-heat-templates: Remove batch_create property from the CephMon SFdeploymentGroup https://review.openstack.org/370127 | 13:42 |
| openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart: Collapse devmode definitions on release file https://review.openstack.org/369571 | 13:43 |
| *** jprovazn has joined #tripleo | 13:46 | |
| *** egafford has joined #tripleo | 13:48 | |
| *** jistr is now known as jistr|mtg | 13:49 | |
| *** mbound has joined #tripleo | 13:50 | |
| *** adarazs is now known as adarazs_brb | 13:50 | |
| *** ccamacho|lunch is now known as ccamacho | 13:53 | |
| *** shardy has joined #tripleo | 13:55 | |
| *** penick has quit IRC | 13:56 | |
| *** athomas has joined #tripleo | 13:59 | |
| EmilienM | derekh, bnemec: good morning/afternoon - have you noticed rh1 issues today? sounds like jobs are timeouting for a few hours or so | 13:59 |
| *** akshai has joined #tripleo | 14:01 | |
| *** trown is now known as trown|brb | 14:03 | |
| d0ugal | slagle: Yup, I agree with rbrady | 14:04 |
| *** kbyrne has quit IRC | 14:04 | |
| *** pradk has joined #tripleo | 14:05 | |
| *** pradk- has joined #tripleo | 14:05 | |
| *** pradk- has quit IRC | 14:05 | |
| *** rodrigods has quit IRC | 14:06 | |
| *** rodrigods has joined #tripleo | 14:06 | |
| *** adarazs_brb is now known as adarazs | 14:06 | |
| slagle | d0ugal: thanks, i moved it to implemented | 14:06 |
| slagle | one more blueprint down! | 14:06 |
| *** kbyrne has joined #tripleo | 14:06 | |
| *** trown|brb is now known as trown | 14:06 | |
| d0ugal | slagle: \o/ | 14:07 |
| *** ayoung has quit IRC | 14:08 | |
| slagle | EmilienM: fwiw, all the jobs are timing out at the same spot: | 14:08 |
| openstackgerrit | Merged openstack/tripleo-quickstart: Collapse devmode definitions on release file https://review.openstack.org/369571 | 14:08 |
| slagle | UpdateDeployment]: CREATE_COMPLETE state changed | 14:08 |
| slagle | so i wonder if it's related to a recent patch we merged | 14:08 |
| *** limao has joined #tripleo | 14:09 | |
| EmilienM | yeah, let's see what we merged | 14:09 |
| *** dsariel has quit IRC | 14:09 | |
| *** limao_ has joined #tripleo | 14:10 | |
| derekh | slagle: I rechecked "https://review.openstack.org/#/c/363630" when its running we should be able to see whats happening | 14:12 |
| EmilienM | slagle: https://review.openstack.org/#/c/368760/ | 14:12 |
| *** limao has quit IRC | 14:13 | |
| honza | big shoutout to shadower and d0ugal for helping me with overcloud installation yesterday, all working now | 14:13 |
| shadower | yay | 14:13 |
| d0ugal | honza: yay | 14:13 |
| * d0ugal didn't think he helped that much | 14:13 | |
| d0ugal | honza: Was this overcloud via the GUI? | 14:14 |
| EmilienM | right something now breaks when [UpdateDeployment]: CREATE_COMPLETE state changed | 14:15 |
| honza | d0ugal: no, CLI --- your major contribution was that magic failures command; i hadn't heard of that :) | 14:15 |
| EmilienM | d0ugal, jaosorior, shardy ^ does it ring a bell ^ ? our CI is currently consistently failing on this error | 14:15 |
| d0ugal | honza: aha, cool - it is very useful. | 14:16 |
| d0ugal | EmilienM: No, that's new to me. | 14:16 |
| slagle | EmilienM: maybe we should test a revert of that | 14:17 |
| *** dtantsur is now known as dtantsur|mtg | 14:17 | |
| slagle | trying a deployment locally as well so i can get a closer look | 14:17 |
| EmilienM | mhh wait before revert i'm still investigating our recent merges | 14:17 |
| slagle | EmilienM: i meant just try a revert, see if it passes CI | 14:18 |
| shardy | [UpdateDeployment]: CREATE_COMPLETE state changed isn't an error, is there anything else interesting in the postci logs? | 14:18 |
| slagle | not merge it immediately | 14:18 |
| slagle | shardy: no, it just stalls there and eventually hits the job timeout | 14:18 |
| slagle | and then we dont get any other logs | 14:18 |
| EmilienM | no logs :( | 14:19 |
| EmilienM | pabelanger: that's a problem for us, when jobs timeout, we don't have any logs | 14:19 |
| shardy | hmm | 14:19 |
| slagle | NetworkDeployment is CREATE_IN_PROGRESS, so it could be that | 14:19 |
| shardy | ah, no error path for os-net-config :( | 14:21 |
| * shardy looks at his patch fixing that | 14:21 | |
| shardy | too late for newton, but I'll rebase that when we branch | 14:21 |
| mandre | EmilienM: how can have you change your mind on https://review.openstack.org/#/c/363167/? | 14:22 |
| EmilienM | mandre: will look in a few | 14:22 |
| derekh | EmilienM: I often wonder if we can add a log collect section in the job deinition here http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/tripleo.yaml | 14:23 |
| derekh | EmilienM: so it doesn't get effected by the build-timeout | 14:23 |
| bnemec | The multinode job is also failing pretty consistently on messaging timeouts: http://logs.openstack.org/52/367552/2/check/gate-tripleo-ci-centos-7-nonha-multinode/afcd864/console.html#_2016-09-14_12_31_21_263776 | 14:25 |
| EmilienM | derekh: right, i'll see how we can move the tripleo-ci log collection into a zuul layout that we can use after a job finish or timeout | 14:26 |
| mandre | EmilienM: \o/ let me know if the lack of ordering with apache is really an issue | 14:28 |
| tobias-fiberdata | slagle, could you give me a hand? How do i check in swift that there's information stored during introspection? | 14:28 |
| *** Guest90031 is now known as cmyster | 14:28 | |
| *** cmyster has joined #tripleo | 14:28 | |
| slagle | tobias-fiberdata: swift -h? | 14:29 |
| tobias-fiberdata | slagle, great | 14:29 |
| *** yamahata has joined #tripleo | 14:29 | |
| shardy | http://docs.openstack.org/developer/tripleo-docs/advanced_deployment/introspection_data.html#introspection-data | 14:30 |
| shardy | tobias-fiberdata: ^^ | 14:30 |
| tobias-fiberdata | thanks man shardy | 14:30 |
| tdasilva | tobias-fiberdata: just in case you are looking for some more detailed swift cli help, this might be helpful: http://docs.openstack.org/developer/python-swiftclient/cli.html#cli-commands | 14:31 |
| *** jaosorior has quit IRC | 14:31 | |
| tobias-fiberdata | i'm just getting stuck doing introspection on my new hardware | 14:32 |
| *** saneax is now known as saneax-_-|AFK | 14:32 | |
| *** limao_ has quit IRC | 14:33 | |
| *** limao has joined #tripleo | 14:33 | |
| *** limao has quit IRC | 14:33 | |
| *** fragatina has joined #tripleo | 14:34 | |
| *** itlinux has joined #tripleo | 14:35 | |
| EmilienM | slagle, shardy, derekh: /me working on zuul config to have logs even if our jobs timeout | 14:35 |
| tobias-fiberdata | tdasilva, thanks | 14:35 |
| slagle | EmilienM: i'm leaning towards it being something with ovb setting up the networking | 14:36 |
| slagle | i can't reproduce locally | 14:36 |
| shardy | tobias-fiberdata: http://docs.openstack.org/developer/tripleo-docs/troubleshooting/troubleshooting.html#hardware-introspection-problems may help? | 14:36 |
| akrivoka | shardy: it seems that the UI will need https://blueprints.launchpad.net/tripleo-common/+spec/assign-nodes-workflow to land in newton - I'm not sure if FFE is needed for this as it is tracked under tripleo-common rather than tripleo? | 14:38 |
| *** fragatina has quit IRC | 14:38 | |
| shardy | akrivoka: I thought we agreed to stop tracking things in the tripleo-common org and delete it | 14:39 |
| shardy | akrivoka: Yes, it should have been a FFE like the other mistral things we added | 14:40 |
| akrivoka | shardy: I think that's right, but this bp is a few months old | 14:40 |
| shardy | akrivoka: can you please copy the BP into the tripleo org then we can track landing it? | 14:40 |
| akrivoka | jrist: ^ | 14:40 |
| shardy | akrivoka: Yeah, just got missed I guess | 14:40 |
| shardy | perhaps we should go ahead and delete the tripleo-common project to make it clear | 14:41 |
| shardy | I'm OK with landing it but we need the patch ready asap | 14:41 |
| akrivoka | agreed | 14:41 |
| d0ugal | akrivoka: is there a patch for it now? | 14:41 |
| d0ugal | I can review/test stuff | 14:41 |
| shardy | there is but it has a -1 | 14:41 |
| d0ugal | ah | 14:41 |
| jrist | thanks akrivoka | 14:41 |
| akrivoka | d0ugal: I'm working on a final update | 14:41 |
| akrivoka | d0ugal: but could use some help if you have a moment | 14:41 |
| openstackgerrit | Merged openstack/tripleo-quickstart: Use relative paths in ci-scripts for quickstart.sh https://review.openstack.org/360649 | 14:42 |
| jrist | akrivoka: do you mind moving it or would you rather I do it? | 14:42 |
| d0ugal | akrivoka: Sure, I can try | 14:42 |
| akrivoka | jrist: I'm working on the patch, can you do it? | 14:42 |
| EmilienM | slagle: moving out logs export from tripleo-ci to zuul looks really complicated, using lot of functions, and variable within tripleo-ci. I'm wondering of an alternative, maybe set timeout when deploying overcloud? | 14:42 |
| *** panda is now known as panda|bbl | 14:43 | |
| akrivoka | d0ugal: so the only thing I need to add is some flavors checking in nova, for that I need to add nova client here https://github.com/openstack/tripleo-common/blob/master/tripleo_common/actions/base.py | 14:44 |
| slagle | EmilienM: we already set OVERCLOUD_DEPLOY_TIMEOUT=$((DEVSTACK_GATE_TIMEOUT-90)) | 14:44 |
| akrivoka | d0ugal: but I'm puzzled as to how to achieve that as it seems that nova client needs username/password auth and does not accept auth_token like the rest of the clients listed there | 14:44 |
| shardy | akrivoka: can't we use the built in nova actions from mistral? | 14:45 |
| slagle | EmilienM: so it gets set to 80 minutes | 14:45 |
| shardy | and add that into the workflow? | 14:45 |
| slagle | EmilienM: interesting enough, it looks like Heat is ignoring that | 14:45 |
| jrist | akrivoka: unfortunately I don't think I have access to retarget | 14:45 |
| jrist | I can create a brand new one in tripleo | 14:45 |
| d0ugal | akrivoka: what shardy said would be best, otherwise I'd have to take a look. I'm not sure I've used novaclient before. | 14:46 |
| shardy | jrist: I can add you to tripleo drivers, but I think you'll need to create a new one | 14:46 |
| EmilienM | slagle: reason is simple, OVERCLOUD_DEPLOY_TIMEOUT is only used by multinode jobs. | 14:46 |
| jrist | shardy: thanks. creating a new one | 14:46 |
| jrist | shardy: https://blueprints.launchpad.net/tripleo/+spec/assign-nodes-workflow | 14:47 |
| akrivoka | shardy: Im not sure, I need to inspect every flavor and look into extra_specs, some custom logic is needed there | 14:47 |
| akrivoka | shardy: can you point me to mistral nova actions? | 14:47 |
| * akrivoka has no experience with mistral whatsoever... | 14:47 | |
| slagle | EmilienM: no. it's used for all jobs | 14:47 |
| d0ugal | akrivoka: mistral action-list | grep nova | 14:48 |
| slagle | it's added to $OVERCLOUD_DEPLOY_ARGS by default | 14:48 |
| slagle | EmilienM: if you look here, we are passing "-t 80": http://logs.openstack.org/12/370012/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/9b5bcae/console.html | 14:48 |
| EmilienM | indeed | 14:49 |
| shardy | akrivoka: https://github.com/openstack/mistral-extra/blob/master/examples/v2/openstack/nova_actions.yaml | 14:49 |
| slagle | but the overcloud create starts at 11:27 and gets killed by the job timeout at 13:16 | 14:49 |
| shardy | that shows how it works | 14:49 |
| shardy | akrivoka: basically the actions are generated by mistral based on the novaclient interfaces | 14:49 |
| shardy | same for all other services | 14:49 |
| akrivoka | thanks, I'll look | 14:49 |
| EmilienM | slagle: looking at tripleoclient | 14:50 |
| d0ugal | akrivoka: as a warning, if you do need to add novaclient that will require a packaging update which creates a bit of a chain of events. So it's easier if you can avoid it :) | 14:51 |
| EmilienM | slagle: I'm wondering if mistral takes care of the timeout | 14:51 |
| EmilienM | slagle: see https://github.com/openstack/python-tripleoclient/blob/133e8cf91a6dbef8c6de9051d16c8322e40dd72a/tripleoclient/v1/overcloud_deploy.py#L238-L289 | 14:51 |
| d0ugal | EmilienM: What timeout? | 14:51 |
| EmilienM | d0ugal: overcloud deployment | 14:52 |
| EmilienM | d0ugal: it seems like overcloud timeout option is not addressed | 14:52 |
| d0ugal | EmilienM: Yeah, I think it might not be passed to the workflow (which does accept it). I'm just checking. | 14:53 |
| slagle | it's the -t option you pass to heat | 14:53 |
| *** dsariel has joined #tripleo | 14:53 | |
| slagle | *heatclient | 14:53 |
| d0ugal | EmilienM, slagle: it needs to be passed here: https://github.com/openstack/python-tripleoclient/blob/133e8cf91a6dbef8c6de9051d16c8322e40dd72a/tripleoclient/v1/overcloud_deploy.py#L294 | 14:53 |
| d0ugal | EmilienM, slagle: patch incoming. | 14:54 |
| EmilienM | cool | 14:54 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Pass the timeout to the deploy workflow https://review.openstack.org/370186 | 14:56 |
| d0ugal | EmilienM, slagle: ^ | 14:56 |
| d0ugal | Totally untested. | 14:56 |
| d0ugal | EmilienM, slagle: it is hard-coded to 240 mins otherwise. https://github.com/openstack/tripleo-common/blob/master/workbooks/deployment.yaml#L103 | 14:57 |
| d0ugal | honza: ^ turns out changing the timeout wouldn't do anything :) | 14:57 |
| * honza dies | 14:57 | |
| EmilienM | d0ugal: well, we'll see if it works, all ovb jobs are currently timeouting | 14:59 |
| tobias-fiberdata | this parameter in undercloud.conf | 14:59 |
| d0ugal | EmilienM: given the CLI default is also 240, I doubt it will actually change anything? | 14:59 |
| *** akshai has quit IRC | 14:59 | |
| tobias-fiberdata | inspection_extras = true | 15:00 |
| tobias-fiberdata | is it possible to change that without rerunning the whole thing? | 15:00 |
| tobias-fiberdata | i think i've asked that question earlier aswell | 15:00 |
| *** jlinkes has quit IRC | 15:00 | |
| *** akshai has joined #tripleo | 15:01 | |
| *** jistr|mtg is now known as jistr | 15:01 | |
| *** rook_ is now known as rook | 15:01 | |
| jrist | rdo meeting if anyoen is interested in #rdo | 15:03 |
| *** jlinkes has joined #tripleo | 15:05 | |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart: Add all the gating functionality to full-deploy.sh https://review.openstack.org/370194 | 15:07 |
| shardy | tobias-fiberdata: dtantsur|mtg can confirm, but I think you may be able to hack /httpboot/inspector.ipxe to remove the extra-hardware option | 15:08 |
| shardy | tobias-fiberdata: safest option is to re-run the openstack undercloud install with a modified undercloud.conf tho | 15:08 |
| dtantsur|mtg | both will work | 15:08 |
| trown | thanks jrist, there is a tripleo topic on the agenda about making exceptions for tripleo gated packages for the upper-constraints based dlrn instance (now the default, causing us to only get released versions of things in upper-constraints) | 15:08 |
| jrist | what is "upper-constraints"? | 15:09 |
| jrist | I am uninformed | 15:09 |
| gfidente | guys, have a link to the proposed summit topics for tripleo? | 15:09 |
| gfidente | the etherpad I mean | 15:09 |
| *** oshvartz has quit IRC | 15:10 | |
| trown | jrist: it is the list of upper bounds for versions of libraries used in openstack gates | 15:10 |
| jrist | ah | 15:10 |
| tobias-fiberdata | shardy, i understand | 15:11 |
| trown | but DIB, python-tripleoclient are on that list which is a bit awkward | 15:11 |
| trown | rdo switched to using that list, because many projects on that list do not prioritize keeping master working because they expect users to only use releases | 15:11 |
| jrist | trown: good to know. are you on the meeting then? | 15:12 |
| jrist | I can back you up if needed :) | 15:12 |
| trown | ya, I put the topic up :) | 15:12 |
| *** snecklifter_ has joined #tripleo | 15:13 | |
| *** eggmaster has joined #tripleo | 15:14 | |
| *** DrBacchus is now known as rbowen | 15:15 | |
| *** dsariel has quit IRC | 15:17 | |
| *** jlinkes has quit IRC | 15:17 | |
| *** jlinkes has joined #tripleo | 15:17 | |
| *** snecklifter has quit IRC | 15:18 | |
| *** snecklifter_ is now known as snecklifter | 15:18 | |
| *** snecklifter_ has joined #tripleo | 15:18 | |
| *** ebarrera has quit IRC | 15:19 | |
| *** pblaho has quit IRC | 15:21 | |
| *** pblaho has joined #tripleo | 15:23 | |
| *** abehl has quit IRC | 15:25 | |
| pabelanger | EmilienM: have you determined the issue with missing logs? | 15:26 |
| trown | jrist: looks actually solved in meeting etherpad before the topic came up :) | 15:27 |
| jrist | NICE | 15:27 |
| trown | https://etherpad.openstack.org/p/RDO-Meeting http://review.rdoproject.org/r/2294 | 15:27 |
| d0ugal | shardy, gfidente - this is a bug for what we were walking about earlier. If you have any ideas/suggestions please add comments. https://bugs.launchpad.net/tripleo/+bug/1623552 | 15:27 |
| openstack | Launchpad bug 1623552 in tripleo "heatclient resolves paths for types and get_file calls that then don't make sense in swift" [Critical,Confirmed] | 15:27 |
| gfidente | d0ugal, tks! | 15:28 |
| d0ugal | shardy, gfidente: at the moment I am not totally sure how to move forward... but that might just be because at this time of the day my head is fried. | 15:28 |
| gfidente | d0ugal, right sounds like we could pursue what shardy said | 15:28 |
| EmilienM | pabelanger: not 100% clear to me now if d0ugal's patch on https://review.openstack.org/370186 will actually help us | 15:32 |
| d0ugal | EmilienM: I don't see that it will. | 15:32 |
| trown | d0ugal: thrash, is there plans to add python-tripleoclient to UC? for some reason I thought it was there, but I may have been confused by the early branching stuff https://github.com/openstack/requirements/blob/master/upper-constraints.txt | 15:32 |
| shardy | d0ugal: is it actually critical? | 15:33 |
| trown | looks like DIB was the only package we were not getting updated in master | 15:33 |
| shardy | d0ugal: or do we just end up with confusing paths which actually work? | 15:33 |
| d0ugal | shardy: it is breaking deployments, so I think so. gfidente's was crashing. | 15:33 |
| shardy | d0ugal: Ok, I'll take a look later - I assume we've got a mismatch somewhere | 15:33 |
| shardy | d0ugal: basically we just need to make the names/keys match | 15:34 |
| tobias-fiberdata | shardy, i can confirm that Huawei Hardware is working great with tripleO | 15:34 |
| d0ugal | shardy: Yeah, I think maybe I took the wrong approach with processing the files to upload - so it might be worth taking a look over that and seeing if it makes sense. | 15:34 |
| sshnaidm | EmilienM, do you know what is broken in CI now? | 15:34 |
| tobias-fiberdata | shardy, introspection works now | 15:34 |
| sshnaidm | EmilienM, I see timeouts in jobs, is it? | 15:34 |
| *** rcernin has quit IRC | 15:34 | |
| d0ugal | trown: I have no plans, but I am not totally sure what the purpose of UC is | 15:35 |
| *** dtantsur|mtg is now known as dtantsur | 15:35 | |
| gfidente | shardy, the issue it is causing is that some of those file:// references can't be followed | 15:35 |
| openstackgerrit | Ryan Brady proposed openstack/tripleo-common: Port password generation from tripleoclient to tripleo-common https://review.openstack.org/368150 | 15:35 |
| trown | d0ugal: I guess it is for libraries for testing in upstream gates, so all the projects use the same upper version for the libraries... | 15:35 |
| trown | d0ugal: all the other clients are there is why I asked, but I don't think we would get any benefit by adding python-tripleoclient there | 15:36 |
| *** aufi has quit IRC | 15:37 | |
| EmilienM | sshnaidm: yes | 15:37 |
| *** chem has quit IRC | 15:38 | |
| shardy | gfidente: ack - I guess we need to instead strip the prefix completely, or use the object URL prefix | 15:38 |
| d0ugal | trown: Yeah, I'm not sure. I don't see any benefit but I am not against it either. | 15:38 |
| gfidente | shardy, yah | 15:39 |
| gfidente | shardy, looks like type: file:// can be followed | 15:39 |
| shardy | d0ugal: ack - tbh I thought there was scope for some rework of that stuff when I wired in the object calls but not had time to properly look at it yet | 15:39 |
| shardy | gfidente: you mean on the heat server? | 15:39 |
| *** ccamacho has quit IRC | 15:40 | |
| shardy | or in heatclient? | 15:40 |
| jrist | honza: are you aware of https://review.openstack.org/#/c/368150 ? | 15:40 |
| d0ugal | shardy: I'm sure, I'm going to take a break because my head is killing me but I'll try and continue with this later. | 15:40 |
| thrash | trown: +1 to what d0ugal said | 15:40 |
| trown | d0ugal: k no worries just updating RDO list for tripleo exceptions to that list, so we get latest packages... looks like only DIB needs to be added | 15:40 |
| jrist | honza: this might effect overcloudrc generation | 15:40 |
| sshnaidm | trown, also ironic afaik | 15:40 |
| shardy | d0ugal: np, I'll try to take a look but it won't be until tomorrow | 15:40 |
| *** pcaruana has quit IRC | 15:40 | |
| trown | sshnaidm: ironic is not gated by tripleo | 15:41 |
| trown | sshnaidm: also only ironic-lib and the clients are on that list | 15:41 |
| trown | https://github.com/openstack/requirements/blob/master/upper-constraints.txt | 15:41 |
| sshnaidm | trown, ooops, my mistake | 15:42 |
| sshnaidm | trown, and tripleo-common? | 15:42 |
| trown | sshnaidm: tripleo-common is already there: https://review.rdoproject.org/r/2294 as are the os-*-config libs | 15:43 |
| *** shardy has quit IRC | 15:43 | |
| sshnaidm | trown, so these one will be built each commit, right? | 15:44 |
| trown | sshnaidm: ya they are excepted from the upper-constraints policy | 15:44 |
| sshnaidm | trown, ++ | 15:44 |
| trown | was only missing DIB | 15:44 |
| *** tremble has quit IRC | 15:45 | |
| slagle | bnemec: are you able to look at the CI failures? | 15:46 |
| slagle | wasnt sure if it was due to the ovb networking setup | 15:46 |
| slagle | since everything seems to be stuck on NetworkDeployment | 15:46 |
| bnemec | slagle: I'm trying to reproduce it in my local ovb environment. | 15:46 |
| slagle | ok | 15:46 |
| b00tcat | o/ hi - I can't seem to build the default overcloud-full image when I issue `openstack overcloud image build --type overcloud-full --node-dist centos7` | 15:47 |
| b00tcat | I'm always getting this https://paste.fedoraproject.org/427894/73847534/ | 15:47 |
| b00tcat | (and I'm building the image on a centos7 box) | 15:47 |
| b00tcat | anybody seen this recently? | 15:47 |
| derekh | slagle: bnemec "ssh jenkins@66.187.229.231" | 15:47 |
| derekh | slagle: bnemec 4th overcloud instance just went ACTIVE | 15:48 |
| slagle | ah, nice | 15:48 |
| bnemec | I did notice yesterday that my network and update deployments were taking over 5 minutes locally. | 15:49 |
| bnemec | But they weren't in ci at the time, so it may not be related. | 15:49 |
| *** absubram has joined #tripleo | 15:51 | |
| *** sshnaidm is now known as sshnaidm|afk | 15:51 | |
| slagle | controller-0 looks like it succeeded, but heat did not process the signal | 15:52 |
| slagle | or it didnt get sent | 15:52 |
| *** abregman has joined #tripleo | 15:53 | |
| slagle | huh | 15:54 |
| slagle | os-apply-config --key instance-id --type raw --key-default "" | 15:54 |
| slagle | is null | 15:54 |
| slagle | so /usr/libexec/os-refresh-config/post-configure.d/99-refresh-completed terminates immediately | 15:54 |
| gfidente | d0ugal, do you have any idea what the template_object | 15:57 |
| gfidente | we pass to get_template_contents contains? | 15:57 |
| *** ayoung has joined #tripleo | 15:57 | |
| slagle | bnemec: EmilienM : https://review.openstack.org/364021 | 15:58 |
| *** abregman has quit IRC | 15:58 | |
| slagle | that has got to be it | 15:58 |
| slagle | reverting now | 15:58 |
| d0ugal | gfidente: not sure, I think rbrady knows that bit well. | 15:58 |
| rbrady | gfidente: the template_object is the full path of the template residing in swift | 16:00 |
| gfidente | rbrady, right | 16:01 |
| gfidente | then it goes to this guy https://github.com/openstack/python-heatclient/blob/master/heatclient/common/utils.py#L351 | 16:01 |
| EmilienM | slagle: mhh yeah? | 16:01 |
| openstackgerrit | James Slagle proposed openstack/os-collect-config: Revert "Treat ec2 collector data as immutable" https://review.openstack.org/370250 | 16:02 |
| openstackgerrit | James Slagle proposed openstack/os-collect-config: Revert "Treat ec2 collector data as immutable" https://review.openstack.org/370250 | 16:02 |
| bnemec | slagle: Weird. I wonder how that passed CI. Did we break CI of packages installed on the overcloud again? | 16:02 |
| slagle | bnemec: must have :( | 16:02 |
| *** ohamada_ has quit IRC | 16:02 | |
| bnemec | Note that the problem is it isn't even looking at the ec2 metadata: [2016/09/14 04:02:02 PM] [ERROR] key instance-id does not exist in [u'/var/lib/os-collect-config/request.json', u'/var/lib/os-collect-config/NetworkDeployment.json'] | 16:03 |
| bnemec | But if you look at /var/lib/os-collect-config/ec2.json, instance-id is there. | 16:03 |
| slagle | bnemec: where are you seeing that error? i dont see that | 16:04 |
| rbrady | gfidente: did you mean to paste that specific line? | 16:04 |
| bnemec | slagle: I dropped the default from your os-apply-config command above. | 16:04 |
| *** ayoung has quit IRC | 16:04 | |
| gfidente | rbrady, that line yes, but it's about https://bugs.launchpad.net/tripleo/+bug/1623552?comments=all | 16:05 |
| openstack | Launchpad bug 1623552 in tripleo "heatclient resolves paths for types and get_file calls that then don't make sense in swift" [Critical,Confirmed] | 16:05 |
| slagle | bnemec: ah, ok | 16:05 |
| slagle | bnemec: that means ec2.json is not in /var/lib/os-collect-config/os_config_files.json | 16:06 |
| slagle | which it isn't | 16:06 |
| *** hjensas has quit IRC | 16:06 | |
| bnemec | slagle: I'm betting it's because now we don't add the path in https://review.openstack.org/#/c/364021/1/os_collect_config/collect.py | 16:06 |
| slagle | yep | 16:07 |
| *** rwsu has quit IRC | 16:09 | |
| *** mcornea has quit IRC | 16:11 | |
| *** jpena is now known as jpena|off | 16:11 | |
| rbrady | gfidente: here's the original patches involved with adding support https://url.corp.redhat.com/00e6d86 | 16:12 |
| *** zoli|trng is now known as zoli|gone | 16:12 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart: Remove external requirements https://review.openstack.org/370264 | 16:13 |
| EmilienM | mandre: commented | 16:14 |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart: Update newton release config for new image location https://review.openstack.org/370267 | 16:14 |
| *** mgarciam has quit IRC | 16:14 | |
| *** zoli|gone is now known as zoli_gone-proxy | 16:15 | |
| derekh | slagle: bnemec sshnaidm|afk we didn't just break packages installed on the overcloud, we broke all package installs except DIB | 16:15 |
| derekh | yesterday in an attempt to work around the outdated DIB we added a second call to "tripleo.sh --delorean-build diskimage-builder" | 16:16 |
| derekh | to make sure we always have the most uptodate DIB | 16:16 |
| honza | jrist: yeah, rbrady and i have already discussed | 16:16 |
| bnemec | derekh: :-( | 16:17 |
| derekh | bnemec: slagle sshnaidm|afk the delorean_build functions resets the DB, so we end up with a repo only containing DIB http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/tripleo.sh#n363 | 16:17 |
| bnemec | Why is this so hard? | 16:17 |
| *** dsariel has joined #tripleo | 16:17 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Use osd_pool_default_* puppet parameters when creating the pools https://review.openstack.org/370270 | 16:17 |
| trown | derekh: so I figured out the right way to make sure we have master of DIB and not latest release of DIB https://review.rdoproject.org/r/#/c/2294/2 | 16:18 |
| bnemec | derekh: So what we probably needed to do was just append diskimage-builder to the list of packages to build in the first place? | 16:18 |
| derekh | bnemec: we could probably do with a rethink of some of this logic, lots of things have organicly changed since it was first put in place | 16:18 |
| derekh | bnemec: yup | 16:19 |
| *** abregman has joined #tripleo | 16:19 | |
| derekh | bnemec: that would work, if DIB is pass in twice, delorean would ignore it the second time (at least it did last time I looked) | 16:19 |
| trown | also I saw DIB was released this morning, so maybe we dont need any of this | 16:20 |
| derekh | trown: bnemec ya, if DIB was released we could probably just remove the workaround | 16:20 |
| EmilienM | bnemec: what to do with https://review.openstack.org/#/c/238192/ ? | 16:22 |
| EmilienM | bnemec: should we abandon it? | 16:22 |
| *** lucasagomes has quit IRC | 16:22 | |
| EmilienM | rhallisey: 1 year without update on https://review.openstack.org/#/c/223182/ - what's up? | 16:23 |
| bnemec | EmilienM: So that's almost all implemented except the repo tool, I believe. | 16:23 |
| trown | EmilienM: bnemec the repo setup part would be really nice to have | 16:23 |
| *** lucasagomes has joined #tripleo | 16:23 | |
| bnemec | Node registration was simplified, flavor creation is done automatically, and node profiles are documented as being assignable in instackenv.json. | 16:23 |
| rhallisey | EmilienM, well | 16:24 |
| openstackgerrit | Merged openstack/tripleo-specs: Propose a config-agent bug tag https://review.openstack.org/363960 | 16:24 |
| EmilienM | bnemec: so i'll defer the spec to Ocata and it will need to be updated | 16:24 |
| bnemec | EmilienM: Well, I'm inclined to push a patch that removes the repo-setup bit of the spec and merge the rest. | 16:24 |
| bnemec | Since the rest of it was implemented. | 16:24 |
| EmilienM | bnemec: good news! | 16:25 |
| trown | seems reasonable | 16:25 |
| EmilienM | yes | 16:25 |
| bnemec | trown: But yes, I still want the repo setup tool. | 16:25 |
| bnemec | EmilienM: I'll do that shortly. | 16:25 |
| EmilienM | bnemec: no pressure at all | 16:25 |
| EmilienM | bnemec: just /me retargeting blueprints | 16:25 |
| trown | bnemec: I think I saw it as a potential summit topic, but even if it does not make the cut we should chat about it in BCN | 16:26 |
| *** trown is now known as trown|lunch | 16:28 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Remove forced delorean dib build https://review.openstack.org/370281 | 16:28 |
| bnemec | trown|lunch: Yeah, I have agendas to push in Barcelona and that's one of them. :-) | 16:29 |
| slagle | EmilienM: bnemec : should we just merge https://review.openstack.org/#/c/370250/ ? | 16:31 |
| *** abregman has quit IRC | 16:31 | |
| slagle | i think it's pretty evident that patch was the problem | 16:32 |
| EmilienM | slagle: approved | 16:32 |
| *** rcernin has joined #tripleo | 16:34 | |
| *** tesseract- has quit IRC | 16:35 | |
| *** dtantsur is now known as dtantsur|afk | 16:36 | |
| *** fzdarsky has quit IRC | 16:36 | |
| *** rasca has quit IRC | 16:40 | |
| *** florianf has quit IRC | 16:41 | |
| *** yamahata has quit IRC | 16:44 | |
| *** sshnaidm|afk is now known as sshnaidm | 16:47 | |
| *** dsariel has quit IRC | 16:47 | |
| sshnaidm | bnemec, derekh, seems like I broke the CI with the workaround? Can we just revert it if DIB is ok. The promotion jobs already promoted jobs and should be without the epel | 16:48 |
| bnemec | sshnaidm: I've already pushed a patch: https://review.openstack.org/370281 | 16:50 |
| derekh | sshnaidm: bnemec has submitted a patch that partially reverts, I'd nearly just approve that now | 16:50 |
| bnemec | I'm going to push a test patch that verifies it still works in a project that rebuilds images, but I'm holding off until the ec2 revert lands since it's guaranteed to fail right now. | 16:50 |
| derekh | bnemec: slagle thoughts on approving that now ? CI is gonna probably fail as the ..... ya the reason bnemec just said | 16:51 |
| sshnaidm | bnemec, derekh cool, sorry for that | 16:51 |
| bnemec | sshnaidm: We all approved it. :-) | 16:51 |
| derekh | sshnaidm: no prob, it happens ;-) and I +2'd it | 16:51 |
| bnemec | Why is https://review.openstack.org/#/c/370250/ not in the gate? | 16:53 |
| slagle | i approved https://review.openstack.org/#/c/370281/ | 16:53 |
| derekh | ok, I gotta run, if anybody wants to just merge bnemec's patch +1 from me, its only reverting that function to what it was yesterday | 16:53 |
| slagle | bnemec: the check job probably hasn't passed yet | 16:53 |
| *** derekh has quit IRC | 16:53 | |
| skramaja | EmilienM: i found a bug in os-net-config related to DPDK and raised https://bugs.launchpad.net/os-net-config/+bug/1619330 and i am working on with dsneddon.. | 16:53 |
| openstack | Launchpad bug 1619330 in os-net-config "os-net-config fails on 2nd run with DPDK net config" [Undecided,In progress] - Assigned to Saravanan KR (skramaja) | 16:53 |
| bnemec | slagle: Oh crap, that's right. It has to pass twice. | 16:53 |
| skramaja | EmilienM: how to prioritize it? | 16:53 |
| skramaja | i can;t change it.. | 16:54 |
| * bnemec kind of misses the days when we could force merge ci patches in 30 seconds. | 16:54 | |
| *** bnemec changes topic to "CI down until https://review.openstack.org/#/c/370250/ merges | TripleO : http://tripleo.org/ | https://wiki.openstack.org/wiki/TripleO | Meetings On Tuesdays at 14:00 UTC in #openstack-meeting-alt" | 16:54 | |
| *** tosky has quit IRC | 16:56 | |
| EmilienM | skramaja: I can do it | 16:56 |
| EmilienM | oh ben did it :) | 16:57 |
| bnemec | Yeah, I can't target it though because we don't have milestones on os-net-config. | 16:57 |
| *** skramaja_ has joined #tripleo | 16:57 | |
| EmilienM | bnemec, slagle: all the recent packaging issues make me thing about our current workflow in testing. and the fact we're not using zuul-cloner but we use RDO. | 16:58 |
| *** karthiks has quit IRC | 16:59 | |
| *** skramaja has quit IRC | 16:59 | |
| slagle | EmilienM: well, we can always discuss that again, but we've been around on it a few times before | 17:00 |
| slagle | testing with source in CI buys us nothing, b/c we have no users that deploy from source | 17:00 |
| slagle | so we test with packaging | 17:00 |
| *** skramaja_ is now known as skramaja | 17:01 | |
| skramaja | /msg NickServ identify saravanan | 17:01 |
| EmilienM | skramaja: I think you can change your password now :) | 17:01 |
| skramaja | yup.. | 17:01 |
| EmilienM | slagle: in puppet CI, our scripts use repos if zuul-cloner is not installed | 17:01 |
| EmilienM | slagle: /me just trying to think how to solve the fact we don't catch everything in our CI | 17:04 |
| *** fragatina has joined #tripleo | 17:04 | |
| skramaja | thanks bnemec for assigning priority | 17:06 |
| *** mgarciam has joined #tripleo | 17:06 | |
| sshnaidm | cores, please take a look at: https://review.openstack.org/#/c/365369/ it'll help to debug issues when booting up vms, and which are not shown usually in regular logs | 17:09 |
| *** fragatina has quit IRC | 17:10 | |
| *** fragatina has joined #tripleo | 17:11 | |
| EmilienM | sshnaidm: I'm wondering why we're hacking like this | 17:11 |
| *** athomas has quit IRC | 17:11 | |
| EmilienM | we should rather fix THT | 17:11 |
| EmilienM | err, instack-undercloud | 17:11 |
| *** abregman has joined #tripleo | 17:12 | |
| EmilienM | and set it in hieradata | 17:12 |
| sshnaidm | EmilienM, so it will be the default, right? | 17:12 |
| EmilienM | i'm sure our users would be interested to have this feature too | 17:12 |
| EmilienM | sshnaidm: yes | 17:12 |
| sshnaidm | EmilienM, exactly, I wonder if we want it by default.. | 17:12 |
| bnemec | sshnaidm: Do we get console logs from our CI jobs? | 17:13 |
| bnemec | I didn't think Ironic could get those through the regular IPMI driver. Especially one talking to an OVB BMC that doesn't expose any way to get the log. | 17:13 |
| sshnaidm | bnemec, which console logs you mean? | 17:13 |
| bnemec | The ones you're trying to collect in https://review.openstack.org/#/c/365369/11/scripts/deploy.sh | 17:14 |
| *** hjensas has joined #tripleo | 17:14 | |
| sshnaidm | bnemec, right now - no | 17:14 |
| sshnaidm | bnemec, these are the logs: http://logs.openstack.org/69/365369/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/45401b9/logs/undercloud/var/log/ironic/deploy/ | 17:14 |
| *** ipsecguy has quit IRC | 17:14 | |
| EmilienM | slagle: revert failed in CI | 17:15 |
| slagle | already looking bro | 17:15 |
| bnemec | sshnaidm: Oh, it's not getting the console log, it getting logs from the IPA ramdisk? | 17:15 |
| bnemec | sshnaidm: That does seem like a good thing to turn on by default, not just in ci. | 17:16 |
| sshnaidm | bnemec, I think so | 17:16 |
| sshnaidm | bnemec, EmilienM so you are for enabling it be default, right? | 17:16 |
| bnemec | sshnaidm: Yeah | 17:16 |
| sshnaidm | ok, will change it then | 17:16 |
| EmilienM | http://logs.openstack.org/50/370250/2/check/gate-tripleo-ci-centos-7-nonha-multinode/9e8c125/logs/var/log/mistral/mistral-server.txt.gz | 17:17 |
| *** zaneb has quit IRC | 17:17 | |
| EmilienM | 2016-09-14 16:58:07.466 32010 ERROR wsme.api [-] Server-side error: "MessagingTimeout: Timed out waiting for a reply to message ID 1faf8b08051842478b09af5eee1e2e9c". Detail: | 17:17 |
| slagle | EmilienM: MessagingTimeout. seen a few of those today. wonder if something else changed? | 17:17 |
| EmilienM | slagle: well, we promoted CI so might be a new mistral | 17:17 |
| bnemec | I think that's a heat message though. | 17:17 |
| bnemec | I've seen a lot of multinode jobs failing with it. | 17:17 |
| EmilienM | bnemec: today or before? | 17:18 |
| *** ipsecguy has joined #tripleo | 17:18 | |
| bnemec | EmilienM: Today. I noticed it when I was looking into the ovb failures. | 17:19 |
| EmilienM | I wonder if https://github.com/openstack/mistral/commit/4e1e358c8b595eba42ead91a118fd14b8b5546e7 is related | 17:19 |
| *** dtrainor has quit IRC | 17:20 | |
| EmilienM | seriously ^ | 17:20 |
| EmilienM | at the end of a cycle | 17:20 |
| bnemec | I hate feature freeze so much. | 17:21 |
| EmilienM | sigh. | 17:21 |
| bnemec | This happens every cycle. | 17:21 |
| EmilienM | well, it seems like Mistral folks care a lot (cough) about FF :) | 17:21 |
| bnemec | (not throwing stones, we're no better) | 17:21 |
| EmilienM | this patch is huge | 17:21 |
| EmilienM | bnemec: well, we're *getting* better | 17:21 |
| EmilienM | I'm proposing to run non-ha multinode job in mistral's gate | 17:22 |
| *** dsariel has joined #tripleo | 17:22 | |
| EmilienM | as non voting | 17:22 |
| EmilienM | so at least we can blame them if something breaks in their side | 17:22 |
| *** dtrainor has joined #tripleo | 17:22 | |
| bnemec | EmilienM: We've still slammed in a bunch of huge features at the end of the cycle, whether right before FF or as FFEs. | 17:22 |
| bnemec | I mean, significant portions of our API don't work right now. | 17:22 |
| EmilienM | I'm not sure if this commit broke us but it looks super related | 17:23 |
| *** trown|lunch is now known as trown | 17:23 | |
| EmilienM | I don't think the problem is in Heat | 17:24 |
| sshnaidm | EmilienM, bnemec is this a good place to set ironic agent config vars? https://github.com/openstack/instack-undercloud/blob/master/elements/puppet-stack-config/puppet-stack-config.yaml.template#L454 | 17:25 |
| *** lucasagomes is now known as lucas-dinner | 17:25 | |
| *** fzdarsky has joined #tripleo | 17:25 | |
| EmilienM | sshnaidm: if you want it by default, yes. | 17:25 |
| EmilienM | rbrady, d0ugal: we might need your help on this one too | 17:26 |
| bnemec | sshnaidm: Oh, that must be a generic oslo.messaging exception. I've only ever seen it from heat though. :-) | 17:26 |
| slagle | EmilienM: filed a bug: https://bugs.launchpad.net/tripleo/+bug/1623606 | 17:26 |
| openstack | Launchpad bug 1623606 in tripleo "CI: jobs failing with mistral MessagingTimeout" [Undecided,New] | 17:26 |
| EmilienM | slagle: nice | 17:27 |
| EmilienM | I'm jumping on mistral trying to get some help | 17:27 |
| slagle | d0ugal: rbrady : can you look at ^^ ? | 17:27 |
| sshnaidm | bnemec, sorry? | 17:27 |
| bnemec | sshnaidm: Oh, sorry. I meant to direct that at EmilienM. | 17:27 |
| bnemec | ETOOMANYIRCCONVERSATIONSATONCE | 17:27 |
| sshnaidm | ok :) | 17:28 |
| EmilienM | ahah | 17:28 |
| *** mgarciam has quit IRC | 17:29 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/instack-undercloud: Save console logs from all vms to files https://review.openstack.org/370329 | 17:29 |
| bnemec | ERROR: Failed to stop stack (overcloud) on other engine (b6dde44f-1e60-474f-8044-f4f02caed8fb) | 17:29 |
| bnemec | Well, that's a new one. | 17:29 |
| *** zaneb has joined #tripleo | 17:29 | |
| mwhahaha | not sure if related but i just ran a quickstart deploy and I'm getting the following when i try to load instackenv.json | 17:32 |
| mwhahaha | Exception registering nodes: No valid host was found. Reason: No conductor service registered which supports driver pxe_ssh. | 17:32 |
| beagles | trozet: was there a particular use case that inspired removing the setting of L3 HA in the client or was it simply a cleanup? | 17:32 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Set VNC common data for compute nodes also https://review.openstack.org/370331 | 17:33 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Set VNC common data for compute nodes also https://review.openstack.org/370331 | 17:33 |
| openstackgerrit | Lucas Alvares Gomes proposed openstack/tripleo-quickstart: Force the use of python/pip version 2 https://review.openstack.org/368013 | 17:35 |
| *** bana_k has joined #tripleo | 17:36 | |
| *** ipsecguy has quit IRC | 17:36 | |
| bnemec | beagles: As I recall, it was done to get business logic out of the client and into the templates. | 17:36 |
| bnemec | beagles: It should be getting set in the pacemaker environment now, since that's required for ha. | 17:37 |
| beagles | bnemec: is it? | 17:37 |
| bnemec | beagles: I think it was. I haven't looked that it still is. :-) | 17:37 |
| *** trown is now known as trown|brb | 17:37 | |
| beagles | bnemec: also if the pacemaker gets pulled in when pcs is being used in a single controller it'll break | 17:37 |
| *** pkovar has quit IRC | 17:38 | |
| beagles | (enabling l3_ha with only one l3 agent doesn't work, as it happens) | 17:38 |
| bnemec | beagles: This is true, although I can't imagine why you would want pacemaker in that case. :-) | 17:38 |
| bnemec | You could always override the param with a later env. | 17:38 |
| *** ipsecguy has joined #tripleo | 17:38 | |
| *** yamahata has joined #tripleo | 17:38 | |
| beagles | yeah... I wonder what kind of people would do such a thing | 17:38 |
| beagles | :) | 17:39 |
| *** itlinux has quit IRC | 17:39 | |
| EmilienM | mwhahaha: ouch, I already saw that a long time ago | 17:40 |
| EmilienM | mwhahaha: you running trunk? | 17:40 |
| mwhahaha | i think it should have been trunk, who knows at this point :D | 17:40 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-specs: Simplify the TripleO Workflow https://review.openstack.org/238192 | 17:41 |
| bnemec | EmilienM: Okay, I think ^ can just be merged as all the work itemse are complete. | 17:41 |
| bnemec | I also moved it to newton since not everything was actually complete in mitaka. | 17:41 |
| bnemec | I'll start a new spec for the repo tool. | 17:41 |
| *** dsariel has quit IRC | 17:42 | |
| bnemec | beagles: Actually, now that I think about it, we were. The updates job ran pacemaker in a single-node setup so we covered updating pacemaker resources. | 17:42 |
| mwhahaha | oo i was about to start a ML topic around such things -^ | 17:42 |
| EmilienM | bnemec: nice | 17:42 |
| * bnemec has a lot of specs to write before Barcelona | 17:43 | |
| *** ipsecguy has quit IRC | 17:43 | |
| *** mbound has quit IRC | 17:44 | |
| openstackgerrit | Saravanan KR proposed openstack/os-net-config: Add mac address to the DPDK mapping file https://review.openstack.org/370012 | 17:44 |
| *** skramaja has quit IRC | 17:47 | |
| *** ipsecguy has joined #tripleo | 17:48 | |
| *** trown|brb is now known as trown | 17:50 | |
| *** milan has quit IRC | 17:51 | |
| *** electrofelix has quit IRC | 17:51 | |
| trown | mwhahaha: could you try restarting openstack-ironic-conductor? we may have had a regression when splitting out all of the post-undercloud-install functionality from quickstart | 18:01 |
| EmilienM | slagle: in the meantime, I suggest we build mistral on something older | 18:02 |
| mwhahaha | trown: it worked after rerunning the openstack undercloud install | 18:02 |
| mwhahaha | trown: which i think restarted a bunch of services | 18:03 |
| trown | mwhahaha: hmm, ya that would have restarted the conductor... | 18:03 |
| rbrady | EmilienM, slagle: taking a look | 18:04 |
| EmilienM | rbrady: we're on mistral channel | 18:06 |
| tbarron | oslo has flavio percocco, julien danjou, victor stinner, ben nemec | 18:07 |
| tbarron | sorry, wrong channel | 18:07 |
| *** jprovazn has quit IRC | 18:08 | |
| EmilienM | rbrady, d0ugal: can we consider https://blueprints.launchpad.net/tripleo/+spec/cli-deployment-via-workflow finished? | 18:20 |
| d0ugal | EmilienM: Yeah, I think so | 18:20 |
| d0ugal | I mean, it's broken but it's merged :P | 18:21 |
| EmilienM | lol | 18:21 |
| EmilienM | thanks ! | 18:21 |
| slagle | EmilienM: any possible resolution on the mistral issue? | 18:23 |
| EmilienM | I'm still having doubts about https://github.com/openstack/mistral/commit/4e1e358c8b595eba42ead91a118fd14b8b5546e7 | 18:24 |
| * EmilienM quick lunch | 18:24 | |
| *** nyechiel has quit IRC | 18:25 | |
| EmilienM | slagle: could we run a ci job with a 1 day old repo? | 18:25 |
| EmilienM | or fake a revert for this patch in mistral? | 18:25 |
| EmilienM | even if I'm not sure that's the broken commit | 18:26 |
| slagle | we could | 18:26 |
| d0ugal | EmilienM: I don't think it is | 18:27 |
| d0ugal | Given that I have it :) | 18:27 |
| EmilienM | after my lunch I'm going to compare packages between latest successful job and now | 18:27 |
| EmilienM | and analize all diffs | 18:27 |
| EmilienM | we promote every day, diff shouldn't be huge | 18:27 |
| *** bfournie has quit IRC | 18:28 | |
| EmilienM | the error itself seems hard to debug, d0ugal have you look at the rpc trace? | 18:28 |
| EmilienM | like a timeout | 18:28 |
| EmilienM | would we need to increase some timeout in mistral.conf? | 18:28 |
| d0ugal | EmilienM: Good question. I'll see what options there are. | 18:29 |
| slagle | d0ugal: can i assign the bug to you? | 18:29 |
| d0ugal | slagle: Sure, but I wont be here that much longer today | 18:30 |
| slagle | yea, true. i'll assign it to you for now | 18:31 |
| slagle | rbrady: maybe you can take it over when d0ugal leaves | 18:31 |
| rbrady | slagle: sure. | 18:31 |
| slagle | the os-collect-config revert is in the gate now at least | 18:32 |
| *** saneax-_-|AFK is now known as saneax | 18:33 | |
| *** jlinkes has quit IRC | 18:35 | |
| d0ugal | rbrady: I have no good ideas. The error doesn't mean anything to me | 18:37 |
| d0ugal | EmilienM: This config could be promising. https://github.com/openstack/mistral/blob/master/mistral/config.py#L91-L97 | 18:38 |
| d0ugal | I have no idea where the config is managed. | 18:38 |
| d0ugal | but 60 seconds seems like a long time anyway, I'd be amazed if increasing it would help | 18:40 |
| *** dciabrin has quit IRC | 18:41 | |
| d0ugal | https://github.com/openstack/oslo.messaging/blob/master/oslo_messaging/_drivers/amqpdriver.py#L235-L238 | 18:42 |
| rbrady | d0ugal: were you able to discern whether this is happening between the engine and executor or engine and api? | 18:42 |
| d0ugal | rbrady: no, not really. | 18:42 |
| d0ugal | That link is where the error comes from in oslo.messaging. So I'd guess the message is never sent for some reason. | 18:42 |
| *** jlinkes has joined #tripleo | 18:45 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Testing Testing 1 2 3 https://review.openstack.org/370375 | 18:50 |
| *** assassin has quit IRC | 18:51 | |
| *** social has joined #tripleo | 18:51 | |
| *** dciabrin has joined #tripleo | 18:55 | |
| EmilienM | slagle: sounds like the mistral issue is transient, the revert is in gate now https://review.openstack.org/#/c/370250/ | 18:55 |
| *** sshnaidm is now known as sshnaidm|afk | 18:55 | |
| slagle | EmilienM: right | 18:56 |
| d0ugal | Fun | 18:56 |
| *** akrivoka has quit IRC | 18:57 | |
| *** liverpooler has quit IRC | 18:58 | |
| *** ayoung has joined #tripleo | 19:01 | |
| *** fragatina has quit IRC | 19:02 | |
| openstackgerrit | Ryan Brady proposed openstack/python-tripleoclient: Remove identifier and stack action params https://review.openstack.org/370383 | 19:03 |
| *** absubram has quit IRC | 19:03 | |
| openstackgerrit | Ryan Brady proposed openstack/python-tripleoclient: Remove identifier and stack action params https://review.openstack.org/370383 | 19:04 |
| *** panda|bbl is now known as panda | 19:05 | |
| EmilienM | it sounds like a race condition, http://tripleo.org/cistatus.html | 19:07 |
| EmilienM | the % of failure is high | 19:07 |
| *** pcaruana has joined #tripleo | 19:07 | |
| EmilienM | I'm curious to see if we have the same problem on ovb job, but for that need to wait for slagle's revert patch to land | 19:08 |
| slagle | EmilienM: actually you don't | 19:09 |
| slagle | b/c the MessagignTimeout would occur before the NetworkDeployment timeout that the os-collect-config revert fixes | 19:09 |
| slagle | so you could review logs and see | 19:09 |
| slagle | actually just look for failed jobs, instead of timed out jobs | 19:10 |
| *** rwsu has joined #tripleo | 19:11 | |
| *** fzdarsky has quit IRC | 19:14 | |
| *** r-mibu has quit IRC | 19:16 | |
| *** r-mibu has joined #tripleo | 19:16 | |
| EmilienM | slagle: yeah so it's only multinode jobs, right? | 19:17 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Do Not Merge - test an older delorean repo https://review.openstack.org/370393 | 19:20 |
| slagle | EmilienM: i guess it might be | 19:22 |
| EmilienM | rbrady: do you have any idea about the mistral problem? | 19:28 |
| slagle | EmilienM: i pushed a patch trying our older delorean repo pin ^ | 19:28 |
| *** akshai has quit IRC | 19:28 | |
| slagle | just for testing | 19:28 |
| slagle | we can recheck a few times | 19:28 |
| slagle | i have the package diff between the 2 repos | 19:28 |
| *** jlinkes has quit IRC | 19:29 | |
| *** jlinkes_ has joined #tripleo | 19:29 | |
| pabelanger | bnemec: have time tomorrow to sync about the new projects for tripleo-test-cloud-rh2? | 19:30 |
| rbrady | EmilienM: not yet. I didn't realize d0ugal handed it off for the day...lol. I'll try to see if I can debug the engine and api services and trace the messages | 19:30 |
| bnemec | pabelanger: Probably. The projects are there, I just need some way to get you the credentials. | 19:30 |
| *** fzdarsky has joined #tripleo | 19:31 | |
| bnemec | EmilienM: slagle: I don't see any mistral messaging timeouts in ovb jobs over the past 24 hours: http://logstash.openstack.org/#dashboard/file/logstash.json?query=build_name%3A%20*ovb*%20AND%20build_status%3A%20FAILURE%20AND%20message%3A%20%5C%22MessagingTimeout%5C%22 | 19:32 |
| pabelanger | bnemec: sure, derekh added them to a host last time, I'll fine the IP address and send it over in the morning | 19:32 |
| EmilienM | bnemec: right | 19:32 |
| EmilienM | I think the mistral timeout happens in our multinode jobs in a random time | 19:32 |
| d0ugal | rbrady: I am still looking a bit, but that is no reason for you not to look also ;) | 19:32 |
| *** nyechiel has joined #tripleo | 19:35 | |
| *** bfournie has joined #tripleo | 19:35 | |
| rbrady | d0ugal: was working on something else. when everything is priority, nothing is priority | 19:36 |
| d0ugal | rbrady: ah okay, I thought this was the only one now | 19:38 |
| d0ugal | anyway | 19:38 |
| *** rwsu has quit IRC | 19:41 | |
| openstackgerrit | Ben Nemec proposed openstack/python-tripleoclient: Remove one last reference to openstackclient.tests https://review.openstack.org/370401 | 19:42 |
| bnemec | Note: we can't build the tripleoclient package until we get ^ merged | 19:43 |
| bnemec | Which means that overcloud image builds are still screwed. | 19:43 |
| *** rwsu has joined #tripleo | 19:43 | |
| *** akshai has joined #tripleo | 19:45 | |
| *** akshai_ has joined #tripleo | 19:46 | |
| *** nyechiel has quit IRC | 19:46 | |
| pabelanger | We should have been able to catch that in rdo 3rd party CI for DLRN, but I think our zuul configuration is not correct. | 19:47 |
| pabelanger | will work on that tomorrow with RDO team | 19:47 |
| mwhahaha | trown: it looks like there's an issue with the current-tripleo stable undercloud image. i deployed an environment using that and then did an undercloud install and got the same error. a service restart of openstack-ironic-conductor fixed it | 19:48 |
| trown | mwhahaha: k, that image was a bit old, but was just updated about an hour ago | 19:48 |
| mwhahaha | i just pulled it fresh | 19:49 |
| mwhahaha | i think | 19:49 |
| mwhahaha | about an hour ago :D | 19:49 |
| trown | mwhahaha: could you check if the image you have matches http://artifacts.ci.centos.org/rdo/images/master/delorean/current-tripleo/stable/undercloud.qcow2.md5 | 19:49 |
| mwhahaha | trown: nope, let me try that | 19:49 |
| trown | mwhahaha: could look in /var/cache/tripleo-quickstart/images | 19:49 |
| *** akshai has quit IRC | 19:49 | |
| trown | mwhahaha: there will be a symlink in there to an $md5sum.qcow2 | 19:50 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Test Mistral "event trigger REST API" revert https://review.openstack.org/370404 | 19:50 |
| EmilienM | ok let's see if revert would actually help | 19:50 |
| *** saneax is now known as saneax-_-|AFK | 19:51 | |
| d0ugal | bnemec: Thanks for removing that import, I think it must have been added in an old review | 19:53 |
| bnemec | d0ugal: Yeah, it happens. | 19:53 |
| bnemec | It's hard to keep those big patches in sync as things change out from under them. | 19:53 |
| d0ugal | Indeed :) | 19:53 |
| *** jlinkes_ has quit IRC | 19:57 | |
| *** jlinkes has joined #tripleo | 19:57 | |
| *** can8dnSix has joined #tripleo | 19:57 | |
| *** pcaruana has quit IRC | 20:00 | |
| can8dnSix | hello. Question, can I use tripleO to deploy segrated services, i.e. dedicated HA Neutron, dedicated HA MariaDB, dedicated controllers, etcs? I need to know if I'm wasting my time trying to get this configured using OSP8 Directory. thanks | 20:02 |
| slagle | can8dnSix: that should be possible in the tripleo newton release | 20:04 |
| *** nyechiel has joined #tripleo | 20:06 | |
| mwhahaha | trown: actually i just pulled it down and it still does it | 20:07 |
| *** can8dnSix has quit IRC | 20:08 | |
| *** abregman has quit IRC | 20:09 | |
| mwhahaha | trown: i'll investigate further and an report a bug later if i figure it out | 20:09 |
| trown | mwhahaha: awesome thanks... if you get fed up, and just need to get some work done, there is now a way to use the images from tripleo-ci to get a dev environment up | 20:09 |
| trown | mwhahaha: I have been testing it heavily this week and it works quite well | 20:10 |
| trown | mwhahaha: just pass in "--release master-tripleo-ci" and it will use the overcloud-full.qcow2 produced in tripleo-ci for an undercloud image | 20:10 |
| mwhahaha | trown: yea it's no biggie for me since i can just restart the ironic-conductor service and it works fine | 20:10 |
| trown | mwhahaha: but I will shift focus on what is wrong with the image or quickstart early tomorrow | 20:11 |
| mwhahaha | it's not quickstart cause i'm not even using that | 20:11 |
| mwhahaha | it's the image itself | 20:11 |
| trown | interesting | 20:12 |
| *** nyechiel has quit IRC | 20:12 | |
| mwhahaha | might be a bug in ironic for all I know :D the pxe_ssh says it loaded in the logs so who knows | 20:13 |
| EmilienM | I remember that | 20:14 |
| EmilienM | but I don't remember what was the output | 20:14 |
| *** can8dnSix has joined #tripleo | 20:14 | |
| EmilienM | you must hate me at this time :D | 20:14 |
| can8dnSix | Thank you very much. | 20:14 |
| can8dnSix | @ slagle. | 20:14 |
| mwhahaha | EmilienM: there is https://bugzilla.redhat.com/show_bug.cgi?id=1313907 | 20:15 |
| openstack | bugzilla.redhat.com bug 1313907 in openstack-ironic "No valid host was found. Reason: No conductor service registered which supports driver pxe_ssh. (HTTP 400)" [Medium,Closed: currentrelease] - Assigned to ifarkas | 20:15 |
| *** fragatina has joined #tripleo | 20:17 | |
| *** mbound has joined #tripleo | 20:19 | |
| mwhahaha | does undercloud install write a puppet log somewhere? | 20:19 |
| mwhahaha | EmilienM, trown: enabled_drivers for the ironic_config may be getting set after the ironic-conductor service is started | 20:20 |
| EmilienM | I don't know how it's possible | 20:21 |
| mwhahaha | *magic* | 20:21 |
| mwhahaha | which is why i asked if there's a puppet log somewhere :D searching in terminal window makes it to track | 20:22 |
| EmilienM | mwhahaha: in /var/log/messages afik | 20:22 |
| mwhahaha | nope | 20:22 |
| * mwhahaha makes notes for future improvements | 20:23 | |
| *** mbound has quit IRC | 20:24 | |
| trown | mwhahaha: puppet log is part of `openstack undercloud install` output | 20:26 |
| mwhahaha | yea it should also get written out on the node somewhere | 20:26 |
| EmilienM | ah, on undercloud | 20:26 |
| trown | quickstart logs that output to /home/stack/undercloud_install.log | 20:26 |
| trown | I think it is written somewhere else too... | 20:26 |
| EmilienM | trown: imho that's the wrong place | 20:26 |
| mwhahaha | in generally the puppet logs should get written out somewhere locally, i also find it very hard to follow on the overcloud | 20:27 |
| bnemec | mwhahaha: ~/.instack/install-undercloud.log | 20:27 |
| EmilienM | instead of fixing it in quickstart, we should rather do it in instack-undercloud | 20:27 |
| EmilienM | trown: why do you rewrite the logs? | 20:27 |
| trown | EmilienM: I am not talking about fixing anything in quickstart, just where logs are | 20:27 |
| mwhahaha | cause you know i'm going to know to look in ~/.instack for my puppet logs ಠ_ಠ | 20:27 |
| trown | EmilienM: for a tool that aims to be intuitive it is nice to have all logs in the stack users home dir | 20:28 |
| panda | sorry, I lost the reason why gate ovb is timing out | 20:28 |
| EmilienM | panda: we highly think this revert will help https://review.openstack.org/#/c/370250/ | 20:29 |
| EmilienM | trown: my point is that if something is not intuitive, let's fix it for everyone, not just for quickstart users, from a philosophy perspective. | 20:29 |
| mwhahaha | for quickstart i'd assume the logs would be transfer back to the original host | 20:30 |
| trown | EmilienM: thats fine, doesnt mean we need to unfix it for quickstart :P | 20:30 |
| mwhahaha | for undercloud, it should be either in the home dir or someplace you know, not hidden :D | 20:30 |
| panda | EmilienM: thanks. | 20:31 |
| EmilienM | trown: when we find a problem, it's a matter of where we fix it. And sometimes I see quickstart fixing stuffs we could fix elsewhere. | 20:31 |
| trown | EmilienM: yep, I have heard that feedback plenty | 20:31 |
| EmilienM | not blaming anyone, just trying to find solutions | 20:32 |
| trown | the same could be said of tripleo.sh | 20:32 |
| EmilienM | yes exactly | 20:32 |
| EmilienM | what can be fixed in projects directly, must be fixed in projects | 20:32 |
| EmilienM | trown: I've -1 a lot of patches in tripleo-ci because we could fix it elsewhere. Ask sshnaidm|afk with his ironic hiera config | 20:33 |
| mwhahaha | anyway, now that i've hit a nerve :D | 20:34 |
| trown | :) | 20:34 |
| * mwhahaha ninja vanish | 20:34 | |
| trown | mwhahaha: no worries this is a long running topic | 20:34 |
| mwhahaha | it's ok, i have my own list o' things along this similar topic. so long as we're all working to make it better :) | 20:35 |
| mwhahaha | yea my thought about the default driver not configured until after the service started was wrong. eh i'll figure it out tomorrow | 20:36 |
| *** rcernin has quit IRC | 20:36 | |
| *** gfidente has quit IRC | 20:36 | |
| *** rcernin has joined #tripleo | 20:40 | |
| *** rcernin has quit IRC | 20:41 | |
| *** rcernin has joined #tripleo | 20:41 | |
| *** rcernin has quit IRC | 20:42 | |
| *** rcernin has joined #tripleo | 20:42 | |
| EmilienM | ok the revert didn't help and I still see http://paste.openstack.org/show/6h6eC0bNaQyr39YVv3yj/ | 20:45 |
| *** trown is now known as trown|outtypewww | 20:48 | |
| *** absubram has joined #tripleo | 20:49 | |
| *** akshai_ has quit IRC | 20:51 | |
| *** coolsvap has quit IRC | 20:52 | |
| EmilienM | slagle: your patch to come back to old rdo doesnt work | 20:52 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: TEST: DO NOT MERGE https://review.openstack.org/370426 | 20:52 |
| EmilienM | same error | 20:52 |
| *** jayg is now known as jayg|g0n3 | 20:54 | |
| *** dciabrin has quit IRC | 20:56 | |
| EmilienM | here's the diff between 2 jobs where one failed and another worked. They ran almost in same time | 20:56 |
| EmilienM | https://www.diffchecker.com/EyICHA3r | 20:56 |
| *** fzdarsky has quit IRC | 20:57 | |
| *** can8dnSix has quit IRC | 20:59 | |
| EmilienM | the only diff is https://github.com/openstack/tripleo-common/commit/af27127508eabf2b6873713e5e1507fa92b5f5b3 | 20:59 |
| EmilienM | everything else has same commits | 20:59 |
| *** ipsecguy has quit IRC | 20:59 | |
| *** ipsecguy has joined #tripleo | 21:01 | |
| *** lblanchard has quit IRC | 21:04 | |
| *** ipsecguy has quit IRC | 21:06 | |
| *** thrash is now known as thrash|bbl | 21:06 | |
| EmilienM | and the only patch is os-* was https://review.openstack.org/#/c/364021/ but slagle already initiated the revert and we still hit the issue | 21:06 |
| EmilienM | so by dichotomy I would say https://github.com/openstack/tripleo-common/commit/af27127508eabf2b6873713e5e1507fa92b5f5b3 is breaking us | 21:06 |
| *** ubijtsa has joined #tripleo | 21:07 | |
| d0ugal | EmilienM: That is one of the rushed in changes, so probably not all that surprising if it is the issue | 21:07 |
| *** ipsecguy has joined #tripleo | 21:08 | |
| *** ubijtsa is now known as Guest25001 | 21:08 | |
| bnemec | Pushing a revert for CI to chew on is cheap... | 21:08 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-common: Revert "Add template processing to the update plan workflow." https://review.openstack.org/370434 | 21:08 |
| EmilienM | d0ugal: i'm really unsure, but again I'm exploring everything I can | 21:09 |
| EmilienM | tbh that was my last card :( | 21:09 |
| slagle | EmilienM: k, just saw my old rdo repo failed | 21:10 |
| *** ooolpbot has joined #tripleo | 21:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1623606 | 21:10 |
| *** ooolpbot has quit IRC | 21:10 | |
| openstack | Launchpad bug 1623606 in tripleo "CI: jobs failing with mistral MessagingTimeout" [Critical,Confirmed] - Assigned to Dougal Matthews (d0ugal) | 21:10 |
| EmilienM | slagle: yeah, I really think it's one of our projects now | 21:10 |
| *** Goneri has quit IRC | 21:11 | |
| slagle | EmilienM: certainly possible | 21:12 |
| slagle | strange thing is how it does not reproduce in ovb or locally | 21:12 |
| *** ipsecguy has quit IRC | 21:12 | |
| EmilienM | right | 21:14 |
| *** ipsecguy has joined #tripleo | 21:15 | |
| *** paramite has quit IRC | 21:16 | |
| ayoung | EmilienM, so...I want to test the Fernet and Credentials work you did. Any suggestions how? | 21:16 |
| EmilienM | I found useful to share this URL | 21:17 |
| EmilienM | https://review.openstack.org/#/q/project:%22%255Eopenstack/os-.*-config%2524%22+OR+project:%22%255Eopenstack/tripleo-.*%2524%22+status:merged | 21:17 |
| slagle | EmilienM: i think you might have nailed it | 21:17 |
| EmilienM | it gives you what we merged in tripleo recently | 21:17 |
| slagle | EmilienM: b/c on https://review.openstack.org/#/c/368760/, I see it failed with this MessagingTimeout on one of the earlier rechceks | 21:18 |
| slagle | on 9/12 | 21:18 |
| EmilienM | ah | 21:19 |
| EmilienM | so there is a race | 21:19 |
| EmilienM | and we landed this patch because we thought it was good | 21:19 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: WIP: DONT MERGE: Test oooq with osinfra https://review.openstack.org/370437 | 21:19 |
| EmilienM | nice shot James! | 21:19 |
| EmilienM | ayoung: I only did crendentials | 21:20 |
| EmilienM | I need to do fernet | 21:20 |
| EmilienM | but I haven't spent time on it yet | 21:20 |
| ayoung | EmilienM, its ok. I can test just the credentials piece | 21:20 |
| EmilienM | is the algo the same to generate the keys? | 21:20 |
| ayoung | yep | 21:20 |
| EmilienM | so if you deploy undercloud and overcloud, your keystone will have credentials setup by puppet | 21:20 |
| EmilienM | and if you want to override the keys on overcloud (you can't on undercloud), you have a KeystoneCredential1 parameter | 21:21 |
| EmilienM | a sec | 21:21 |
| ayoung | EmilienM, OK, so just doing an overcloud install with the latest should work | 21:21 |
| EmilienM | yes sir | 21:21 |
| ayoung | getting images is taking forever | 21:21 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-heat-templates: Bracket swift storage_local_net_ip https://review.openstack.org/370438 | 21:22 |
| EmilienM | and if you're brave, and you want to override the keys that tripleoclient would generate for you, you can use these 2 params: https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/keystone.yaml#L90-L95 | 21:22 |
| EmilienM | but don't do that | 21:22 |
| EmilienM | tripleoclient is smart and generate secure keys for you | 21:22 |
| EmilienM | https://review.openstack.org/#/q/project:%22%255Eopenstack-infra/tripleo-ci%22+AND+status:merged+OR+project:%22%255Eopenstack/os-.*-config%2524%22+AND+status:merged+OR+project:%22%255Eopenstack/tripleo-.*%2524%22+AND+status:merged+NOT+project:openstack/tripleo-quickstart | 21:24 |
| EmilienM | even better | 21:24 |
| pabelanger | sshnaidm|afk: have you see the experimental job in tripleo-quickstart? It relates to ^. I haven't had the time to work on it, but we have an existing ansible functional test that runs ansible playbooks in the gate. It would be great to start with that experimental job to see if quickstart can run in the gate. | 21:26 |
| *** jlinkes has quit IRC | 21:27 | |
| sshnaidm|afk | pabelanger, hi, which job is it? | 21:27 |
| sshnaidm|afk | pabelanger, can you give a link? | 21:28 |
| ayoung | EmilienM, fernet would look just like this, right: https://review.openstack.org/#/c/366400/4/puppet/services/keystone.yaml | 21:28 |
| ayoung | does anything else need to be done there? | 21:28 |
| ayoung | You did that puppet work a while ago IIRC | 21:28 |
| EmilienM | ayoung: could we use the same keys for credentials AND fernet? | 21:28 |
| EmilienM | would it be a problem? | 21:28 |
| ayoung | EmilienM, I would not recommend it | 21:28 |
| EmilienM | ok | 21:28 |
| pabelanger | sshnaidm|afk: so the idea is it would use the ansible-role-jobs template in JJB: https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/ansible-role-jobs.yaml#L104 | 21:29 |
| EmilienM | so you need to dupplicate the parameters and my patch in tripleoclient | 21:29 |
| EmilienM | and my patch in puppet keystone | 21:29 |
| EmilienM | a sec: | 21:29 |
| ayoung | they go in different locations on the Keystone side to be able to vary them independantly for a reason | 21:29 |
| EmilienM | https://github.com/openstack/python-tripleoclient/commit/5f0694a64efcf6527306e8507efa92d03d60a05a | 21:29 |
| pabelanger | sshnaidm|afk: which is the https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/ansible-role-jobs.yaml#L66 template | 21:29 |
| EmilienM | https://github.com/openstack/puppet-keystone/commit/0571e95b36c3c5f268dd678a04451d09467db484 | 21:29 |
| EmilienM | ayoung: you need to do similar things for fernet | 21:29 |
| ayoung | EmilienM, right | 21:29 |
| pabelanger | sshnaidm|afk: basically, the entry point would be tox, to setup ansible dependencies, they use the local ssh connection to actually do the deploy. | 21:30 |
| EmilienM | and then boom we have fernets in tripleo. Next iteration will be rotation support | 21:30 |
| ayoung | EmilienM, mfisher got Fernet support into Puppet a while back, so we would only need the Heat template change, right? | 21:30 |
| EmilienM | ayoung: mfish is using ansible | 21:30 |
| pabelanger | sshnaidm|afk: I haven't had the time to add the functionality to tripleo-quickstart, but that is the template of we use between some ansible roles I maintain and ansible openstack team | 21:30 |
| EmilienM | and mfish doesn't use puppet to generate the keys | 21:30 |
| ayoung | Maybe, but he submitted a change: | 21:30 |
| ayoung | commit bca04a399e6de63bd6d3d3d985aeaf3134802ac1 | 21:30 |
| ayoung | Author: Matt Fischer <matt@mattfischer.com> | 21:30 |
| ayoung | Date: Mon May 25 12:47:09 2015 -0600 | 21:30 |
| ayoung | Add support for Fernet Tokens | 21:31 |
| EmilienM | yes, he added the configuration management | 21:31 |
| EmilienM | anyway, i'm telling you, current puppet-keystone doesn't manage keystone fernets in multinode | 21:31 |
| EmilienM | you need my work | 21:31 |
| *** jlinkes has joined #tripleo | 21:31 | |
| sshnaidm|afk | pabelanger, wow, I'd like to know more about this, seems very related..\ | 21:31 |
| EmilienM | probably mfish use ansible to export the keys on the other keystone nodes | 21:31 |
| pabelanger | sshnaidm|afk: yes, I've had it in the back on my pipeline for a while now. I just need more bandwidth | 21:32 |
| morazi | pabelanger, nifty on the experimental job! | 21:33 |
| sshnaidm|afk | pabelanger, are you available for a quick chat just to give me a context? | 21:33 |
| pabelanger | sshnaidm|afk: to see how it would work, look at openstack/windmill. That is a integration testing that uses ansible to deploy stuff in the gate. Basically what tripleo-quickstart wants to do | 21:33 |
| pabelanger | sshnaidm|afk: not today, I am at the train station leaving Montreal for Ottawa | 21:33 |
| pabelanger | but we can talk tomorrow morning | 21:33 |
| pabelanger | morazi: yes! It is missing the tox.ini bit, but once added, check experimental should work | 21:34 |
| sshnaidm|afk | pabelanger, sure, in your time | 21:34 |
| EmilienM | morazi: which one? | 21:34 |
| ayoung | do we mirror the image artifacts anywhere? | 21:36 |
| EmilienM | slagle, bnemec: I'm off for a few hours, I'll be back on the evening. Can you look https://review.openstack.org/#/c/370434/ please and maybe +2 ? if it pass CI we could approve this revert and find another solution to fix the original bug. | 21:38 |
| bnemec | EmilienM: I will +2 anything that proposes to unbreak CI at this point. :-) | 21:39 |
| EmilienM | thanks | 21:39 |
| EmilienM | I'm off for real, ttyl | 21:39 |
| *** fragatin_ has joined #tripleo | 21:39 | |
| *** fragatina has quit IRC | 21:42 | |
| *** panda is now known as panda|Zz | 21:45 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: WIP: DONT MERGE: Test oooq with osinfra https://review.openstack.org/370437 | 21:53 |
| stevebaker | hey, did this revert unstick CI? https://review.openstack.org/#/c/370250/2 | 21:57 |
| *** fragatin_ has quit IRC | 21:58 | |
| *** fragatina has joined #tripleo | 21:59 | |
| bnemec | stevebaker: Not sure, it hasn't gotten through the gate yet. | 22:01 |
| bnemec | We have an unrelated problem with Mistral that is breaking the gating multinode job. :-( | 22:01 |
| *** rbrady has quit IRC | 22:02 | |
| *** rhallisey has quit IRC | 22:03 | |
| slagle | and...it failed the same way | 22:08 |
| *** ooolpbot has joined #tripleo | 22:10 | |
| *** jlinkes has quit IRC | 22:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1623606 | 22:10 |
| *** ooolpbot has quit IRC | 22:10 | |
| openstack | Launchpad bug 1623606 in tripleo "CI: jobs failing with mistral MessagingTimeout" [Critical,Confirmed] - Assigned to Dougal Matthews (d0ugal) | 22:10 |
| *** morazi has quit IRC | 22:11 | |
| openstackgerrit | James Slagle proposed openstack/instack-undercloud: Increase default mistral rpc response timeout https://review.openstack.org/370457 | 22:19 |
| *** yamahata has quit IRC | 22:20 | |
| *** jlinkes has joined #tripleo | 22:20 | |
| *** yamahata has joined #tripleo | 22:21 | |
| *** jlinkes has quit IRC | 22:25 | |
| slagle | EmilienM: bnemec : I'm just trying that now ^ | 22:27 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: WIP: DONT MERGE: Test oooq with osinfra https://review.openstack.org/370437 | 22:29 |
| EmilienM | slagle: Nice | 22:30 |
| EmilienM | slagle: i guess the option doesn't exist in puppet-mistral | 22:31 |
| slagle | EmilienM: it doesnt. i checked | 22:33 |
| EmilienM | If your patch works I'll directly patch the puppet module to avoid duplicate resource later | 22:33 |
| EmilienM | But we'll see | 22:33 |
| *** noslzzp has quit IRC | 22:38 | |
| *** phpcodemonkey has joined #tripleo | 22:40 | |
| *** rcernin has quit IRC | 22:43 | |
| slagle | EmilienM: i doubt it will fix it, b/c when it does pass, it's instantaneous | 22:46 |
| slagle | or close to it | 22:46 |
| *** benoit has quit IRC | 22:47 | |
| *** pradk has quit IRC | 22:47 | |
| *** benoit has joined #tripleo | 22:57 | |
| *** absubram has quit IRC | 22:57 | |
| *** slagle has quit IRC | 23:01 | |
| *** absubram has joined #tripleo | 23:01 | |
| *** benoit has quit IRC | 23:03 | |
| *** benoit has joined #tripleo | 23:03 | |
| *** ooolpbot has joined #tripleo | 23:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1623606 | 23:10 |
| *** ooolpbot has quit IRC | 23:10 | |
| openstack | Launchpad bug 1623606 in tripleo "CI: jobs failing with mistral MessagingTimeout" [Critical,In progress] - Assigned to James Slagle (james-slagle) | 23:10 |
| *** saneax-_-|AFK is now known as saneax | 23:12 | |
| *** thrash|bbl is now known as thrash | 23:19 | |
| *** pmannidi has joined #tripleo | 23:20 | |
| *** noslzzp has joined #tripleo | 23:24 | |
| *** rwsu has quit IRC | 23:32 | |
| *** myoung is now known as myoung|gone | 23:41 | |
| thrash | EmilienM: you around? | 23:44 |
| thrash | EmilienM: if you are... is this normal? https://github.com/openstack/puppet-zaqar/blame/master/manifests/db/sync.pp#L10 | 23:44 |
| thrash | EmilienM: this is happening with zaqar: http://logs.openstack.org/82/331682/30/check/gate-tripleo-ci-centos-7-scenario002-multinode/b66acc1/logs/postci.txt.gz#_2016-09-13_15_54_40_000 | 23:44 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!