| *** michapma_alt has quit IRC | 00:01 | |
| *** michapma_alt has joined #tripleo | 00:01 | |
| *** TSCHAK has quit IRC | 00:05 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Remove Glance Registry service https://review.openstack.org/417109 | 00:05 |
|---|---|---|
| openstackgerrit | Steve Baker proposed openstack/tripleo-heat-templates: set -e in run-os-net-config.sh https://review.openstack.org/411509 | 00:06 |
| *** alee has joined #tripleo | 00:09 | |
| openstackgerrit | Jason E. Rist proposed openstack/tripleo-ui: Adds ids namespaced artifacts on plans page. https://review.openstack.org/411102 | 00:12 |
| *** morazi has quit IRC | 00:14 | |
| *** dprince has quit IRC | 00:16 | |
| *** b00tcat has quit IRC | 00:17 | |
| *** dmarlin_ has quit IRC | 00:18 | |
| *** tzumainn has quit IRC | 00:22 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Add puppet-openstack-integration to delorean exclude list https://review.openstack.org/419128 | 00:23 |
| openstackgerrit | Alex Schultz proposed openstack/instack-undercloud: Validate vips when generating certificate too https://review.openstack.org/400364 | 00:26 |
| *** dsneddon_afk is now known as dsneddon | 00:28 | |
| *** social_ has joined #tripleo | 00:28 | |
| *** social has quit IRC | 00:29 | |
| *** michapma_alt has quit IRC | 00:30 | |
| *** michapma_alt has joined #tripleo | 00:30 | |
| *** jaosorior has quit IRC | 00:31 | |
| *** tzumainn has joined #tripleo | 00:32 | |
| *** dmarlin_ has joined #tripleo | 00:33 | |
| owalsh | mwhahaha: you still around? | 00:42 |
| *** zoli|gone has quit IRC | 00:45 | |
| *** bfournie has quit IRC | 00:48 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Target map-packages deprecation message https://review.openstack.org/421483 | 00:48 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Use strings in package-installs follow output https://review.openstack.org/420996 | 00:48 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Don't set base element path in run_tests.sh https://review.openstack.org/421538 | 00:48 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Basic logging for package-installs-squash https://review.openstack.org/421600 | 00:48 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: yum/install-packages output cleanup https://review.openstack.org/421601 | 00:48 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Fix package-installs-v2 output https://review.openstack.org/421602 | 00:48 |
| *** dmacpher has quit IRC | 00:48 | |
| *** zoliXXL has joined #tripleo | 00:50 | |
| *** michapma_alt has quit IRC | 00:58 | |
| *** michapma_alt has joined #tripleo | 00:58 | |
| *** bfournie has joined #tripleo | 00:59 | |
| mwhahaha | owalsh: what's up? | 01:02 |
| *** tzumainn has quit IRC | 01:07 | |
| *** jerrygb has quit IRC | 01:11 | |
| *** jerrygb has joined #tripleo | 01:14 | |
| *** limao has joined #tripleo | 01:17 | |
| *** michapma_alt has quit IRC | 01:21 | |
| *** sdake has quit IRC | 01:21 | |
| *** michapma_alt has joined #tripleo | 01:21 | |
| *** bswartz has quit IRC | 01:24 | |
| *** athomas has quit IRC | 01:27 | |
| *** lblanchard has quit IRC | 01:27 | |
| *** apetrich has quit IRC | 01:27 | |
| *** fpan has quit IRC | 01:27 | |
| *** pmannidi has quit IRC | 01:27 | |
| *** brault has quit IRC | 01:27 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Add Rabbit to the endpoint map https://review.openstack.org/420920 | 01:27 |
| *** athomas has joined #tripleo | 01:29 | |
| *** lblanchard has joined #tripleo | 01:29 | |
| *** apetrich has joined #tripleo | 01:29 | |
| *** fpan has joined #tripleo | 01:29 | |
| *** pmannidi has joined #tripleo | 01:29 | |
| *** brault has joined #tripleo | 01:29 | |
| *** xuao has joined #tripleo | 01:32 | |
| *** fultonj has quit IRC | 01:34 | |
| *** michapma_alt has quit IRC | 01:48 | |
| *** michchap_alt has joined #tripleo | 01:49 | |
| *** yamahata has quit IRC | 01:52 | |
| *** yamahata has joined #tripleo | 01:54 | |
| *** bana_k has quit IRC | 01:55 | |
| *** shinobu__ has quit IRC | 02:05 | |
| *** michapma_alt has joined #tripleo | 02:14 | |
| *** michchap_alt has quit IRC | 02:14 | |
| *** jrist has quit IRC | 02:18 | |
| openstackgerrit | Merged openstack/tripleo-quickstart: Revert Ansible version back to 2.2.0.0 https://review.openstack.org/421514 | 02:20 |
| *** dmarlin_ has quit IRC | 02:22 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras: remove docker-network-isolation heat parameters https://review.openstack.org/421630 | 02:24 |
| *** jrist has joined #tripleo | 02:32 | |
| *** michapma_alt has quit IRC | 02:37 | |
| *** michapma_alt has joined #tripleo | 02:37 | |
| *** davidlenwell_ has joined #tripleo | 02:38 | |
| *** lblanchard has quit IRC | 02:45 | |
| *** fzdarsky_ has joined #tripleo | 02:46 | |
| *** fzdarsky|afk has quit IRC | 02:50 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Preinstall pyOpenSSL https://review.openstack.org/421637 | 02:51 |
| *** dsariel has quit IRC | 02:51 | |
| *** bana_k has joined #tripleo | 02:53 | |
| *** davidlenwell_ has quit IRC | 03:06 | |
| *** michapma_alt has quit IRC | 03:06 | |
| *** bana_k has quit IRC | 03:07 | |
| *** michapma_alt has joined #tripleo | 03:07 | |
| *** bana_k has joined #tripleo | 03:08 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: DO NOT MERGE - testing TLS everywhere job https://review.openstack.org/419529 | 03:08 |
| *** yamahata has quit IRC | 03:26 | |
| *** tbonds has joined #tripleo | 03:31 | |
| openstackgerrit | Ryan Brady proposed openstack/tripleo-common: Adds support to create deployment plan from git repository https://review.openstack.org/417280 | 03:32 |
| *** tbonds has quit IRC | 03:33 | |
| *** tbonds has joined #tripleo | 03:33 | |
| *** michapma_alt has quit IRC | 03:34 | |
| *** michapma_alt has joined #tripleo | 03:35 | |
| *** ramishra has quit IRC | 03:36 | |
| *** rwsu has quit IRC | 03:37 | |
| *** ramishra has joined #tripleo | 03:38 | |
| *** michapma_alt has quit IRC | 03:42 | |
| *** sudipto_ has joined #tripleo | 03:50 | |
| *** sudipto has joined #tripleo | 03:50 | |
| *** bana_k has quit IRC | 03:50 | |
| *** sdake has joined #tripleo | 03:53 | |
| *** gkadam has joined #tripleo | 03:57 | |
| *** links has joined #tripleo | 04:02 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Add endpoint map matching validation to validate-yaml.py https://review.openstack.org/421339 | 04:04 |
| openstackgerrit | Merged openstack/python-tripleoclient: Release notes for 5.8.0 (final Ocata) https://review.openstack.org/420744 | 04:05 |
| openstackgerrit | Merged openstack/diskimage-builder: Replace yaml.load() with yaml.safe_load() https://review.openstack.org/420553 | 04:08 |
| *** psahoo has joined #tripleo | 04:11 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Fix coverage report https://review.openstack.org/421652 | 04:11 |
| *** ramishra has quit IRC | 04:12 | |
| *** sudipto has quit IRC | 04:14 | |
| *** sudipto_ has quit IRC | 04:14 | |
| *** ramishra has joined #tripleo | 04:25 | |
| *** sudipto_ has joined #tripleo | 04:35 | |
| *** sudipto has joined #tripleo | 04:35 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Add Constraints support https://review.openstack.org/412747 | 04:36 |
| *** udesale has joined #tripleo | 04:37 | |
| *** ramishra has quit IRC | 04:39 | |
| *** ramishra has joined #tripleo | 04:41 | |
| *** TSCHAK has joined #tripleo | 04:46 | |
| *** TSCHAK has quit IRC | 04:55 | |
| *** prateek has joined #tripleo | 05:04 | |
| *** sudipto_ has quit IRC | 05:10 | |
| *** sudipto has quit IRC | 05:10 | |
| *** sudipto has joined #tripleo | 05:12 | |
| *** sudipto_ has joined #tripleo | 05:13 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Fix coverage report https://review.openstack.org/421652 | 05:14 |
| *** sdake has quit IRC | 05:16 | |
| *** nmathew has joined #tripleo | 05:16 | |
| *** skramaja has joined #tripleo | 05:21 | |
| *** sdake has joined #tripleo | 05:25 | |
| *** sdake has quit IRC | 05:25 | |
| *** sdake has joined #tripleo | 05:25 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Update platform support to reality https://review.openstack.org/418204 | 05:26 |
| *** psahoo has quit IRC | 05:36 | |
| *** psahoo has joined #tripleo | 05:36 | |
| *** yamahata has joined #tripleo | 05:43 | |
| *** jerrygb has quit IRC | 05:45 | |
| *** jerrygb has joined #tripleo | 05:46 | |
| *** sdake has quit IRC | 05:46 | |
| *** jerrygb has quit IRC | 05:50 | |
| *** abregman has joined #tripleo | 05:55 | |
| *** padkrish has joined #tripleo | 06:01 | |
| *** dmacpher has joined #tripleo | 06:19 | |
| *** nyechiel_ has joined #tripleo | 06:20 | |
| *** dmacpher has quit IRC | 06:31 | |
| *** xuao has quit IRC | 06:44 | |
| *** ramishra has quit IRC | 06:48 | |
| *** ramishra has joined #tripleo | 06:48 | |
| *** masco has joined #tripleo | 06:54 | |
| *** bana_k has joined #tripleo | 06:56 | |
| *** saneax-_-|AFK is now known as saneax | 07:00 | |
| openstackgerrit | Cao Xuan Hoang proposed openstack/instack-undercloud: Add Python 3.5 classifier and venv https://review.openstack.org/421700 | 07:01 |
| *** ealcaniz has joined #tripleo | 07:10 | |
| *** jrist has quit IRC | 07:11 | |
| *** jrist has joined #tripleo | 07:11 | |
| *** jrist has joined #tripleo | 07:11 | |
| *** jrist has quit IRC | 07:14 | |
| *** jrist has joined #tripleo | 07:15 | |
| *** jrist has joined #tripleo | 07:15 | |
| *** iranzo has joined #tripleo | 07:20 | |
| *** jprovazn has joined #tripleo | 07:20 | |
| *** dsariel has joined #tripleo | 07:22 | |
| *** ohamada has joined #tripleo | 07:25 | |
| *** tesseract has joined #tripleo | 07:25 | |
| *** rasca has joined #tripleo | 07:25 | |
| *** tesseract has joined #tripleo | 07:26 | |
| *** florianf has joined #tripleo | 07:29 | |
| *** flepied has joined #tripleo | 07:30 | |
| *** aufi has joined #tripleo | 07:37 | |
| *** tremble has joined #tripleo | 07:37 | |
| *** tremble has joined #tripleo | 07:37 | |
| *** padkrish has quit IRC | 07:43 | |
| *** tobias_fiberdata has joined #tripleo | 07:43 | |
| *** pmannidi has quit IRC | 07:53 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Adds a AllNodesPreUpgradeConfig before the upgrade steps https://review.openstack.org/419886 | 07:55 |
| *** ealcaniz has quit IRC | 07:58 | |
| *** ebarrera has joined #tripleo | 08:01 | |
| *** yprokule has joined #tripleo | 08:04 | |
| *** hjensas has joined #tripleo | 08:08 | |
| *** hjensas has quit IRC | 08:08 | |
| *** hjensas has joined #tripleo | 08:08 | |
| *** mcornea has joined #tripleo | 08:11 | |
| *** shardy has joined #tripleo | 08:12 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: WIP: Fix pingtest for nonha ovb oooq job https://review.openstack.org/421729 | 08:15 |
| *** jlinkes has joined #tripleo | 08:18 | |
| *** panda|zZ is now known as panda | 08:19 | |
| *** sshnaidm|afk is now known as sshnaidm | 08:20 | |
| *** psahoo has quit IRC | 08:25 | |
| *** abehl has joined #tripleo | 08:25 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Fix package-installs-v2 output https://review.openstack.org/421602 | 08:26 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Target map-packages deprecation message https://review.openstack.org/421483 | 08:26 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: yum/install-packages output cleanup https://review.openstack.org/421601 | 08:27 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Basic logging for package-installs-squash https://review.openstack.org/421600 | 08:27 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Use strings in package-installs follow output https://review.openstack.org/420996 | 08:27 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Don't set base element path in run_functests.sh https://review.openstack.org/421538 | 08:27 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Preinstall pyOpenSSL https://review.openstack.org/421637 | 08:27 |
| *** bana_k has quit IRC | 08:27 | |
| *** abregman has quit IRC | 08:27 | |
| *** abregman has joined #tripleo | 08:29 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci: [WIP] DO NOT MERGE - test tempest job using quickstart https://review.openstack.org/421272 | 08:35 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci: OOQ/OVB: Add configurations for tempest job https://review.openstack.org/420620 | 08:35 |
| panda | sshnaidm: ^ updated commit, let's see the results for this | 08:35 |
| *** amoralej|off is now known as amoralej | 08:35 | |
| panda | sshnaidm: mmmh, periodic job did not run yesterday ... | 08:37 |
| panda | sshnaidm: sorry, I mean this morning | 08:38 |
| sshnaidm | panda, maybe didn't finish yet | 08:38 |
| sshnaidm | panda, yeah, they still run | 08:39 |
| sshnaidm | panda, because of this ovb queue I believe | 08:39 |
| panda | yep seening them in zuul | 08:40 |
| *** chem has joined #tripleo | 08:45 | |
| bkero | sshnaidm: we should get https://review.openstack.org/#/c/420954/ +w'd | 08:46 |
| bkero | It's required as a dependency to get the multinode jobs failing properly | 08:47 |
| sshnaidm | bkero, yeah, needs additional +2 there | 08:47 |
| bkero | Oh because you're core | 08:48 |
| sshnaidm | bkero, no, I'm not unfortunately :) | 08:49 |
| bkero | Oh. What's with the second +2 then? | 08:49 |
| sshnaidm | bkero, even if I was, it's not common to approve your own patch.. | 08:49 |
| sshnaidm | bkero, added EmilienM there | 08:49 |
| sshnaidm | bkero, we have a problem with approving oooq patches in tripleo-ci, big problem | 08:50 |
| bkero | I see. It's an oooq thing. | 08:50 |
| *** abregman has quit IRC | 08:50 | |
| sshnaidm | bkero, only three people review and merge them | 08:50 |
| bkero | Yeahhh | 08:51 |
| ccamacho | done | 08:51 |
| sshnaidm | ccamacho, thanks! | 08:51 |
| bkero | ccamacho++ | 08:51 |
| sshnaidm | four people :D | 08:52 |
| bkero | Thanks for helping review and land it | 08:52 |
| *** jpich has joined #tripleo | 08:52 | |
| bkero | Then there's https://review.openstack.org/#/c/420912 and https://review.openstack.org/#/c/420795 | 08:53 |
| sshnaidm | bkero, I think you need to merge this one before: https://review.openstack.org/#/c/420795 | 08:53 |
| ccamacho | folks, my oooq kung-fu is not much strong, but if there are submissions with changes similar to this one please just ping me for checking them. | 08:53 |
| sshnaidm | ccamacho, ++ | 08:53 |
| *** abregman has joined #tripleo | 08:53 | |
| bkero | ccamacho: I try to explain the rationale in the commit message so that you can get some context for helping understand it. | 08:53 |
| sshnaidm | bkero, you need to land the base patch before ones that rebase on it | 08:53 |
| bkero | sshnaidm: You're right. The base commit is the one you mentioned. | 08:55 |
| bkero | That needs a +2 as well. | 08:55 |
| bkero | ccamacho: not sure if you're up to grokking 420795 | 08:55 |
| * ccamacho checking | 08:55 | |
| ccamacho | but will take me some time | 08:56 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Add swift service support for composable upgrades https://review.openstack.org/405245 | 08:56 |
| sshnaidm | it's little risk, because it touches only multinode stuff | 08:57 |
| *** ealcaniz has joined #tripleo | 08:57 | |
| bkero | The multinode stuff is still only an experimental job | 08:57 |
| *** oneswig has joined #tripleo | 08:57 | |
| *** zzzeek has quit IRC | 09:00 | |
| *** zzzeek has joined #tripleo | 09:00 | |
| *** gfidente has joined #tripleo | 09:00 | |
| ccamacho | gfidente morning, yesterday I manage to check 405627, 405999, 410695 took longer that I expected, but checked at the end.. | 09:06 |
| gfidente | ccamacho thank you very much | 09:08 |
| gfidente | and no worries, I am lagging behind with reviews as well | 09:08 |
| *** zoliXXL is now known as zoli | 09:11 | |
| *** fzdarsky_ is now known as fzdarsky | 09:15 | |
| *** cylopez has joined #tripleo | 09:16 | |
| *** jpena|off is now known as jpena | 09:22 | |
| openstackgerrit | Julie Pichon proposed openstack/python-tripleoclient: Release note for deprecated baremetal commands https://review.openstack.org/421198 | 09:23 |
| *** derekh has joined #tripleo | 09:23 | |
| *** karimb has joined #tripleo | 09:25 | |
| *** limao has quit IRC | 09:25 | |
| *** ramishra has quit IRC | 09:26 | |
| *** limao has joined #tripleo | 09:26 | |
| *** ramishra has joined #tripleo | 09:26 | |
| *** limao has quit IRC | 09:30 | |
| therve | jpich, Do you know what kind of issues dtrainor is getting? | 09:31 |
| therve | I answered his message, but things seem to work fine for me | 09:31 |
| jpich | therve: Thanks for looking into it! I don't really know anything more that is in the email, he also added a paragraph to the whiteboard at https://blueprints.launchpad.net/tripleo/+spec/proxy-undercloud-api-services to describe the issue yesterday | 09:32 |
| therve | Mokay | 09:34 |
| therve | Timezones are bad :) | 09:34 |
| *** jpena is now known as jpena|off | 09:35 | |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: Add new feature to consume external images https://review.openstack.org/421763 | 09:36 |
| *** pcaruana has joined #tripleo | 09:41 | |
| *** jpena|off is now known as jpena | 09:42 | |
| *** salmankhan has joined #tripleo | 09:43 | |
| *** lucas-afk is now known as lucasagomes | 09:45 | |
| jtomasek | florianf: what do you think? https://bugs.launchpad.net/tripleo/+bug/1657388 | 09:56 |
| openstack | Launchpad bug 1657388 in tripleo "Change the way of injecting Intl" [Medium,Triaged] | 09:56 |
| openstackgerrit | Chris Jones proposed openstack/python-tripleoclient: Add baremetal_deploy.GenerateFencing https://review.openstack.org/417898 | 09:57 |
| *** owalsh_ has joined #tripleo | 09:59 | |
| *** owalsh has quit IRC | 09:59 | |
| florianf | jtomasek: that sounds like an excellent idea. will the context automatically be passed to all components down the line? | 10:01 |
| jtomasek | florianf: yes, but it will be accessible only by components which explicitly ask for it by defining contextTypes they accept | 10:01 |
| florianf | jtomasek: cool | 10:02 |
| jtomasek | florianf: can we merge your i18n patches so I can rebase mine on top? | 10:04 |
| florianf | jtomasek: I'm not sure they have the necessary reviews yet | 10:05 |
| florianf | jtomasek: Oh, I see you +2'd some of them | 10:05 |
| *** mgoddard has joined #tripleo | 10:05 | |
| jtomasek | florianf: I think I went through all of them, just 1 requires tiny update | 10:05 |
| florianf | jtomasek: and we have to discuss notifications | 10:06 |
| jtomasek | florianf: yeah, I've been looking into it a bit, I'll add notes later this week if thats ok | 10:06 |
| *** yamahata has quit IRC | 10:06 | |
| florianf | jtomasek: I think time is becoming a factor :) | 10:07 |
| jtomasek | florianf: yeah:/ | 10:07 |
| jtomasek | florianf: same for parameters work | 10:08 |
| florianf | jtomasek: I'll look into it later today! | 10:08 |
| florianf | jtomasek: I started the workflow on most of the i18n patches and will update the plans one... | 10:08 |
| jtomasek | florianf: thanks | 10:08 |
| mgoddard | hi tripleo, can anyone tell me whether upgrading an overcloud (L to M) is supported in environments without pacemaker? | 10:09 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-docs: Add swap usage in the docs. https://review.openstack.org/421780 | 10:10 |
| jtomasek | jistr, marios: ^^ | 10:13 |
| *** oneswig has quit IRC | 10:13 | |
| *** oneswig has joined #tripleo | 10:13 | |
| *** oneswig has quit IRC | 10:14 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Use multiline var in multinode job https://review.openstack.org/420954 | 10:15 |
| openstackgerrit | Sanjay Upadhyay proposed openstack/os-net-config: Remove child members activation for OVS-DPDK bond https://review.openstack.org/417805 | 10:18 |
| jistr | mgoddard: hi, AFAIK we haven't had enough bandwidth to have anyone focus on non-pacemaker environments. So i wouldn't consider it supported. | 10:18 |
| marios | jtomasek: mgoddard jistr the short answer is that wasn't really tested | 10:18 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Add THT for fossw ML2 plugin in networking-fujitsu https://review.openstack.org/413778 | 10:18 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Environment parameters https://review.openstack.org/418798 | 10:19 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Normalize parameters https://review.openstack.org/403422 | 10:19 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Add js-yaml dependency https://review.openstack.org/419964 | 10:19 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Update app state when parameters get updated https://review.openstack.org/409825 | 10:19 |
| *** hjensas has quit IRC | 10:19 | |
| marios | jtomasek: mgoddard jistr for sure trying to get the controller_pacemaker scripts that normally run on controllers to work when pacemaker isn't there (they assume it is and try to stop cluster etc etc ) will need some TLC | 10:19 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Conditionally set OVS agent firewall driver https://review.openstack.org/420947 | 10:19 |
| *** owalsh_ is now known as owalsh | 10:20 | |
| mgoddard | jtomasek, marios: understood, thanks | 10:20 |
| marios | jtomasek: mgoddard as jistr says we made a decision at some point that we would support pcmk/ha deployments since that was the most highly requested/used by people at the time | 10:20 |
| shardy | for newton->ocata with composable upgrades things work without pacemaker however (actually, only without pacemaker at this point but pacemaker support is under review) | 10:20 |
| *** karimb has quit IRC | 10:20 | |
| openstackgerrit | Merged openstack/puppet-tripleo: Set ceph key when using manila ceph backend https://review.openstack.org/405999 | 10:20 |
| shardy | previous versions do have scripts that expect pacemaker, and I suspect that will remain the main/only well tested path | 10:20 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Set manila cephfs backend if ceph is deployed https://review.openstack.org/405627 | 10:20 |
| openstackgerrit | Merged openstack/tripleo-common: Generate a CephX for manila ceph client service https://review.openstack.org/410695 | 10:21 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Bump missing template names to ocata https://review.openstack.org/418438 | 10:21 |
| Ng | shardy: would you have some time today to look at https://review.openstack.org/#/c/390397 ? | 10:22 |
| openstackgerrit | Sanjay Upadhyay proposed openstack/os-net-config: Remove child members activation for OVS-DPDK bond https://review.openstack.org/417805 | 10:22 |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-quickstart-extras: Fix ansible-lint.sh script for roles. https://review.openstack.org/421788 | 10:22 |
| shardy | Ng: lgtm but it appears to be blocked on https://review.openstack.org/#/c/390396/ ? | 10:27 |
| owalsh | marios: hi, just looking at your comment on https://review.openstack.org/419886 | 10:27 |
| Ng | shardy: yeah, I was just wondering if I could get my +2s in a row on the puppet-tripleo part while I sort out the puppet-pacemaker tests with mwaha for 390396 :) | 10:28 |
| shardy | Ng: ack, it's in my queue now so will look out for it being ready to land :) | 10:28 |
| Ng | :) | 10:28 |
| Ng | thanks | 10:28 |
| *** chem has quit IRC | 10:30 | |
| *** karimb has joined #tripleo | 10:31 | |
| *** hjensas has joined #tripleo | 10:32 | |
| *** hjensas has quit IRC | 10:32 | |
| *** hjensas has joined #tripleo | 10:32 | |
| jpich | jtomasek: https://review.openstack.org/#/c/419964/ never got packager ack fwiw... | 10:36 |
| *** milan has joined #tripleo | 10:36 | |
| *** b00tcat has joined #tripleo | 10:38 | |
| *** pcaruana has quit IRC | 10:38 | |
| marios | owalsh: o/ was mostly wondering if we now need to also pin nova-compute before we deploy placement api, even if we aren't upgrading nova-compute yet | 10:38 |
| jtomasek | jpich: oh, I thought that he mentioned it in the react-cookie patch | 10:39 |
| jtomasek | jpich: hmm, we're getting quite tight on time with all those patches being blocked | 10:39 |
| panda | there's an error in my change for promotion :( | 10:39 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras: remove docker-network-isolation heat parameters https://review.openstack.org/421630 | 10:40 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart-extras: Add support for whole disk images https://review.openstack.org/421805 | 10:40 |
| jpich | jtomasek: Hm, I don't know what "both deps" covered in the react-cookie comment, maybe you are right - I don't have the background on that conversation. honza, do you know? | 10:41 |
| sshnaidm | panda, CHANGES[@]: unbound variable ? | 10:41 |
| panda | sshnaidm: yes, I didn't test with set -u :( | 10:42 |
| panda | pushing a patch | 10:42 |
| owalsh | marios: it's was to set version pin to auto on Newton computes before we upgrade the controller to Ocata where we will now run puppet instead of delaying until the computes are all upgraded | 10:43 |
| owalsh | marios: but i think you might be correct ,we don't need to do this... | 10:43 |
| honza | jpich: jtomasek: we need explicit approval for js-yaml, I'd say | 10:44 |
| openstackgerrit | Merged openstack/instack: Fix test-requirements https://review.openstack.org/420348 | 10:44 |
| jpich | honza: Makes sense to me | 10:45 |
| owalsh | marios: we just need to pin the controller to auto when we upgrade to Ocata so it will continue speaking to Newton computes, ditto for the computes as they are upgaded... | 10:45 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Parameterizes Nova API default floating IP pool https://review.openstack.org/419631 | 10:45 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Correct typo in ManagementAllocationPools in network-environment.yaml https://review.openstack.org/418186 | 10:45 |
| marios | owalsh: right and it will happen in that order, i.e. controllers first (even though we do some compute config first, we don't upgrade them) | 10:46 |
| honza | jpich: This two-phase process is a little annoying though because it necessarily leads to broken builds | 10:46 |
| marios | owalsh: so lets discuss on the review and see what others think. if it isn't a pre-requisite for the placement config then lets do what we did before and will remove it on next pass | 10:46 |
| milan | guys, w/r https://bugs.launchpad.net/tripleo/+bug/1651719 would it be possible for you to try whether this patch fixes the error: https://review.openstack.org/#/c/421285/ | 10:47 |
| openstack | Launchpad bug 1651719 in Ironic Inspector "Introspection fails on nodes with the same IPMI address but different IPMI ports" [High,In progress] - Assigned to milan k (vetrisko) | 10:47 |
| jtomasek | honza, jpich: maybe we can include adding deps to the patch which requires it and make a proper note to the commit message, so it is easy for packagers to review it | 10:50 |
| jtomasek | (next time) | 10:50 |
| jpich | honza: It's because our process is too lightweight (e.g. this is the process and considerations for adding new Python dependencies https://github.com/openstack/requirements/blob/master/README.rst ). Probably our CI should block patches from being merged until the deps the patch adds are included in the -deps package, just like it would for any other openstack project | 10:50 |
| honza | jtomasek: that would be my preference but i got push back on this idea recently | 10:51 |
| honza | jpich: i can look into that if you want | 10:52 |
| honza | jtomasek: jpich: js-yaml just got approved | 10:53 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci: Promotion: fix promotion check https://review.openstack.org/421822 | 10:53 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Fix pingtest for nonha ovb oooq job https://review.openstack.org/421729 | 10:53 |
| jtomasek | honza: great | 10:53 |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo: Do not depend on bootstrap_nodeid for any pacemaker profile https://review.openstack.org/421405 | 10:53 |
| panda | derekh: fix on the fix ^ (tested with set -u this time) | 10:54 |
| *** paramite has joined #tripleo | 10:54 | |
| jpich | jtomasek, honza: I think it's better to separate "approving dependencies" from "approving a code change" since only the first requires packager input, and it's already easy to miss if we got it or not. Hopefully it makes it easier for the packaging team to review as well | 10:54 |
| *** jprovazn has quit IRC | 10:55 | |
| jpich | honza: I think the CI failing thing might be related to the issue amoralej mentioned on the rdoproject review - CI is building from master, not from the patch being reviewed for some reason | 10:55 |
| honza | jpich: ha | 10:55 |
| jpich | honza: So... I guess we might want to rebase/recheck something to see if that theory holds, as in everything should be failing now if that's the case?? | 10:56 |
| honza | jpich: yes, that's my understanding | 10:57 |
| jpich | honza: yeah no need to rebase, it's happening already | 10:57 |
| jpich | honza: So we're in the same situation as last week with all our patches failing until we get the new deps package. I'm going to open a bug about getting CI to fail early/build correctly from the patch-being-reviewed rather than master for tripleo-ui | 11:01 |
| *** salmankhan has quit IRC | 11:04 | |
| *** salmankhan has joined #tripleo | 11:05 | |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-quickstart: Add composable deployment config files and mixed release https://review.openstack.org/410831 | 11:05 |
| *** zoli is now known as zoli|lunch | 11:07 | |
| panda | sshnaidm: looks like with the quotes fixed, the job tested only smoke .. | 11:09 |
| *** tosky has joined #tripleo | 11:10 | |
| *** adarazs is now known as adarazs_lunch | 11:11 | |
| bandini | any takers for https://review.openstack.org/#/c/421405/ fairly simple one | 11:11 |
| bandini | gfidente: ^ ;) | 11:11 |
| *** udesale has quit IRC | 11:13 | |
| *** panda is now known as panda|lunch | 11:15 | |
| openstackgerrit | Saravanan KR proposed openstack/tripleo-common: Provided an option to disable the password generation https://review.openstack.org/414424 | 11:15 |
| *** cylopez has quit IRC | 11:16 | |
| jpich | honza: FYI - https://bugs.launchpad.net/tripleo/+bug/1657416 | 11:17 |
| openstack | Launchpad bug 1657416 in tripleo "CI not building tripleo-ui correctly" [High,Triaged] | 11:17 |
| *** nmathew has quit IRC | 11:18 | |
| *** karimb has quit IRC | 11:19 | |
| *** panda|lunch is now known as panda | 11:19 | |
| panda | jpich: what's the name of the dlrn package for tripleo-ui ? | 11:19 |
| jpich | panda: openstack-tripleo-ui I think? | 11:20 |
| sshnaidm | panda, but it ran with -e 'test_regex=.*smoke' | 11:20 |
| *** psahoo has joined #tripleo | 11:21 | |
| sshnaidm | panda, http://logs.openstack.org/72/421272/5/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/3f81e16/console.html#_2017-01-18_10_04_54_848994 | 11:21 |
| jtomasek | honza: regarding your comment on https://review.openstack.org/#/c/403422/11/src/js/selectors/parameters.js | 11:22 |
| panda | sshnaidm: so it's not honouring -e test_regex from command line at all | 11:22 |
| jtomasek | honza: would you be ok with me just adding comments to that function? It seems fairly complicated but I think that splitting it might make it look even more complicated | 11:23 |
| sshnaidm | panda, this is the command line | 11:24 |
| jtomasek | honza: first 5 lines is just traversing nested resources tree to find required resource | 11:24 |
| sshnaidm | panda, oh, I see what is the problem | 11:25 |
| *** hjensas has quit IRC | 11:25 | |
| *** sudipto_ has quit IRC | 11:25 | |
| *** sudipto has quit IRC | 11:25 | |
| sshnaidm | panda, tempest run doesn't use your "OOOQ_ARGS", it uses only OOOQ_DEFAULT_AGRS | 11:25 |
| openstackgerrit | avnish proposed openstack/tripleo-quickstart: Use upper-constraints for tox envs https://review.openstack.org/419261 | 11:26 |
| sshnaidm | panda, left you comment in the patch | 11:28 |
| *** psahoo has quit IRC | 11:28 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Check that all templates are using the release alias https://review.openstack.org/421326 | 11:30 |
| *** karimb has joined #tripleo | 11:30 | |
| *** dtantsur|afk is now known as dtantsur | 11:32 | |
| *** akrivoka has joined #tripleo | 11:35 | |
| ccamacho | Guys mind to check https://review.openstack.org/#/c/421326 had 2 +2 but had to add again a simple nit fix that got merged before this, I'm not self expedited approving it. | 11:35 |
| *** abregman is now known as abregman|afk | 11:39 | |
| marios | ccamacho: lgtm... should we wait for jenkins V5 THO? | 11:40 |
| ccamacho | hey marios thanks, yeah if you +2 it again I can +A once got answer from Jenkins | 11:41 |
| marios | ccamacho: yah did that | 11:41 |
| ccamacho | awesome thanks! | 11:41 |
| marios | np | 11:41 |
| *** jkilpatr has quit IRC | 11:42 | |
| *** hjensas has joined #tripleo | 11:45 | |
| *** dprince has joined #tripleo | 11:45 | |
| *** b00tcat has quit IRC | 11:48 | |
| *** pkovar has joined #tripleo | 11:51 | |
| *** ccamacho is now known as ccamacho|lunc | 11:55 | |
| *** ccamacho|lunc is now known as ccamacho|lunch | 11:55 | |
| openstackgerrit | Merged openstack/tripleo-puppet-elements: Octavia integration https://review.openstack.org/412012 | 11:55 |
| d0ugal | jpich, jtomasek: it would be good for you to take a look at https://review.openstack.org/#/c/417280/ as potential consumers. | 11:55 |
| *** pcaruana has joined #tripleo | 11:56 | |
| panda | jpich: I know what's happening | 11:56 |
| jpich | d0ugal: Thanks for the heads-up | 11:56 |
| *** salmankhan has quit IRC | 11:57 | |
| jpich | panda: Ah?? | 11:57 |
| *** salmankhan has joined #tripleo | 11:59 | |
| panda | jpich: still don't know why, but I just have to find that evil rm | 11:59 |
| jpich | panda: ?? Feel free to update the bug / assign it to yourself if you know what you're doing? :) | 12:00 |
| panda | jpich: tripleo-ui is cloned from repo, but it shouldn't be, changes to test are automatically downloaded but devstack-gate | 12:00 |
| panda | by* | 12:00 |
| panda | som something deletes it, and when it's time to build a dlrn package on that ref, the ref is gone, and the sript download it from default location | 12:01 |
| jtomasek | d0ugal: nice, will take a look | 12:01 |
| *** gkadam has quit IRC | 12:01 | |
| panda | jpich: so I just have to find what deletes the ref before building | 12:02 |
| jpich | panda: I see. I wonder... I was looking with suspicion at https://github.com/openstack-infra/tripleo-ci/blob/master/scripts/tripleo-jobs-gerrit.py#L23 , because tripleo-ui doesn't seem to be in the list of projects at http://logs.openstack.org/64/419964/3/check/gate-tripleo-ci-centos-7-undercloud/4ccb57f/console.html#_2017-01-18_10_22_42_124795 - but I'm not familiar enough with the gate to know if that's relevant - could it be why? | 12:02 |
| jpich | It might not be treated like an openstack project? | 12:03 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Adds a pre-upgrade check that service is running (step0) https://review.openstack.org/414598 | 12:03 |
| panda | jpich: that script is used only to build reports | 12:05 |
| *** salmankhan has quit IRC | 12:06 | |
| jpich | panda: Darn. Thanks for the info. So not the right place, but would the ui not being in the projects list on the gate be relevant, or is it beside the point? | 12:06 |
| panda | jpich: which project list ? | 12:09 |
| panda | jpich: checking if the project name is mapped correctly to the local dir, that may also be the problem | 12:09 |
| jpich | panda: http://logs.openstack.org/64/419964/3/check/gate-tripleo-ci-centos-7-undercloud/4ccb57f/console.html#_2017-01-18_10_22_42_124795 but never mind, thank you for looking into this and let me know if I can help with anything. I'll eagerly review the patches to learn about how this all fit together :) | 12:11 |
| *** salmankhan has joined #tripleo | 12:14 | |
| *** michapma_alt has joined #tripleo | 12:14 | |
| openstackgerrit | Alejandro Andreu proposed openstack/tripleo-heat-templates: Create MidoNet services https://review.openstack.org/396692 | 12:15 |
| *** zoli|lunch is now known as zoli | 12:17 | |
| *** zoli is now known as zoliXXL | 12:17 | |
| *** jkilpatr has joined #tripleo | 12:18 | |
| *** b00tcat has joined #tripleo | 12:18 | |
| *** b00tcat has joined #tripleo | 12:18 | |
| *** akrivoka has quit IRC | 12:20 | |
| *** cylopez has joined #tripleo | 12:20 | |
| *** akrivoka has joined #tripleo | 12:21 | |
| openstackgerrit | Alejandro Andreu proposed openstack/puppet-tripleo: Add new tripleo::network::midonet::agent class https://review.openstack.org/400618 | 12:21 |
| openstackgerrit | Steven Hardy proposed openstack/puppet-tripleo: Add auditd profile https://review.openstack.org/421872 | 12:26 |
| openstackgerrit | Steven Hardy proposed openstack/tripleo-heat-templates: Add audit composable service https://review.openstack.org/421873 | 12:26 |
| shardy | lhinds: ^^ | 12:26 |
| lhinds | shardy: thanks | 12:26 |
| shardy | lhinds: untested but hopefully illustrates the approach - feel free to to take those reviews over and push whatever fixes you need :) | 12:27 |
| lhinds | thanks , will do shardy | 12:27 |
| lhinds | shardy: thats actually a lot simpler then I envisioned. | 12:28 |
| lhinds | i like | 12:28 |
| *** chem has joined #tripleo | 12:28 | |
| shardy | :) | 12:28 |
| *** shardy is now known as shardy_lunch | 12:29 | |
| EmilienM | hello | 12:29 |
| lhinds | hey EmilienM | 12:29 |
| *** lhinds is now known as lhinds|afk | 12:30 | |
| *** masco has quit IRC | 12:31 | |
| *** chem has quit IRC | 12:31 | |
| *** chem has joined #tripleo | 12:32 | |
| *** adarazs_lunch is now known as adarazs | 12:32 | |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Environment parameters https://review.openstack.org/418798 | 12:34 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Normalize parameters https://review.openstack.org/403422 | 12:34 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Update app state when parameters get updated https://review.openstack.org/409825 | 12:34 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci: [WIP] DO NOT MERGE - test tempest job using quickstart https://review.openstack.org/421272 | 12:34 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci: OOQ/OVB: Add configurations for tempest job https://review.openstack.org/420620 | 12:34 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart-extras: Collect logs of containers https://review.openstack.org/419018 | 12:36 |
| *** lblanchard has joined #tripleo | 12:37 | |
| sshnaidm | cores, please review/merge fixes for OVB quickstart job: https://review.openstack.org/#/c/418468/ | 12:42 |
| *** lblanchard has quit IRC | 12:43 | |
| *** b00tcat has quit IRC | 12:44 | |
| *** sudipto_ has joined #tripleo | 12:44 | |
| *** sudipto has joined #tripleo | 12:44 | |
| *** links has quit IRC | 12:46 | |
| sshnaidm | panda, you need to add extra args as last args | 12:48 |
| sshnaidm | panda, otherwise they're overridden by later args again | 12:48 |
| rook | weshay: do you guys use a prebuilt undercloud image? | 12:52 |
| *** jprovazn has joined #tripleo | 12:52 | |
| panda | jpich: lol, you were right, we're not adding triplo-ui to the projects to clone | 12:54 |
| jpich | panda: Cool! Easy to fix?? | 12:55 |
| panda | jpich: just a line to add | 12:55 |
| panda | jpich: but checking if it's enough | 12:56 |
| EmilienM | sshnaidm: where are the other logs? http://logs.openstack.org/68/418468/5/experimental/gate-tripleo-ci-centos-7-nonha-multinode-oooq-nv/1fdfce6/logs/subnode-2/var/log/ | 12:57 |
| *** sdake_ has joined #tripleo | 12:57 | |
| *** jayg|g0n3 is now known as jayg | 12:58 | |
| *** ccamacho|lunch is now known as ccamacho | 12:58 | |
| sshnaidm | EmilienM, it's multinode, it's not fixed yet afaiu | 12:58 |
| sshnaidm | bkero, ^^ | 12:58 |
| *** sdake__ has joined #tripleo | 12:58 | |
| sshnaidm | EmilienM, the patch deals with ovb only in this stage | 12:59 |
| EmilienM | sshnaidm: ok | 12:59 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci: [WIP] DO NOT MERGE - test tempest job using quickstart https://review.openstack.org/421272 | 13:00 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci: OOQ/OVB: Add configurations for tempest job https://review.openstack.org/420620 | 13:00 |
| *** salmankhan has quit IRC | 13:00 | |
| *** salmankhan has joined #tripleo | 13:00 | |
| openstackgerrit | Chris Jones proposed openstack/python-tripleoclient: Add baremetal_deploy.GenerateFencing https://review.openstack.org/417898 | 13:01 |
| panda | EmilienM: bnemec lates periodic jobs are all false negatives (except ha-mitaka). My patch from yesterday was not considering the set -u, please review this change https://review.openstack.org/421822 to fix promotion | 13:01 |
| *** sdake_ has quit IRC | 13:02 | |
| *** paramite has quit IRC | 13:02 | |
| bkero | sshnaidm, EmilienM, adarazs: I'm guessing this is because the quickstart.sh collect-logs.yml isn't using the --retain-inventory option to use the subnode-containing hosts file | 13:03 |
| *** social_ is now known as social | 13:03 | |
| bkero | adarazs: Can you confirm? ^ | 13:03 |
| *** social has quit IRC | 13:03 | |
| *** social has joined #tripleo | 13:03 | |
| adarazs | bkero: collect-logs.sh does use --retain-inventory: https://github.com/openstack/tripleo-quickstart/blob/master/ci-scripts/collect-logs.sh#L29 | 13:04 |
| bkero | adarazs: http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_instack_oooq_multinode.sh#n155 | 13:04 |
| sshnaidm | bkero, nope, it uses --retain-inventory | 13:05 |
| bkero | I cribbed that from the ovb toci runner. I could explore just using the quickstart collect-logs.sh script | 13:05 |
| *** salmankhan has quit IRC | 13:05 | |
| sshnaidm | bkero, --retain-inventory is part of $OOOQ_DEFAULT_ARGS | 13:05 |
| adarazs | bkero: oh yeah, there should be a retain inventory there around line 155, and we should remove the extras requirement file too | 13:05 |
| *** morazi has joined #tripleo | 13:05 | |
| sshnaidm | adarazs, ^^ | 13:06 |
| adarazs | sshnaidm: I see. | 13:06 |
| *** udesale has joined #tripleo | 13:06 | |
| *** trown|outtypewww is now known as trown | 13:07 | |
| rook | myoung: hey, are there undercloud images you build? jkilpatr maybe you know? | 13:08 |
| bkero | sshnaidm: Hm, you're right | 13:08 |
| owalsh | EmilienM: hi, I'm thinking where to run cell_v2 discover_hosts, must be *after* compute nodes are deployed | 13:08 |
| EmilienM | owalsh: in a mtg now | 13:09 |
| bkero | adarazs: can you think of why the oooq logs here would be missing? http://logs.openstack.org/68/418468/5/experimental/gate-tripleo-ci-centos-7-nonha-multinode-oooq-nv/1fdfce6/logs/subnode-2/var/log/ | 13:09 |
| *** sdake__ is now known as sdake | 13:09 | |
| sshnaidm | trown, EmilienM I suggest to remove ovb-nonha job from oooq check gate to reduce load on ovb cloud: https://review.openstack.org/#/c/421743/ the multinode job is pretty close to it | 13:09 |
| jkilpatr | rook what do you need to know about images? That's more the quickstart team but TripleO is using that for testing now AFAIK | 13:09 |
| adarazs | bkero: I'll look through the job | 13:10 |
| rook | jkilpatr is there a qs irc? | 13:10 |
| rook | jkilpatr anyway, I wanted to know if there is a undercloud image prebuilt I can use with ovb | 13:10 |
| owalsh | EmilienM: ok, no rush | 13:11 |
| *** lhinds|afk has quit IRC | 13:11 | |
| jkilpatr | rook, go to #oooq | 13:11 |
| *** rnoriega has quit IRC | 13:12 | |
| *** myoung has quit IRC | 13:12 | |
| *** jschlueter has quit IRC | 13:12 | |
| *** chem has quit IRC | 13:12 | |
| *** jschlueter has joined #tripleo | 13:12 | |
| *** myoung has joined #tripleo | 13:12 | |
| bkero | Coincidentally, any tripleo-ci cores semi-familiar with oooq who can approve https://review.openstack.org/#/c/420912/ ? | 13:12 |
| adarazs | bkero: looks like the job timed out right in the middle of the job collection: http://logs.openstack.org/68/418468/5/experimental/gate-tripleo-ci-centos-7-nonha-multinode-oooq-nv/1fdfce6/console.html#_2017-01-18_00_01_13_643257 | 13:12 |
| *** rbrady is now known as rbrady-mtg | 13:12 | |
| bkero | EmilienM: ^ | 13:13 |
| *** rnoriega has joined #tripleo | 13:13 | |
| adarazs | bkero: sorry, disregard that, I suck at reading the upstream logs :) | 13:14 |
| *** morazi has quit IRC | 13:14 | |
| adarazs | still looking for the reason | 13:14 |
| EmilienM | adarazs: they are very important and we need absolutely need them. | 13:14 |
| adarazs | EmilienM: I agree. :) | 13:15 |
| *** ansmith has joined #tripleo | 13:15 | |
| adarazs | trying to find the reason we're missing them, but I have to follow up the way the log collection is ran. | 13:15 |
| *** lucasagomes is now known as lucas-hungry | 13:15 | |
| sshnaidm | adarazs, bkero the overcloud nodes are not in inventory, collect-logs just doesn't know about them | 13:15 |
| trown | rook: there is no tripleo produced undercloud, but there is one in RDO | 13:16 |
| openstackgerrit | Ryan Brady proposed openstack/tripleo-common: Adds support to create deployment plan from git repository https://review.openstack.org/417280 | 13:16 |
| bkero | sshnaidm: they are now | 13:16 |
| rook | trown: ah ha, cool | 13:16 |
| trown | rook: the RDO produced images are at http://buildlogs.centos.org/centos/7/cloud/x86_64/tripleo_images/ | 13:16 |
| panda | jpich: https://review.openstack.org/421891 | 13:16 |
| bkero | sshnaidm: http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/quickstart/multinode-playbook.yml#n38 | 13:16 |
| sshnaidm | bkero, ok, but not in this patch | 13:16 |
| bkero | sshnaidm: The review that added that task was added a few hours ago, so perhaps it needs to be rerun to get them. | 13:17 |
| *** karimb has quit IRC | 13:17 | |
| sshnaidm | bkero, when was it landed? | 13:17 |
| openstackgerrit | Merged openstack/puppet-tripleo: Do not depend on bootstrap_nodeid for any pacemaker profile https://review.openstack.org/421405 | 13:17 |
| sshnaidm | bkero, yeah, maybe | 13:17 |
| bkero | sshnaidm: https://review.openstack.org/#/c/420795/ | 13:17 |
| *** lhinds has joined #tripleo | 13:17 | |
| sshnaidm | bkero, let's look at the next jobs then | 13:18 |
| bkero | EmilienM: let's hope that this review ^ fixed it | 13:18 |
| bkero | adarazs: does the tripleo-inventory role write out the generated inventory, or is it in-memory for the single run? | 13:19 |
| adarazs | bkero: it writes it out. | 13:19 |
| *** ccamacho has quit IRC | 13:19 | |
| bkero | Ok, cool. So it should work. | 13:19 |
| adarazs | bkero: https://github.com/openstack/tripleo-quickstart/blob/master/roles/tripleo-inventory/tasks/main.yml#L82 | 13:19 |
| adarazs | this is where the magic happens :) | 13:20 |
| bkero | Cool, so it should be fixed already! | 13:20 |
| adarazs | okay, let me know if it still doesn't work | 13:20 |
| *** rhallisey has joined #tripleo | 13:21 | |
| *** shardy_lunch is now known as shardy | 13:21 | |
| bkero | EmilienM: can we retrigger the job associated with the log you posted? | 13:22 |
| EmilienM | like I said, i'm in a meeting now | 13:22 |
| bkero | okay | 13:22 |
| bkero | I'll wait then, no worries. | 13:23 |
| bkero | Sorry for interrupting | 13:23 |
| *** beagles is now known as beagles_brb | 13:24 | |
| sshnaidm | bkero, it was job running on my ovb patch, so I don't think it need to be rerun | 13:24 |
| bkero | sshnaidm: I'm attempting on a different job just to make sure | 13:25 |
| bkero | (a multinode-oooq job) | 13:25 |
| sshnaidm | bkero, ok | 13:25 |
| jpich | panda: Awesome, thank you!! | 13:26 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: DO NOT MERGE - testing TLS everywhere job https://review.openstack.org/419529 | 13:27 |
| *** morazi has joined #tripleo | 13:27 | |
| bkero | adarazs: can follow along if interested: telnet://146.20.103.25:19885 | 13:27 |
| *** abregman|afk is now known as abregman | 13:28 | |
| openstackgerrit | Chris Jones proposed openstack/puppet-pacemaker: Add support for fence_ironic fencing agent. https://review.openstack.org/390396 | 13:30 |
| *** rlandy has joined #tripleo | 13:31 | |
| *** ccamacho has joined #tripleo | 13:37 | |
| *** pradk has joined #tripleo | 13:39 | |
| *** salmankhan has joined #tripleo | 13:40 | |
| *** liverpooler has joined #tripleo | 13:41 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Add a environment file to disable ceilo api https://review.openstack.org/420087 | 13:45 |
| panda | jpich: thanks for the update, I didn't find "Fix proposed" in LP and I thought FIx committed was the closest. Still don't know why it's not happening in automatic | 13:45 |
| *** akrivoka has quit IRC | 13:45 | |
| jpich | panda: No problems. I think that's because the fix isn't in one of the tripleo repositories but in an 'infra' repo that isn't connected to the 'tripleo' launchpad, which is fair enough I suppose | 13:46 |
| *** jpena is now known as jpena|lunch | 13:47 | |
| *** ebarrera_ has joined #tripleo | 13:47 | |
| *** akrivoka has joined #tripleo | 13:47 | |
| *** dprince has quit IRC | 13:52 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/puppet-tripleo: Remove legacy flag and use composable interface https://review.openstack.org/420081 | 13:53 |
| *** liverpooler has quit IRC | 13:54 | |
| *** liverpooler has joined #tripleo | 13:54 | |
| *** trown is now known as trown|afk | 13:54 | |
| *** prateek has quit IRC | 13:55 | |
| *** rbrady-mtg is now known as rbrady | 13:56 | |
| *** jerrygb has joined #tripleo | 13:56 | |
| openstackgerrit | Ryan Brady proposed openstack/tripleo-common: Adding a deprecation policy https://review.openstack.org/421916 | 13:56 |
| *** amoralej is now known as amoralej|lunch | 13:59 | |
| *** jerrygb has quit IRC | 13:59 | |
| shardy | owalsh: Hey, I'm trying to debug why nova-manage db online_data_migrations is hanging during upgrade, any ideas on the best place to look for clues? | 14:02 |
| shardy | https://review.openstack.org/#/c/405241/10/puppet/services/nova-api.yaml | 14:02 |
| shardy | we run it in step2 there, but AFAICS it hangs forever until the upgrade job times out | 14:02 |
| shardy | and I'm seeing the same locally | 14:02 |
| * shardy shakes fist at nova for being so hard to upgrade ;) | 14:02 | |
| * sshnaidm deployed a frankenstein job: ha with net isolation, ceph, introspection and undercloud ssl and all this with quickstart | 14:03 | |
| *** hjensas has quit IRC | 14:03 | |
| sshnaidm | weshay, trown|afk ^^ maybe we'll use this job for gate | 14:03 |
| sshnaidm | s/gate/check/ | 14:03 |
| *** jerrygb has joined #tripleo | 14:03 | |
| *** jerrygb has quit IRC | 14:03 | |
| weshay | sshnaidm, I like the idea.. | 14:03 |
| *** jerrygb has joined #tripleo | 14:04 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Check that all templates are using the release alias https://review.openstack.org/421326 | 14:04 |
| shardy | owalsh: Oh wait, I think we're doing something dumb with the steps, as the vip for haproxy is disabled for step2 | 14:04 |
| *** liverpooler has quit IRC | 14:04 | |
| shardy | sorry for the noise, /me tries juggling steps | 14:04 |
| *** fultonj has joined #tripleo | 14:04 | |
| *** karimb has joined #tripleo | 14:05 | |
| *** liverpooler has joined #tripleo | 14:05 | |
| shardy | the question is do we run that migration before or after the package upgrade? | 14:06 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras: remove docker-network-isolation heat parameters https://review.openstack.org/421630 | 14:06 |
| shardy | I assume after, along with all the cells stuff etc? | 14:06 |
| shardy | owalsh, matbu: ^^ | 14:06 |
| *** athomas has quit IRC | 14:06 | |
| EmilienM | flaper87: I have a bad news about Glance | 14:07 |
| EmilienM | I found a critical problem when we remove Glance v2 API in TripleO | 14:07 |
| EmilienM | but it might be a configuration thing | 14:07 |
| EmilienM | http://logs.openstack.org/14/417214/7/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/90283ac/logs/undercloud/var/log/nova/nova-compute.txt.gz#_2017-01-18_06_12_05_575 | 14:07 |
| owalsh | shardy: after the package upgrade & sync db | 14:07 |
| EmilienM | flaper87: it seems like ironic tries to reach v1 endpoint | 14:07 |
| owalsh | ansiwen: correct ^^ | 14:07 |
| shardy | owalsh: Ok, thanks, let me try moving it | 14:07 |
| *** abehl has quit IRC | 14:07 | |
| EmilienM | dtantsur, lucas-hungry: do you know how to tell ironic to reach glance v2? | 14:07 |
| ansiwen | owalsh: question? | 14:08 |
| owalsh | ansiwen: do we run the online migrations before or after the package upgrade - it's after, correct? | 14:08 |
| dtantsur | EmilienM, not from the top of my head, I can look at the code after lunch | 14:09 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Add a environment file to disable ceilo api https://review.openstack.org/420087 | 14:09 |
| ansiwen | owalsh: the online migration must be run after the sync according to dan smith. if you don't the next upgrade will fail later | 14:09 |
| EmilienM | dtantsur: looking at the trace it might be in nova | 14:09 |
| shardy | Ok so we're doing it much too early | 14:10 |
| shardy | thanks both, let me try again with it as the last nova_api task in step 5 | 14:10 |
| *** owalsh is now known as owalsh-brb | 14:10 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Fix current bugs for oooq job https://review.openstack.org/418468 | 14:10 |
| *** jcoufal has joined #tripleo | 14:10 | |
| *** pradk has quit IRC | 14:12 | |
| *** dmacpher has joined #tripleo | 14:14 | |
| *** Goneri has joined #tripleo | 14:14 | |
| *** udesale has quit IRC | 14:15 | |
| *** athomas has joined #tripleo | 14:15 | |
| *** owalsh-brb has quit IRC | 14:15 | |
| *** owalsh-brb has joined #tripleo | 14:16 | |
| *** owalsh-brb is now known as owalsh | 14:16 | |
| *** udesale has joined #tripleo | 14:17 | |
| *** udesale has quit IRC | 14:18 | |
| *** beagles_brb is now known as beagles | 14:18 | |
| *** udesale has joined #tripleo | 14:19 | |
| *** udesale has quit IRC | 14:21 | |
| *** udesale has joined #tripleo | 14:21 | |
| *** jmelvin has joined #tripleo | 14:22 | |
| *** trown|afk is now known as trown | 14:22 | |
| EmilienM | dtantsur: fyi, I think I broke ironic on the overcloud. We removed Glance api v1 on the overcloud, it passed CI but it's because we don't test ironic on overcloud | 14:23 |
| dtantsur | ugh :( | 14:24 |
| dtantsur | EmilienM, do you have a bug report for it? | 14:24 |
| EmilienM | dtantsur: not yet | 14:25 |
| trown | sshnaidm: ya I like trying to cover everything we can't cover via a multinode job in a single ovb job (if it is possible of course) | 14:25 |
| EmilienM | but I think it might be a configuration thing | 14:25 |
| *** ccamacho has quit IRC | 14:25 | |
| EmilienM | dtantsur: because AFIK nova/ironic supports Glance v2 | 14:25 |
| EmilienM | but it seems like v1 is default | 14:25 |
| *** lucas-hungry is now known as lucasagomes | 14:25 | |
| EmilienM | I would need you and lucasagomes to help me though | 14:25 |
| EmilienM | see http://logs.openstack.org/14/417214/7/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/90283ac/logs/undercloud/var/log/ironic/ironic-conductor.txt.gz#_2017-01-18_06_08_55_354 | 14:25 |
| * lucasagomes reads | 14:25 | |
| lucasagomes | EmilienM, hi there hmm lemme check | 14:25 |
| EmilienM | lucasagomes, dtantsur : let me file a bug so it's clear for you | 14:25 |
| lucasagomes | I believe we might have a glance_url or similar option which you can specify the domain /v2 /v3 idk | 14:26 |
| * lucasagomes checks | 14:26 | |
| *** abregman is now known as abregman|mtg | 14:28 | |
| EmilienM | dtantsur, lucasagomes: https://bugs.launchpad.net/tripleo/+bug/1657475 | 14:28 |
| openstack | Launchpad bug 1657475 in tripleo "Impossible to spawn overcloud without Glance v1 API" [High,Triaged] - Assigned to Emilien Macchi (emilienm) | 14:28 |
| EmilienM | flaper87: ^ FYI | 14:29 |
| *** dprince has joined #tripleo | 14:29 | |
| skramaja | EmilienM: i want to give another try on https://review.openstack.org/#/c/402061/. rabbitmq_user resource should be invoked only when rabbitmq is running. puppet-pacemaker is responsible to start puppet-rabbitmq. so rabbitmq_user resource cannot be invoked inside puppet-rabbitmq for TripleO usecase. I am not able to find any other way to fix this use-case!! | 14:29 |
| EmilienM | dtantsur, lucasagomes: do you use glance api v1 in ironic gate? | 14:30 |
| * EmilienM checks in devstack | 14:30 | |
| ansiwen | shardy: the endpoint urls of ec2api are still broken in CI, but they are not in my local deployment. but now it's not failing anymore, did you remove the uri validation? http://logs.openstack.org/34/398634/20/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/0370e4e/logs/postci.txt.gz | 14:30 |
| skramaja | EmilienM: is it ok, if we have this fix in puppet-tripleo for now, until we have another way out? | 14:30 |
| *** shardy has quit IRC | 14:30 | |
| *** shardy has joined #tripleo | 14:31 | |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/puppet-tripleo: updates to collectd support https://review.openstack.org/417509 | 14:31 |
| *** hjensas has joined #tripleo | 14:31 | |
| *** hjensas has quit IRC | 14:31 | |
| *** hjensas has joined #tripleo | 14:31 | |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart-extras: Integrate undercloud_user var into oooq-extras https://review.openstack.org/417503 | 14:32 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart-extras: Ensure that each role in extras has a dependency on extras-common https://review.openstack.org/413800 | 14:32 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart-extras: Remove hardcoded instances of `stack` from roles https://review.openstack.org/421463 | 14:32 |
| lucasagomes | EmilienM, I thought we were using v2 there, checking | 14:36 |
| EmilienM | lucasagomes: switching on ironic channel | 14:36 |
| sshnaidm | EmilienM, trown, weshay - to relax ovb a little: https://review.openstack.org/#/c/421743/ | 14:36 |
| *** sshnaidm is now known as sshnaidm|afk | 14:36 | |
| lucasagomes | ack | 14:36 |
| *** karimb has quit IRC | 14:37 | |
| openstackgerrit | Attila Darazs proposed openstack/tripleo-quickstart-extras: collect-logs: rename text files to txt.gz https://review.openstack.org/419039 | 14:39 |
| *** yolanda has quit IRC | 14:40 | |
| *** yolanda has joined #tripleo | 14:41 | |
| *** skramaja has quit IRC | 14:41 | |
| *** chem has joined #tripleo | 14:42 | |
| *** ccamacho has joined #tripleo | 14:42 | |
| dtrainor | therve, timezone interference indeed :) i'm working on an on-list reply for more visibility | 14:43 |
| therve | OK cool | 14:43 |
| *** sshnaidm|afk is now known as sshnaidm|bbl | 14:49 | |
| *** jpena|lunch is now known as jpena | 14:50 | |
| *** jeckersb_gone is now known as jeckersb | 14:51 | |
| *** morazi has quit IRC | 14:51 | |
| *** lblanchard has joined #tripleo | 14:52 | |
| honza | EmilienM: I have some specs up for ocata that have -2s on them. I'm working on updating them for Pike. Should I continue in the original patch, or abandon and start over? | 14:52 |
| EmilienM | honza: update it to be for pike? | 14:52 |
| honza | EmilienM: yes, sorry | 14:52 |
| jtomasek | honza, florianf: do you see the roles icons in master GUI in Chrome? | 14:54 |
| honza | jtomasek: checking | 14:55 |
| therve | dtrainor, I may not be around for too long, so if you have info I'd be glad to have them :) | 14:55 |
| *** links has joined #tripleo | 14:56 | |
| *** links has quit IRC | 14:56 | |
| dtrainor | just got off a call - sorry for the delay. based on what i've read in my searches i've seen that it's very possible to have a websocket tunnel created with apache, but difficult to have a websocket tunnel in addition to other normal http requests on the same host | 14:58 |
| dtrainor | my response to the list is going to be something to that effect | 14:58 |
| florianf | jtomasek: pen and plus-sign? I can see them in chrome in the latest master. | 14:58 |
| EmilienM | dtantsur: backup plan - if we don't get this solved on time, I'll revert my patches that removed glance v1 api on overcloud. So we don't regress | 14:59 |
| jtomasek | florianf: ook, something is wrong with my chrome | 14:59 |
| * jtomasek wants icons back | 14:59 | |
| EmilienM | dtantsur: I guess you're fine with this backup plan | 14:59 |
| dtrainor | i don't know if it was entirely clear but the simple example that I gave describing how i'm attempting to handle the websocket lives in tandem with other ProxyPass/ProxyPassReverse directives that handle other regular http connections | 14:59 |
| honza | jtomasek: yes, i see them, too florianf | 14:59 |
| dtantsur | EmilienM, yep, sounds fine | 14:59 |
| EmilienM | k | 14:59 |
| jtomasek | honza, florianf: thanks for confirmation | 14:59 |
| openstackgerrit | Merged openstack/tripleo-ui: Disallow underscores in plan names https://review.openstack.org/420599 | 14:59 |
| jtomasek | jpich: ^ seems like we're merging now:) | 15:00 |
| jpich | jtomasek: This is Newton | 15:00 |
| jtomasek | jpich: aah | 15:01 |
| jpich | jtomasek: Sorry!! | 15:01 |
| jtomasek | lol | 15:01 |
| *** nmathew has joined #tripleo | 15:02 | |
| dtrainor | therve, by "no data after that" I mean, I see zero bytes in transmission between my client (the browser) and the zaqar websocket transport | 15:02 |
| therve | dtrainor, Sure, but should there be data? | 15:02 |
| therve | The client is waiting for notifications, but there may not be any | 15:03 |
| *** trown is now known as trown|mtg | 15:03 | |
| dtrainor | there should be data. there is data when i do not use this configuration and do not try to proxy websocket connections like this | 15:03 |
| therve | OK | 15:03 |
| therve | It's weird because I don't see why the initial requests would be different | 15:04 |
| dtrainor | context: the data shuold come from the result of validations that automatically get run when a user logs in to the ui (initiated by the client (browser)). | 15:04 |
| dtrainor | hm, that raises an interesting point, i don't know if the validations are "asked' to run, i don't have data showing that | 15:05 |
| *** jistr is now known as jistr|mtg | 15:05 | |
| dtrainor | i'll look in to that | 15:05 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Revert "Revert "Specify cell0 db creation"" https://review.openstack.org/421999 | 15:05 |
| therve | Well they are not asked on the same connections | 15:05 |
| dtrainor | are you saying that as a matter of fact based on the data and logs i showed, or a function of how websockets work? | 15:06 |
| dprince | EmilienM, mwhahaha: I would like to still create the database. Not creating it breaks my docker environment for the Nova services (we are using more recent packages and manually drive the cells setup) | 15:06 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/puppet-tripleo: updates to collectd support https://review.openstack.org/417509 | 15:06 |
| dprince | EmilienM, mwhahaha: I get there are upgrade issues, but creating an empty database doesn't hurt anything and isn't the part that was broken. | 15:06 |
| therve | dtrainor, I mean, that's how the system works. Workflows are run on mistral, and the results are pushed into Zaqar | 15:07 |
| therve | The UI subscribes to the queue, and get notified that way | 15:07 |
| mwhahaha | dprince: that's fine, we just needed to rip it out. if we need to add bits back in, feel free to propose | 15:07 |
| dprince | mwhahaha: It is here https://review.openstack.org/#/c/421999/ | 15:07 |
| dtrainor | right, i knew that, thanks for confirming (i'm not crazy) | 15:07 |
| mwhahaha | dprince: fine by me +1 | 15:08 |
| therve | dtrainor, I meant that there should be a different between those notifications, and responses to queue_create subscription_create | 15:08 |
| therve | They are messages from the server side in the same way | 15:09 |
| *** alee has quit IRC | 15:09 | |
| therve | Unless... Apache doesn't keep the connection opened? But I don't see how that would work | 15:09 |
| *** leanderthal has joined #tripleo | 15:11 | |
| leanderthal | o/ | 15:11 |
| dtrainor | ok, i have some ideas now. i'll take a closer look at timeouts and the process by which the client requests data | 15:12 |
| *** amoralej|lunch is now known as amoralej | 15:14 | |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: Remove Glance Registry from undercloud https://review.openstack.org/417214 | 15:14 |
| EmilienM | dtantsur: done ^ | 15:14 |
| EmilienM | dtantsur: let's see how it works | 15:15 |
| *** openstack has joined #tripleo | 15:18 | |
| *** StevenK_ has joined #tripleo | 15:18 | |
| *** thegodfather has joined #tripleo | 15:18 | |
| *** bkreitch has quit IRC | 15:18 | |
| *** slagle_ has joined #tripleo | 15:18 | |
| *** fabbione has quit IRC | 15:18 | |
| *** slagle has quit IRC | 15:18 | |
| *** liverpooler has quit IRC | 15:18 | |
| *** nyechiel_ has quit IRC | 15:18 | |
| *** pcaruana is now known as pablo|500| | 15:18 | |
| *** cmyster_ has joined #tripleo | 15:18 | |
| *** soc_off_ has joined #tripleo | 15:18 | |
| *** sudipto has quit IRC | 15:19 | |
| *** pkovar has quit IRC | 15:19 | |
| *** tosky has quit IRC | 15:19 | |
| *** arxcruz has quit IRC | 15:19 | |
| *** rbrady has quit IRC | 15:19 | |
| *** trown|mtg has quit IRC | 15:19 | |
| *** dsneddon has quit IRC | 15:19 | |
| *** cmyster has quit IRC | 15:19 | |
| *** CaptTofu has quit IRC | 15:19 | |
| *** mwhahaha has quit IRC | 15:19 | |
| *** jd__ has quit IRC | 15:19 | |
| *** radez has quit IRC | 15:19 | |
| *** soc_off has quit IRC | 15:19 | |
| *** therve has quit IRC | 15:19 | |
| *** honza has quit IRC | 15:19 | |
| *** mgagne has quit IRC | 15:19 | |
| *** mrunge has quit IRC | 15:19 | |
| *** jtomasek has quit IRC | 15:19 | |
| *** rhefner has quit IRC | 15:19 | |
| *** StevenK has quit IRC | 15:19 | |
| *** fungi has quit IRC | 15:19 | |
| *** jd__ has joined #tripleo | 15:19 | |
| *** therve has joined #tripleo | 15:19 | |
| *** rhefner has joined #tripleo | 15:19 | |
| *** rbrady has joined #tripleo | 15:20 | |
| *** rbrady has joined #tripleo | 15:20 | |
| *** mrunge has joined #tripleo | 15:20 | |
| *** fungi has joined #tripleo | 15:20 | |
| *** tzumainn has joined #tripleo | 15:20 | |
| *** alee has joined #tripleo | 15:22 | |
| gfidente | bandini you paid me back! | 15:22 |
| *** milan has quit IRC | 15:22 | |
| *** milan has joined #tripleo | 15:22 | |
| *** mgagne has joined #tripleo | 15:22 | |
| *** sudipto has joined #tripleo | 15:22 | |
| *** pkovar has joined #tripleo | 15:22 | |
| *** arxcruz has joined #tripleo | 15:22 | |
| *** trown|mtg has joined #tripleo | 15:22 | |
| *** dsneddon has joined #tripleo | 15:22 | |
| *** radez has joined #tripleo | 15:22 | |
| *** honza has joined #tripleo | 15:22 | |
| *** jtomasek has joined #tripleo | 15:22 | |
| *** milan has quit IRC | 15:22 | |
| *** milan has joined #tripleo | 15:22 | |
| *** [1]cdearborn has joined #tripleo | 15:22 | |
| *** mgagne has quit IRC | 15:23 | |
| *** bkreitch has joined #tripleo | 15:23 | |
| *** mgagne has joined #tripleo | 15:23 | |
| bandini | gfidente: :) | 15:23 |
| *** mgagne is now known as Guest58531 | 15:23 | |
| *** mwhahaha has joined #tripleo | 15:27 | |
| dtrainor | thanks for the ideas therve | 15:27 |
| *** nmathew has quit IRC | 15:27 | |
| therve | dtrainor, Not sure I provided much, but sure :) | 15:29 |
| therve | dtrainor, I'd be nice to test your setup without the UI | 15:30 |
| *** CaptTofu has joined #tripleo | 15:30 | |
| dtrainor | well, you gave me some thinking points | 15:30 |
| therve | To see if it's not browser issue | 15:30 |
| openstackgerrit | Alexander Chuzhoy proposed openstack/tripleo-docs: adding a note how to reconfigure overcloud to erase only the metadata, instead of the entire disk. https://review.openstack.org/422022 | 15:30 |
| openstackgerrit | Pradeep Kilambi proposed openstack/puppet-tripleo: Remove legacy flag and use composable interface https://review.openstack.org/420081 | 15:30 |
| dtrainor | i can provide that - we had talked about that last week, didn't we. if you want to do that tomorrow we can certainly do that, i don't want to keep you any later than you already are this evening | 15:30 |
| openstackgerrit | Giulio Fidente proposed openstack/puppet-tripleo: Add Ceph RBD mirror Pacemaker profile https://review.openstack.org/407038 | 15:31 |
| therve | dtrainor, If you can leave something running, I can access it at some point :) | 15:32 |
| dtrainor | i'll do that this evening before i walk away, given the timezone difference | 15:33 |
| dtrainor | thank you | 15:33 |
| *** bswartz has joined #tripleo | 15:33 | |
| *** dmarlin has joined #tripleo | 15:33 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-ui: DO NOT MERGE, used in CI ONLY https://review.openstack.org/422025 | 15:34 |
| *** saneax is now known as saneax-_-|AFK | 15:36 | |
| *** egafford has quit IRC | 15:36 | |
| *** limao has joined #tripleo | 15:36 | |
| *** limao_ has joined #tripleo | 15:39 | |
| *** pblaho has quit IRC | 15:39 | |
| *** pblaho1 has joined #tripleo | 15:39 | |
| *** limao has quit IRC | 15:40 | |
| *** jaosorior has joined #tripleo | 15:41 | |
| *** dmarlin is now known as dmarlin_ | 15:42 | |
| *** sanagikoki has joined #tripleo | 15:42 | |
| openstackgerrit | Alexander Chuzhoy proposed openstack/tripleo-docs: attempting to enhance formatting. https://review.openstack.org/422033 | 15:42 |
| jaosorior | bnemec: hey dude, I attempted to extract the logs in a similar fashion as in the other nodes, however, there seems to be some issue http://logs.openstack.org/29/419529/20/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/5e89f46/logs/postci.txt.gz#_2017-01-18_14_40_25_000 | 15:44 |
| jaosorior | bnemec: this is what I'm doing https://review.openstack.org/#/c/419529/20/scripts/common_functions.sh | 15:44 |
| lhinds | jaosorior: I think this has landed now? https://bugs.launchpad.net/tripleo/+bug/1640491 | 15:44 |
| openstack | Launchpad bug 1640491 in tripleo "Enable SESSION_COOKIE_SECURE & CSRF_COOKIE_SECURE" [Medium,Triaged] | 15:44 |
| jaosorior | lhinds: it has AFAIK | 15:45 |
| jaosorior | derekh: as an update from yesterday, it seems that the overcloud nodes can't even ping that extra node I added | 15:45 |
| jaosorior | so maybe it wasn't the MTU | 15:45 |
| *** david-lyle has joined #tripleo | 15:47 | |
| derekh | jaosorior: hmmm, can the undercloud node talk to the extra node? i.e. do we know its running at all | 15:48 |
| derekh | brb | 15:48 |
| *** morazi has joined #tripleo | 15:49 | |
| jpich | lhinds: If you include a link to the patch(es) in a comment we/I can update the status | 15:49 |
| *** sanagikoki has quit IRC | 15:50 | |
| jaosorior | derekh: it's running | 15:50 |
| jaosorior | derekh: the undercloud node can talk, and actually even enroll to the CA, but that can be done via the private IP | 15:50 |
| *** sanagikoki has joined #tripleo | 15:50 | |
| jaosorior | derekh: somehow when I try to use the IP in the provisioning address it times out | 15:50 |
| derekh | jaosorior: can you point me at the review again | 15:51 |
| *** jistr|mtg is now known as jistr | 15:51 | |
| *** thrash is now known as thrash|biab | 15:51 | |
| jaosorior | derekh: this is the review https://review.openstack.org/#/c/391738/99/scripts/deploy.sh | 15:52 |
| jaosorior | derekh: this is how I set the IP on the host https://review.openstack.org/#/c/420882/ | 15:52 |
| jaosorior | derekh: and this is how I'm currently testing it https://review.openstack.org/#/c/419529/20 tried doing the get_host_info stuff there but somehow it's not working :/ | 15:53 |
| *** udesale has quit IRC | 15:53 | |
| *** pblaho1 has quit IRC | 15:53 | |
| openstackgerrit | Michele Baldessari proposed openstack/puppet-tripleo: Composable HA https://review.openstack.org/362389 | 15:54 |
| bnemec | jaosorior: 2017-01-18 14:40:27.000 | sudo: sorry, you must have a tty to run sudo | 15:54 |
| jaosorior | bnemec: oh, I guess I need the -tt option there as wel | 15:54 |
| jaosorior | *as well | 15:54 |
| openstackgerrit | Alexander Chuzhoy proposed openstack/tripleo-docs: attempting to enhance formatting. https://review.openstack.org/422048 | 15:54 |
| bnemec | Yep, looks like it. | 15:55 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: DO NOT MERGE - testing TLS everywhere job https://review.openstack.org/419529 | 15:56 |
| jaosorior | another try | 15:56 |
| bnemec | https://review.openstack.org/#/c/413248/ is ready to go | 15:56 |
| derekh | jaosorior: ok | 15:57 |
| openstackgerrit | Alexander Chuzhoy proposed openstack/tripleo-docs: attempting to enhance formatting. https://review.openstack.org/422033 | 15:57 |
| bandini | dprince: any thoughts on https://bugs.launchpad.net/tripleo/+bug/1657108 (see comment #2 for the concise problem explanation) ? | 15:57 |
| openstack | Launchpad bug 1657108 in tripleo "tripleo::firewall does not work as intended if the image has prepopulated firewall rules" [High,Triaged] | 15:57 |
| *** trown|mtg is now known as trown | 15:57 | |
| jaosorior | derekh: anyway, but that's just to do the get_host_info.sh (the stuff I updated) if you have any theories on why I can't reach that node via that IP that I'm assigning, would be appreciated | 15:57 |
| *** ealcaniz has quit IRC | 15:57 | |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: Remove Glance Registry from undercloud https://review.openstack.org/417214 | 15:58 |
| derekh | jaosorior: ok, looking | 15:58 |
| *** abregman|mtg has quit IRC | 15:58 | |
| bnemec | https://review.openstack.org/#/c/419048 is also ready to go. gfidente you might be interested in that one since it's a pre-req to the storage-mgmt patch | 15:58 |
| *** hjensas has quit IRC | 15:59 | |
| *** rajinir has joined #tripleo | 16:00 | |
| dprince | bandini: would simply configuring all the firewall ports earlier in the deployment resolve the issues? | 16:01 |
| dprince | bandini: if not, the pre-rules section you mention could work too I suppose | 16:01 |
| *** abregman has joined #tripleo | 16:01 | |
| bandini | dprince: I believe so yes, as long as it happens before the cluster setup | 16:02 |
| gfidente | bnemec ack indeed | 16:02 |
| gfidente | bnemec green and automated so looks like easy +2 | 16:02 |
| bandini | dprince: how about instead of Service<||> -> Class['tripleo::firewall::post'], we do Class['tripleo::firewall::post'] -> Service<||> ? | 16:02 |
| bandini | dprince: would anything speak against that? | 16:03 |
| bnemec | gfidente: Yeah, it's got full ci coverage and it's just shuffling around the yaml keys. | 16:03 |
| shardy | marios: Hey wanted to chat again about the idea of puppet starting the services on upgrade instead on ansible | 16:03 |
| shardy | marios: I keep hitting exciting corner cases where backwards incompatible configuration changes mean workarounds in ansible, even though puppet already knows how to configure the newly upgraded services correctly and start them | 16:03 |
| *** rhallisey has quit IRC | 16:03 | |
| pradk | can i please get some reviews on https://review.openstack.org/#/c/418475/ | 16:04 |
| marios | shardy: so would we just run the ansible up to the yum_update tasks then? | 16:04 |
| marios | shardy: i mean, and let puppet take over? | 16:04 |
| shardy | marios: Yeah I think so, pretty much | 16:04 |
| marios | shardy: i also wanted to ask you something puppet related by i'll not confuse two things lets talk about this first | 16:04 |
| openstackgerrit | Alexander Chuzhoy proposed openstack/tripleo-docs: adding a note how to reconfigure overcloud to erase only the metadata, instead of the entire disk. https://review.openstack.org/422033 | 16:04 |
| shardy | marios: we could still do any migration related things not handled by puppet, but I think it's going to be simpler | 16:04 |
| marios | shardy: so if we could get that to work then it would also possibly help with the time concerns (upgrade steps + puppet config) | 16:05 |
| shardy | marios: FYI the latest issue is nova broke backwards compatibility here https://review.openstack.org/#/q/e8436283e45b6716fb61d6f6590fadb5fb4ba45c,n,z | 16:05 |
| shardy | so we'd have to mangle the nova.conf or the services won't start up again | 16:05 |
| jaosorior | pradk: for panko, wouldn't we need the keystone endpoints and service user as well? | 16:06 |
| marios | shardy: seems to be a bit of nova related pain (cells api, online_data_migrations, placement api) | 16:06 |
| shardy | marios: Yeah I don't think it's going to take much longer, and things will be in a more verified consistent state when the services all come back up | 16:06 |
| marios | shardy: hadn't seen that one yet so thanks for that | 16:06 |
| shardy | marios: yeah, all the nova stuff has been painful | 16:06 |
| shardy | marios: so it'd be cool if we could handle that pain all in one place (e.g the puppet layer) | 16:06 |
| marios | shardy: so what about things like db migration after the yum update | 16:07 |
| marios | shardy: could we do ansible to a point, puppet run, then ansible rest of steps? | 16:07 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates: Add Octavia API service definitions https://review.openstack.org/411872 | 16:07 |
| shardy | marios: I'm easy - if puppet doesn't handle those we can still do the DB stuff | 16:07 |
| shardy | marios: I just don't really want to reimplement all the puppet configuration file management in ansible | 16:07 |
| openstackgerrit | Alexander Chuzhoy proposed openstack/tripleo-docs: adding a note how to reconfigure overcloud to erase only the metadata, instead of the entire disk. https://review.openstack.org/422033 | 16:07 |
| *** pblaho1 has joined #tripleo | 16:08 | |
| marios | shardy: yeah well i'm all for it if anyway puppet will run (and do that stuff) regardless of whether we get ansible to do it, too | 16:08 |
| shardy | marios: what ansible steps need to run after puppet, aren't we done when the services come back up? | 16:08 |
| marios | shardy: didn't have a specific example... i thought we were doing the db migrations after the yum update for example | 16:08 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: add a tq gate check workflow for build-test-packages https://review.openstack.org/422065 | 16:08 |
| openstackgerrit | Alexander Chuzhoy proposed openstack/tripleo-docs: adding a note how to reconfigure overcloud to erase only the metadata, instead of the entire disk. https://review.openstack.org/422033 | 16:09 |
| shardy | marios: Yeah if we want to do the DB stuff in ansible we'll still have to start the DB in step 4 | 16:09 |
| shardy | marios: the main thing I want to avoid is all the openstack workarounds in step5 before we try to start the openstack services | 16:09 |
| shardy | in particular all this annoying pain around nova ;) | 16:09 |
| pradk | jaosorior, do we need that if we're using auth token? thought those are deprecated | 16:09 |
| openstackgerrit | Alexander Chuzhoy proposed openstack/tripleo-docs: adding a note how to reconfigure overcloud to erase only the metadata, instead of the entire disk. https://review.openstack.org/422033 | 16:10 |
| shardy | marios: Ok if you think it's sane I'll give it a go and push some patches | 16:10 |
| shardy | marios: also, I'll look at how to stop both ansible and puppet on the nodes where upgrades are disabled, e.g compute | 16:10 |
| shardy | as we don't want new t-h-t running puppet on the computes with old modules | 16:10 |
| marios | shardy: you're referring to https://review.openstack.org/#/c/405241/10/puppet/services/nova-api.yaml | 16:10 |
| pradk | jaosorior, or are you referring to some other class here | 16:10 |
| jaosorior | pradk: those are not keystone middleware, those set the keystone endpoint and the service user | 16:11 |
| jaosorior | let me verify | 16:11 |
| jaosorior | maybe I messed up the class | 16:11 |
| marios | shardy: right this was the thing i wanted to catch up about | 16:11 |
| pradk | jaosorior, lemme check too | 16:11 |
| marios | shardy: since we're not upgrading them, we should disable the puppet run on compute/ wherever we have disable_upgrade in roles_data.yaml | 16:11 |
| shardy | marios: Yeah, when puppet-nova gets cells support again, we can just remove all those cells workarounds | 16:11 |
| jaosorior | pradk: it is indeed this one https://github.com/openstack/puppet-panko/blob/master/manifests/keystone/auth.pp | 16:11 |
| marios | shardy: but that then means we will have to have a convergein the end I think | 16:11 |
| shardy | marios: ack, yeah I think I know how to do that - we can use heat conditionals | 16:11 |
| pradk | jaosorior, ok i'll look into it now | 16:11 |
| marios | shardy: like composable-steps, manual non-controllers, converge | 16:12 |
| shardy | marios: yeah when not doing an all-in-one upgrade we'll need the converge, you're right | 16:12 |
| marios | shardy: unless we can do update && puppet run on compute into one step | 16:12 |
| marios | shardy: i.e. we've just 'fully' upgraded the controllers | 16:12 |
| marios | shardy: so lets also try and fully upgrade the rest of the nodes in one step if possible | 16:13 |
| shardy | marios: I think we can for the CI case, and other cases where you don't mind just running the ansible/puppet steps on computes | 16:13 |
| marios | shardy: i.e. yum update it/whatever else we normally do on computes, and then also puppet run for the new config. so its converged | 16:13 |
| shardy | marios: it's just for the case where compute upgrades are disabled (e.g the default for most "real" environments) we'll still need the converge to run puppet on the computes | 16:13 |
| shardy | and probably remove the flag which told heat not to run on those roles | 16:13 |
| marios | shardy: yes this is what my understanding is too... i.e. we run non-controllers manually to do the yumupdate, but then apply config (puppet-run) to everything at the end | 16:14 |
| marios | shardy: am just saying, would be nice if we could both yum update && puppet run on each of the non controllers. then we wouldn't need the converge | 16:14 |
| shardy | marios: yup, I think that is the same as before - the only difference is we now run puppet on the controllers before the non-controllers are upgraded | 16:15 |
| marios | shardy: but probably for now we'd just include a converge if it is the easier path | 16:15 |
| jaosorior | alee: added this for novajoin https://review.openstack.org/#/c/421575/ | 16:15 |
| marios | shardy: exactly, so we've already fully upgraded the controllers | 16:15 |
| openstackgerrit | Pradeep Kilambi proposed openstack/instack-undercloud: Add panko service to undercloud https://review.openstack.org/418475 | 16:15 |
| shardy | marios: Ok, let me try this, then we can discuss again if there's any optimisation possible for the non-controllers | 16:16 |
| matbu | mandre: shardy (reading a bit late) but this could save us for the "workload migration" case | 16:16 |
| shardy | marios: thanks for the sanity check :) | 16:16 |
| matbu | marios: ^ | 16:16 |
| marios | shardy: ok, thanks for the clarification on that then, for now we will aim for indeed disable puppet run for non-controllers during composable steps. then we will run gthe non controller scripts to upgrade as before and have a converge at the end to run puppet on them | 16:16 |
| jaosorior | pradk: looks great | 16:16 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-common: Add password and key generation for Octavia https://review.openstack.org/419536 | 16:16 |
| marios | shardy: thnks for taking the time man honestly we would never have gotten this far so quickly (relatively) without you taking the lead on this effort | 16:17 |
| marios | shardy: ++ | 16:17 |
| *** psahoo has joined #tripleo | 16:17 | |
| *** yamahata has joined #tripleo | 16:17 | |
| pradk | jaosorior, cool, thx for the review | 16:18 |
| *** jlinkes has quit IRC | 16:18 | |
| *** lblanchard has quit IRC | 16:18 | |
| openstackgerrit | John Eckersberg proposed openstack/tripleo-heat-templates: WIP: add configurable timeouts for DB sync https://review.openstack.org/416298 | 16:20 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart-extras: Add support for whole disk images https://review.openstack.org/421805 | 16:20 |
| *** thrash|biab is now known as thrash | 16:20 | |
| *** lblanchard has joined #tripleo | 16:21 | |
| *** weshay is now known as weshay_afk | 16:21 | |
| *** ebarrera_ has quit IRC | 16:25 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-quickstart-extras: Add options to set the overcloud's FQDNs and domain https://review.openstack.org/421589 | 16:26 |
| *** ebarrera has quit IRC | 16:27 | |
| jaosorior | shardy: hey, can you get back to this review https://review.openstack.org/#/c/411340/ ? :D | 16:28 |
| *** panda is now known as panda|bbl | 16:30 | |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: Add new feature to consume external images https://review.openstack.org/421763 | 16:33 |
| *** mhenkel has joined #tripleo | 16:34 | |
| openstackgerrit | Alexander Chuzhoy proposed openstack/tripleo-docs: adding a note how to reconfigure overcloud to erase only the metadata, instead of the entire disk. https://review.openstack.org/422033 | 16:34 |
| sanagikoki | Could someone review this ? It needs another +2. https://review.openstack.org/#/c/413778/ | 16:35 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates: Introduce Octavia implememtation services https://review.openstack.org/411874 | 16:37 |
| yolanda | hi bnemec , beagles ... i added you as reviewers in some changes related to tripleo-image-elements | 16:38 |
| yolanda | could you take a look when you have some time? | 16:38 |
| beagles | yolanda: ack | 16:38 |
| yolanda | thx :) | 16:38 |
| yolanda | i know these dates are hard | 16:38 |
| *** weshay_afk is now known as weshay_lunch | 16:39 | |
| *** limao_ has quit IRC | 16:39 | |
| *** eggmaster has joined #tripleo | 16:39 | |
| *** trown is now known as trown|lunch | 16:40 | |
| bnemec | panda|bbl: When you get back, can we talk about https://review.openstack.org/#/c/421822 ? | 16:40 |
| openstackgerrit | Pradeep Kilambi proposed openstack/instack-undercloud: Add panko service to undercloud https://review.openstack.org/418475 | 16:41 |
| *** tremble has quit IRC | 16:41 | |
| *** dmarlin has joined #tripleo | 16:41 | |
| pradk | jaosorior, ^^ one minor update .. i think i still need to explicitly include auth class in .pp | 16:42 |
| jaosorior | pradk: it was needed | 16:43 |
| jaosorior | thanks | 16:43 |
| *** mgoddard has quit IRC | 16:43 | |
| *** abregman has quit IRC | 16:47 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/instack-undercloud: Initialize gnocchi::keystone::auth on undercloud https://review.openstack.org/422095 | 16:47 |
| pradk | jaosorior, ^^ evidently we're missing this for gnocchi too | 16:47 |
| *** milan has quit IRC | 16:47 | |
| dtantsur | Ng, hey! | 16:48 |
| Ng | dtantsur: hey, sorry, now I'm on a call, but it should only be a few minutes | 16:48 |
| dtantsur | ok :) | 16:48 |
| *** zoliXXL is now known as zoli|gone | 16:51 | |
| *** salmankhan has quit IRC | 16:52 | |
| jaosorior | pradk: well spotted! | 16:52 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/instack: Updated from global requirements https://review.openstack.org/422100 | 16:53 |
| *** salmankhan has joined #tripleo | 16:55 | |
| *** psahoo has quit IRC | 16:56 | |
| openstackgerrit | Michael Chapman proposed openstack/tripleo-quickstart: Replace ansible_user with ssh_user https://review.openstack.org/422103 | 16:57 |
| *** egafford has joined #tripleo | 16:58 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Set DIB_GRUB_TIMEOUT to 0 for ci images https://review.openstack.org/413248 | 16:58 |
| bkero | adarazs: here's a job in which subnode logs should have been collected: http://logs.openstack.org/12/420912/3/experimental/gate-tripleo-ci-centos-7-nonha-multinode-oooq-nv/330c45e/ | 16:58 |
| *** psahoo has joined #tripleo | 16:59 | |
| bkero | The inventory file should have contained the overcloud, and targetted hosts are "all:!localhost" | 16:59 |
| *** ohamada has quit IRC | 17:00 | |
| adarazs | bkero: I'll check it in a minute | 17:00 |
| bkero | Thanks | 17:00 |
| EmilienM | bandini: I'm investigating https://bugs.launchpad.net/tripleo/+bug/1657108 | 17:03 |
| openstack | Launchpad bug 1657108 in tripleo "tripleo::firewall does not work as intended if the image has prepopulated firewall rules" [High,Triaged] | 17:03 |
| Ng | derekh: if I wanted to find the DIB command that builds images used in CI for overcloud nodes, where would I want to be looking? :) | 17:03 |
| EmilienM | bandini: by default, we don't purge Firewall rules that are not managed by Puppet | 17:04 |
| EmilienM | bandini: should we? | 17:04 |
| bandini | EmilienM: even with purge (which I tried), I think there is an issue: namely, we purge the rules but we do not empty /etc/sysconfig/iptables so when the iptables systemd service gets started we're back at square one | 17:04 |
| EmilienM | interesting | 17:05 |
| EmilienM | let me see if puppetlabs-firewall can manage this file | 17:05 |
| Ng | we're currently looking at whether it would be reasonable to empty that file at image build time | 17:05 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Re-generate nic-configs https://review.openstack.org/419048 | 17:05 |
| EmilienM | bandini: it's weird, puppet should override this file | 17:06 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras: Ensure that each role in extras has a dependency on extras-common https://review.openstack.org/413800 | 17:06 |
| openstackgerrit | Dmitry Tantsur proposed openstack/python-tripleoclient: [WIP] Create command for building RAID on given nodes https://review.openstack.org/421242 | 17:06 |
| bandini | EmilienM: hohum, I did try that and it did not work. So let me retry and look at logs more closely | 17:06 |
| bandini | EmilienM: does it override the file also on purge operation? | 17:07 |
| derekh | Ng: This is where tripleo-ci runs the image build command http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/tripleo.sh#n552 | 17:07 |
| Ng | derekh: thanks! | 17:07 |
| *** sudipto has quit IRC | 17:07 | |
| *** sudipto_ has quit IRC | 17:07 | |
| derekh | Ng: that builds the dib params | 17:07 |
| *** aufi has quit IRC | 17:07 | |
| EmilienM | bandini: what if you empty this file and deploy again? | 17:07 |
| bandini | EmilienM: then it all works correctly | 17:07 |
| EmilienM | bandini: nice! ok let me look in puppet code how we could do it. Gimme a sec | 17:08 |
| derekh | Ng: looks like trace isn't on so the logs don't show the values to the env variables used | 17:08 |
| *** bana_k has joined #tripleo | 17:08 | |
| Ng | derekh: hrm, would you like a patch to turn trace on? :) | 17:09 |
| Ng | derekh: we're just looking to see if the iptables t-i-e element is included in overcloud images | 17:09 |
| derekh | Ng: ahh, we should be able to find that, look in the file undercloud.tar.xz on a DIB patch, the list of elements is in var/log/image_build.txt | 17:10 |
| EmilienM | bandini: I need to test it in CI so we can see logs easily | 17:10 |
| Ng | derekh: awesome, thanks! | 17:10 |
| bandini | EmilienM: ack. if you have a patch I can test it here on a local env where I can reproduce this issue 100% | 17:11 |
| panda|bbl | bnemec: shockingly as it seems ${#CHANGES[@]} is a perfectly working variable ... even when the array is empty | 17:11 |
| openstackgerrit | Ben Nemec proposed openstack/tripleo-docs: Virtual deployments are not production-ready https://review.openstack.org/422114 | 17:12 |
| derekh | Ng: or maybe it isn't I'm looking for one now | 17:12 |
| bnemec | panda|bbl: So what exactly do these variables look like in a periodic job? Is ZUUL_CHANGES empty or unset? | 17:13 |
| EmilienM | bandini: how did you enable "purge"? | 17:13 |
| bandini | EmilienM: with PurgeFirewallRules: true as a parameter_default | 17:13 |
| florianf | jtomasek: re parameter normalization: Can the warnings appearing when merging params be ignored? | 17:13 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Do not merge - firewall: purge existing rules by default https://review.openstack.org/422117 | 17:13 |
| EmilienM | bandini: ok, sounds good | 17:13 |
| panda|bbl | bnemec: ZUUL_CHANGES is empty | 17:14 |
| jaosorior | I didn't know we weren't purging them already by default :/ | 17:14 |
| EmilienM | jaosorior: yeah, I did that for backwad compatibility | 17:14 |
| EmilienM | jaosorior: I didn't want to break our users | 17:14 |
| florianf | jtomasek: Because the t-h-t bug referenced in the commit message is scheduled for pike, so we will have to live with the warnings in ocata if we merge that one now. | 17:14 |
| panda|bbl | bnemec: I tested that block with two values of ZUUL_CHANGES: "" and "openstack/mistral:master:refs/changes/47/420547/1^openstack-infra/tripleo-ci:master:refs/changes/15/359215/35^openstack-infra/tripleo-ci:master:refs/changes/15/359215/35" | 17:15 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates: Add registry and role service list entries for Octavia https://review.openstack.org/418596 | 17:15 |
| EmilienM | jaosorior: back in 2014 :D https://github.com/stackforge/puppet-openstack-cloud/commit/5102e5130ae42e643db4b95824e3b3bc59541bdf | 17:17 |
| derekh | Ng: hmm, it somehow ends up in home/jenkins/overcloud-full.log | 17:17 |
| derekh | Ng: Expanded element dependencies to: pip-and-virtualenv-override rpm-distro puppet-modules install-bin pip-manifest dib-run-parts pkg-map manifests dhcp-all-interfaces selinux-permissive overcloud-base source-repositories package-installs os-refresh-config os-apply-config dynamic-login centos7 grub2 os-net-config hiera enable-packages-install truncate-nova-config overcloud-agent install-types install-static openssh-server stable-interface-names svc-map d | 17:17 |
| derekh | ib-python element-manifest openvswitch overcloud-opstools base dib-init-system os-svc-install overcloud-controller redhat-common overcloud-ceph-storage runtime-ssh-host-keys baremetal overcloud-full cache-url yum overcloud-compute puppet | 17:17 |
| jaosorior | EmilienM: wooow | 17:17 |
| Ng | derekh: fantastic, thanks very much! | 17:18 |
| derekh | Ng: np | 17:18 |
| jaosorior | EmilienM: hey dude, can you check this out? https://review.openstack.org/#/c/411340/23 we're just testing it with alee and it's working | 17:18 |
| bnemec | panda|bbl: Okay, weird. I still don't get an unbound variable with the - notation, but it looks like what you're doing should work so I'm going to stop bikeshedding over the implementation. :-) | 17:18 |
| jaosorior | EmilienM: we had tested it before, but it's been there for a while, so, better be safe :P | 17:18 |
| EmilienM | jaosorior: ok I'll look at it asap | 17:18 |
| panda|bbl | bnemec: mmmh, wait | 17:19 |
| panda|bbl | bnemec: you're right, I was getting the error in another line --_ | 17:20 |
| pradk | can i get some reviews on https://review.openstack.org/#/c/420087/ | 17:20 |
| panda|bbl | bnemec: changing change | 17:20 |
| jaosorior | bnemec: oh, still can't extract the tar archive http://logs.openstack.org/29/419529/21/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/9c9cd22/logs/postci.txt.gz#_2017-01-18_17_10_07_000 | 17:20 |
| EmilienM | pradk: +2 | 17:21 |
| pradk | thx! | 17:21 |
| rook | bnemec: so I thought I had a working solution, however it seems I was wrong... Do you recall what I had to do for the networking to work to ping the ipmi addresses with the virt-baremetal nodes? | 17:21 |
| *** tesseract has quit IRC | 17:21 | |
| EmilienM | jaosorior: this metadata_settings is new? | 17:22 |
| Ng | dtantsur: still available? :) | 17:22 |
| jaosorior | EmilienM: sort of (added all the pieces last year haha) | 17:22 |
| bnemec | rook: Your exact words to me were: "<rook> ok, i had to light iptables on fire, but good to go now." :-) | 17:22 |
| EmilienM | jaosorior: in 2014 too? lol | 17:22 |
| dtantsur | Ng, yep | 17:23 |
| jaosorior | EmilienM: hahaha a bit later :P | 17:23 |
| EmilienM | jaosorior: lgtm | 17:23 |
| jaosorior | EmilienM: thanks dude | 17:23 |
| rook | ha bnemec i thought there was something with the bmc too... if i see : https://gist.github.com/jtaleric/6fd491a7c4c9a1649bbb01b1bac6c557 things should be solid right? | 17:23 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci: Promotion: fix promotion check https://review.openstack.org/421822 | 17:23 |
| panda|bbl | bnemec: thanks for the feedback | 17:23 |
| *** florianf has quit IRC | 17:24 | |
| bnemec | rook: Yeah, you had a problem with the bmc not being able to get out to the host cloud endpoint, but if you're seeing the Managing instance messages then all is well there. | 17:24 |
| Ng | dtantsur: ok, so I think I just have two questions - the part about moving the logic to a workflow, can I collect up some of the data first in tripleoclient and pass that to the workflow? | 17:24 |
| dtantsur | Ng, to determine it, imagine you have to do the same in our (js-based) UI without logic duplication | 17:25 |
| dtantsur | if it's more CLI-specific, it should happen on CLI side | 17:25 |
| *** rhallisey has joined #tripleo | 17:25 | |
| dtantsur | if it's more of an actual logic, it should happen in a workflow | 17:25 |
| rook | yeah bnemec i wonder now if it is arp spoofing | 17:25 |
| EmilienM | dsneddon: is this one still valid ? https://bugs.launchpad.net/tripleo/+bug/1606316 | 17:25 |
| openstack | Launchpad bug 1606316 in tripleo " loadbalancer.pp Not Properly Accepting Multiple Addresses" [Undecided,Triaged] | 17:25 |
| dtantsur | d0uga, may correct me on that ^^^ | 17:25 |
| * rook head goes down | 17:25 | |
| bnemec | rook: Is it turned off in neutron? Also, keep in mind that there was a typo with the section name for the firewall_driver. | 17:26 |
| *** cylopez has quit IRC | 17:26 | |
| *** hafe has joined #tripleo | 17:26 | |
| bnemec | I know we fixed it in your environment, but it's possible that didn't get updated somewhere else? | 17:26 |
| Ng | dtantsur: ok, so like reading in the instackenv file, that seems like it would be reasonable to do before calling the workflow, analogous to uploading the file into the UI? | 17:27 |
| dtantsur | Ng, right, opening/parsing files should happen in CLI | 17:27 |
| EmilienM | bnemec: https://bugs.launchpad.net/tripleo/+bug/1657173 | 17:28 |
| openstack | Launchpad bug 1657173 in tripleo "sshd failing to start on undercloud" [High,Triaged] | 17:28 |
| EmilienM | bnemec: have we compared sshd version with rpm -qa thing? | 17:28 |
| Ng | dtantsur: awesome. The other question I had was about adding in support for other drivers - I can do my best based on what I see in their fencing interfaces in the puppet modules, but I don't have any way to test things that aren't pxe_ssh/pxe_ipmitool - is that ok? | 17:29 |
| bnemec | EmilienM: I haven't. I'm not actually sure when it started happening. | 17:29 |
| hafe | tripleo heat templates; where should I look to understand config_settings vs service_config_settings? | 17:29 |
| EmilienM | bnemec: logstash doesn't say it? | 17:29 |
| bnemec | Logstash probably knows, but it's been a long time so I suspect there will be many differences. | 17:29 |
| EmilienM | ah | 17:29 |
| Ng | dtantsur: I'm worried that that part might delay this and I'll miss the tripleoclient freeze this week and get fired ;) | 17:29 |
| dtantsur | Ng, pxe_drac and pxe_ilo are IPMI-compatible, so you can just treat them exactly the same | 17:30 |
| *** mcornea has quit IRC | 17:30 | |
| dtantsur | don't get fired please :) | 17:30 |
| Ng | dtantsur: I'll definitely add those, I just wasn't sure if you wanted me to add more than those | 17:30 |
| dtantsur | so instead of driver == 'pxe_ipmitool' you can do driver.split('_')[1] in {'ipmitool', 'ilo', 'drac') | 17:30 |
| dtantsur | I'm fine with ignoring the others | 17:30 |
| Ng | dtantsur: awesome, thank you very much! | 17:31 |
| Ng | I'll get all those changes worked up tonight and re-submitted :) | 17:31 |
| dtantsur | cool | 17:31 |
| shardy | hafe: we're still working on better docs around those, basically "config_settings" configures the service itself, e.g if it's puppet/services/heat-api.yaml it configures the heat API service | 17:31 |
| *** bana_k has quit IRC | 17:31 | |
| EmilienM | bnemec: and why sshd is not starting? no trace? | 17:31 |
| shardy | hafe: service_config_settings is different, as it lets a service pass configuration data to any other service (even if it's running on a different node) | 17:31 |
| EmilienM | Jan 18 11:24:16 tripleo-centos-7-tripleo-test-cloud-rh1-6738268 sshd[2447]: error: Bind to port 22 on 0.0.0.0 failed: Address already in use. | 17:32 |
| EmilienM | bnemec: ^ | 17:32 |
| EmilienM | looks like something is stealing 22 port | 17:32 |
| bnemec | EmilienM: WTF? | 17:32 |
| shardy | hafe: for example, heat-engine needs to configure DBs on the role running mysql, which it does here: | 17:32 |
| bnemec | I mean, who does that?! :-P | 17:32 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/heat-engine.yaml#L129 | 17:32 |
| bnemec | To the netstats! | 17:33 |
| shardy | hafe: basically we automatically put that hieradata on whatever nodes are running mysql | 17:33 |
| shardy | regardless of where heat-engine is running | 17:33 |
| EmilienM | bnemec: and looking at netstat, looks like sshd is started and binded on 22 | 17:33 |
| EmilienM | bnemec: so another process tries to start | 17:33 |
| hafe | shardy: so what's in service_config_settings ends up in hieradata on all nodes? | 17:34 |
| shardy | hafe: not all nodes, if you say mysql: foo, foo only ends up on nodes running the mysql service | 17:34 |
| shardy | same with keystone: foo etc | 17:34 |
| derekh | jaosorior: I have a theory, but I could be wrong, your ip add command doesn't have a CIDR, with mean /32 is being used | 17:35 |
| bnemec | EmilienM: Hmm, but the ssh process started after the undercloud failure. | 17:35 |
| derekh | jaosorior: because its /32 for /24 get added for the subnet | 17:35 |
| EmilienM | bnemec: indeed | 17:35 |
| bnemec | It was created at 10:31, but the undercloud install failed at 10:28 | 17:35 |
| derekh | jaosorior:(correction) because its /32 no route for /24 get added for the subnet | 17:36 |
| EmilienM | bnemec: do we try to start openssh and sshd in same time? i've seen that in the past (not in tripleo) | 17:36 |
| derekh | jaosorior: so the reply traffic is trying to go out via eth0 | 17:36 |
| derekh | jaosorior: try sticking a /24 in the ip addr add command | 17:36 |
| bnemec | EmilienM: I dunno. | 17:37 |
| bnemec | EmilienM: Strangely, we're not generating ssh keys until 10:31 either: Jan 18 10:31:32 localhost sshd-keygen: Generating SSH2 RSA host key: [ OK ] | 17:37 |
| *** david-lyle is now known as bailing-wire | 17:37 | |
| bnemec | That's the first instance of ssh I can find in /var/log/messages. | 17:37 |
| EmilienM | so logstash says we started to have this problem on January 8th | 17:39 |
| EmilienM | let's compare rpms | 17:39 |
| jaosorior | derekh: so something like: ip addr add 192.168.24.250/24 dev eth1 ?? | 17:39 |
| bnemec | EmilienM: Speaking of, https://review.openstack.org/#/c/419081 is ready to merge | 17:39 |
| derekh | jaosorior: yes, I think so, then a route will get added to the host for the /24 subnet | 17:40 |
| *** dougbtv is now known as dougbtv|afk | 17:40 | |
| EmilienM | bnemec: approved | 17:40 |
| bnemec | Thanks | 17:40 |
| jaosorior | derekh: thanks! will try that out | 17:41 |
| openstackgerrit | Chris Jones proposed openstack/python-tripleoclient: Add baremetal_overcloud.GenerateFencing https://review.openstack.org/417898 | 17:42 |
| *** ccamacho has quit IRC | 17:42 | |
| hafe | shardy: so it is like a filter so that just some service specific hieradata is exposed to a node? | 17:42 |
| *** rbrady is now known as rbrady-afk | 17:43 | |
| shardy | hafe: basically yes - we do some processing in tripleo-heat-templates where we know what services are running on what roles | 17:43 |
| shardy | so, lets say you have a role KeystoneController which only runs keystone, it'll get the service_config_settings: keystone: foo data | 17:44 |
| EmilienM | bnemec: puppet-ssh was downgraded 1 day before it started to fail in CI | 17:44 |
| EmilienM | I think it's related | 17:45 |
| EmilienM | bnemec: https://www.diffchecker.com/DjdQzjCK | 17:45 |
| hafe | is it new due to "composable roles"? | 17:45 |
| EmilienM | bnemec: now I'm investigating changes in puppet-ssh | 17:45 |
| shardy | hafe: yes, it was implemented during newton as part of the composable roles work | 17:45 |
| shardy | https://github.com/openstack/tripleo-heat-templates/blob/master/overcloud.j2.yaml#L350 | 17:45 |
| openstackgerrit | Dmitry Tantsur proposed openstack/python-tripleoclient: [WIP] Create command for building RAID on given nodes https://review.openstack.org/421242 | 17:45 |
| shardy | that is the code which does it | 17:45 |
| bnemec | EmilienM: Oh, major version bump too. | 17:45 |
| EmilienM | bnemec: sorry, upgraded | 17:46 |
| EmilienM | not donwgraded | 17:46 |
| hafe | shardy: thanks! digesting... | 17:46 |
| *** bailing-wire has quit IRC | 17:46 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add IP to provisioning interface for FreeIPA if requested https://review.openstack.org/420882 | 17:46 |
| openstackgerrit | Alexander Chuzhoy proposed openstack/tripleo-docs: adding a note how to reconfigure overcloud to erase only the metadata, instead of the entire disk. https://review.openstack.org/422033 | 17:47 |
| shardy | hafe: note there is also global_config_settings which is deployed on all nodes without any filtering | 17:47 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: TLS everywhere job introduction https://review.openstack.org/391738 | 17:48 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: DO NOT MERGE - testing TLS everywhere job https://review.openstack.org/419529 | 17:48 |
| *** lucasagomes is now known as lucas-afk | 17:48 | |
| openstackgerrit | Alexander Chuzhoy proposed openstack/tripleo-docs: adding a note how to reconfigure overcloud to erase only the metadata, instead of the entire disk. https://review.openstack.org/422033 | 17:48 |
| *** amoralej is now known as amoralej|off | 17:48 | |
| EmilienM | bnemec: something in https://github.com/saz/puppet-ssh/compare/v2.9.1...v3.0.1 problably | 17:48 |
| EmilienM | bnemec: we might just want to compare sshd config | 17:49 |
| EmilienM | so we can see what changed | 17:49 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add IP to provisioning interface for FreeIPA if requested https://review.openstack.org/420882 | 17:50 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: TESTING: ping FreeIPA server https://review.openstack.org/421393 | 17:51 |
| *** karimb has quit IRC | 17:51 | |
| *** beagles is now known as beagles-brb | 17:51 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: TLS everywhere job introduction https://review.openstack.org/391738 | 17:51 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: DO NOT MERGE - testing TLS everywhere job https://review.openstack.org/419529 | 17:51 |
| *** psahoo has quit IRC | 17:52 | |
| EmilienM | bnemec: I see no diff :/ | 17:53 |
| *** weshay_lunch is now known as weshay | 17:53 | |
| *** charliejllewelly has joined #tripleo | 17:54 | |
| jaosorior | shardy: if you get some time can you check these out https://review.openstack.org/#/q/status:open+project:openstack/tripleo-heat-templates+branch:master+topic:bp/novajoin ? | 17:55 |
| *** hafe has quit IRC | 17:55 | |
| charliejllewelly | Hey all, sorry if this is a silly question. I have found a bug with Mitaka. If I want to patch this is it just a case of checking out stable/mitaka and then creating a branch off that and submitting it for review? | 17:55 |
| *** mhenkel has quit IRC | 17:55 | |
| bnemec | EmilienM: As much as that seems promising, I'm not sure how puppet-ssh would cause this. systemd shouldn't allow it to start the service twice. | 17:55 |
| bnemec | charliejllewelly: Is it specific to mitaka, or something that would still exist in later versions too? | 17:56 |
| EmilienM | bnemec: https://review.openstack.org/#/c/408637/ might be related | 17:56 |
| EmilienM | bnemec: since now puppet manages sshd service | 17:56 |
| shardy | jaosorior: ack will do | 17:57 |
| EmilienM | bnemec: but only on overcloud, mhh | 17:57 |
| *** trown|lunch is now known as trown | 17:59 | |
| EmilienM | bnemec: let me try something :) | 17:59 |
| openstackgerrit | Alexander Chuzhoy proposed openstack/tripleo-docs: adding a note how to reconfigure overcloud to erase only the metadata, instead of the entire disk. https://review.openstack.org/422033 | 17:59 |
| bnemec | It's funny that we even need https://github.com/openstack/instack-undercloud/blob/master/elements/puppet-stack-config/puppet-stack-config.pp#L618 | 18:00 |
| bnemec | Since undercloud installs are almost always done over ssh. | 18:00 |
| *** derekh has quit IRC | 18:01 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras: virtualbmc setup is not working in devmode.sh https://review.openstack.org/421571 | 18:01 |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: ssh: use tripleo profile to deploy SSH https://review.openstack.org/422146 | 18:01 |
| EmilienM | bnemec: ^ | 18:01 |
| *** ramishra has quit IRC | 18:01 | |
| *** jpich has quit IRC | 18:01 | |
| EmilienM | lhinds: you might want to review https://review.openstack.org/422146 and help to debug https://bugs.launchpad.net/tripleo/+bug/1657173 please | 18:02 |
| openstack | Launchpad bug 1657173 in tripleo "sshd failing to start on undercloud" [High,Triaged] | 18:02 |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: ssh: use tripleo profile to deploy SSH https://review.openstack.org/422146 | 18:02 |
| EmilienM | shardy, marios, chem: this one needs to be in your radar: https://bugs.launchpad.net/tripleo/+bug/1657175 | 18:03 |
| openstack | Launchpad bug 1657175 in tripleo "During Newton->Ocata upgrade we need to install python-heat-agent-* and ansible-pacemaker module" [Critical,Triaged] | 18:03 |
| *** jcoufal has quit IRC | 18:03 | |
| shardy | EmilienM: thanks - we already install python-heat-agent* and work around some issues related to the new hiera hook | 18:03 |
| shardy | I'll link details into the bug | 18:04 |
| *** owalsh is now known as owalsh-afk | 18:04 | |
| *** flepied has quit IRC | 18:04 | |
| bnemec | EmilienM: Hmm, I wonder if https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/sshd.pp#L58 is related. | 18:04 |
| ansiwen | shardy: did you see my message from before? any idea how to proceed, given that locally it doesn't reproduce? | 18:04 |
| bnemec | We didn't have that in the i-u service definition. | 18:04 |
| EmilienM | shardy: thank you | 18:05 |
| EmilienM | bnemec: yes, but I tried to run service sshd status on my host and it works | 18:05 |
| shardy | jaosorior: ouch, https://review.openstack.org/#/c/411340 is pretty heavy on the yaql | 18:05 |
| chem | ansiwen: hi, I've got a strange mariadb error during a gnocchi-upgrade | 18:05 |
| EmilienM | bnemec: hastatus is when "status" doesn't work | 18:05 |
| EmilienM | chem: you better to ping pradk | 18:05 |
| jaosorior | shardy: didn't find a better way to do it :/ | 18:05 |
| shardy | jaosorior: I wonder if we could use a heat conditional to only create the yaql metadata_settings when needed, e.g TLS everywhere is actually enabled? | 18:06 |
| chem | EmilienM: it's really a mysql error, but he ... I don't mind asking pradk. Thanks | 18:06 |
| *** yolanda has quit IRC | 18:06 | |
| *** yolanda has joined #tripleo | 18:06 | |
| pradk | chem, whats up | 18:06 |
| chem | so ansiwen pradk: during gnocchi-upgrade I've got DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') | 18:07 |
| jaosorior | shardy: well, https://review.openstack.org/#/c/411340/23/extraconfig/nova_metadata/krb-service-principals.yaml is disabled by default. and most of the time the metadata settings will be empty, unless TLS-everywhere is enabled | 18:07 |
| *** alee has quit IRC | 18:07 | |
| pradk | chem, hmm anything in gnocchi logs? | 18:07 |
| chem | so ansiwen pradk: my questions are: have you ever seen this; which logs I should collect to debug this | 18:07 |
| ansiwen | chem: you mix me up with someone I think, I didn't work with neither gnocchi nor mysql | 18:08 |
| pradk | chem, please check gnocchi-upgrade.log | 18:08 |
| chem | ansiwen: sorry yes, my bad :) | 18:08 |
| EmilienM | chem: you want to talk to dciabrin then :) | 18:08 |
| shardy | jaosorior: ah, yeah I see the ApacheTLS resource is mapped to None by default, cool | 18:08 |
| EmilienM | he's our MySQL and Galera guru here :D | 18:08 |
| *** owalsh-afk has quit IRC | 18:08 | |
| EmilienM | chem: and you guys can speak french :-P | 18:09 |
| *** athomas has quit IRC | 18:09 | |
| jaosorior | shardy: also the hook https://github.com/openstack/tripleo-heat-templates/blob/master/overcloud-resource-registry-puppet.j2.yaml#L45 | 18:09 |
| chem | pradk: for the moment all the logs I have is that http://pastebin.test.redhat.com/447185 | 18:09 |
| *** owalsh-afk has joined #tripleo | 18:09 | |
| *** owalsh-afk is now known as owalsh | 18:09 | |
| chem | pradk: the python db stack trace | 18:10 |
| shardy | jaosorior: we will re-calculate the exact same data from the same ServiceNetMap for all services that include the apache-internal-tls-certmonger.yaml resource tho | 18:10 |
| shardy | https://review.openstack.org/#/c/411340/23/puppet/services/apache-internal-tls-certmonger.yaml | 18:10 |
| pradk | chem, that just tells me that mysql is not reachable from where gnocchi is running? | 18:10 |
| shardy | that isn't a singleton, it's created in every service template | 18:10 |
| chem | pradk: hum ... it look to me that it can connect and then output DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') | 18:11 |
| *** dtantsur is now known as dtantsur|afk | 18:11 | |
| jaosorior | shardy: I see.... so what are the options? | 18:12 |
| chem | pradk: I think the error message is different when the db is not reachable | 18:12 |
| jaosorior | shardy: either way, we need that on every node that deploys something with apache. which can now vary cause of composable roles | 18:13 |
| pradk | chem, well yea lost meaning it connected and the lost .. this seems like a db issue.. not sure if its specific to gnocchi | 18:13 |
| jaosorior | shardy: would be great to do it more efficiently, but I just couldn't figure it out without the yaql | 18:13 |
| pradk | chem, you see this with any other service? | 18:13 |
| openstackgerrit | Ana Krivokapic proposed openstack/tripleo-heat-templates: Add plan-environment.json https://review.openstack.org/417385 | 18:13 |
| chem | pradk: dunno yet, I have to collect log and was wondering what were the needed one | 18:14 |
| *** shardy has quit IRC | 18:15 | |
| chem | pradk: on my list: /var/log/mariadb/mariadb.log /var/log/gnocchi/gnocchi-upgrade.log journalctl -u mariadb | 18:15 |
| chem | dciabrin: hi, have you ever see this kind of error with mariadb DBConnectionError: (pymysql.err.OperationalError) (2013, 'Lost connection to MySQL server during query') | 18:15 |
| chem | dciabrin: it looks like a "classic" mariadb error: https://dev.mysql.com/doc/refman/5.7/en/gone-away.html | 18:16 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Remove CeilometerStoreEvents parameter https://review.openstack.org/422154 | 18:16 |
| chem | dciabrin: could this just mean that the db has gone away ? | 18:16 |
| *** shardy has joined #tripleo | 18:16 | |
| chem | pradk: do you see other logs that might be relevents ? | 18:17 |
| bnemec | Since I'm burning through all of my review request karma today: https://review.openstack.org/#/c/347503/ is also good to go. | 18:17 |
| bnemec | It's already running on the te-broker and working fine. | 18:17 |
| pradk | chem, from gnocchi pov, thats the only one which show db stuff | 18:17 |
| *** salmankhan has quit IRC | 18:17 | |
| chem | pradk: ack, thanks | 18:17 |
| bnemec | It knocks about a minute off the testenv creation time for nonha jobs. | 18:17 |
| bnemec | Plus it enables us to start testing with bonded nics. | 18:18 |
| openstackgerrit | Charlie Llewellyn proposed openstack/tripleo-heat-templates: Allow puppet configuration of ceph nodes https://review.openstack.org/422156 | 18:20 |
| bnemec | \o/ https://review.openstack.org/#/c/378014/ passed as well | 18:20 |
| * bnemec doesn't know what to do with all of these ci passes | 18:20 | |
| EmilienM | bnemec: the exit 0 patch helped a lot :-D | 18:21 |
| bnemec | EmilienM: Shhhh, don't tell :-P | 18:22 |
| *** yprokule has quit IRC | 18:22 | |
| *** jerrygb has quit IRC | 18:23 | |
| jaosorior | shardy: did you get my messages? | 18:23 |
| bnemec | EmilienM: jaosorior: Thanks for the reviews | 18:24 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Add a environment file to disable ceilo api https://review.openstack.org/420087 | 18:25 |
| EmilienM | bnemec: thanks for your work on improving our CI! | 18:25 |
| bnemec | EmilienM: *knocks on wood* The queue has been a lot shorter today since we stopped rebuilding IPA in every job. | 18:26 |
| bnemec | Although I'm looking at the mitaka jobs nostalgically now. 1:30 nonha jobs even with building images? Madness! | 18:26 |
| *** yamahata has quit IRC | 18:26 | |
| openstackgerrit | Charlie Llewellyn proposed openstack/tripleo-heat-templates: Allow puppet configuration of SNMP on ceph nodes https://review.openstack.org/422156 | 18:27 |
| shardy | jaosorior: sorry I think my network dropped for a moment, I just read the eavesdrop log | 18:28 |
| shardy | jaosorior: made some comments on the review, basically I wonder if we can just move the common queries related to ServiceNetmap into the hook instead of the apache base service template | 18:28 |
| shardy | then it'll only run once per role | 18:28 |
| shardy | jaosorior: not blocking but see what you think | 18:28 |
| openstackgerrit | Pradeep Kilambi proposed openstack/instack-undercloud: Add panko service to undercloud https://review.openstack.org/418475 | 18:29 |
| jaosorior | shardy: hey man, thanks for the feedback, I'll give it another try to see what I can come up with | 18:29 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras: virtualbmc setup is not working in devmode.sh https://review.openstack.org/421571 | 18:30 |
| *** sdake has quit IRC | 18:30 | |
| jaosorior | shardy: but yeah, that sounds reasonable | 18:30 |
| jaosorior | EmilienM: is there a way to make an environment variable available to what's being ran by puppet? | 18:31 |
| jaosorior | lets say that for certain commands, I would like an environment variable to be taken into account, (specifically KRB5_CLIENT_KTNAME). | 18:32 |
| EmilienM | jaosorior: a custom fact would do it | 18:32 |
| EmilienM | we actually already do it somewhere | 18:32 |
| EmilienM | but you can get the env from ruby and export to a fact and use the fact into the manifests | 18:32 |
| *** Goneri has quit IRC | 18:33 | |
| jaosorior | EmilienM: so, the command is ran in a puppet provider, will that work the same there? | 18:34 |
| shardy | ansiwen: sorry I'm not clear what the problem is - AFAICS your scenario worked, and the endpoint now looks OK | 18:34 |
| shardy | http://logs.openstack.org/34/398634/20/check/gate-tripleo-ci-centos-7-scenario002-multinode/1a61ec1/logs/postci.txt.gz#_2017-01-17_19_59_29_000 | 18:34 |
| shardy | so I assume you found the problem and fixed it? | 18:34 |
| shardy | check the Ec2ApiInternal entry there | 18:34 |
| *** beagles-brb is now known as beagles | 18:34 | |
| EmilienM | jaosorior: you have 2 options: give the parameter (fact) to the provider in your manifest OR patch the provider to use env | 18:35 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Failure debugging improvements https://review.openstack.org/419081 | 18:35 |
| EmilienM | jaosorior: what's the use case? | 18:35 |
| shardy | I don't think your scenario actually exercises the API tho, just deploys it? | 18:35 |
| EmilienM | shardy: right | 18:35 |
| EmilienM | shardy: there is no way to test ec2api from a heat template (pingtest) I think | 18:35 |
| *** absubram has joined #tripleo | 18:35 | |
| jaosorior | EmilienM: so, for getting the certs via puppet-certmonger, I used to do a kinit (which authenticates to kerberos) explicitly. However, it seems it's possible to run it without the kinit, which basically requires us to use an environment variable that certmonger will use to fetch the keytab and authenticate | 18:36 |
| pradk | shardy, EmilienM, can you review this as well please https://review.openstack.org/#/c/420081/ .. this goes in hand with https://review.openstack.org/420087 | 18:36 |
| *** nyechiel has quit IRC | 18:36 | |
| shardy | EmilienM: yeah I can't think of a good way to do that | 18:37 |
| pradk | without this api will still remain disabled as the hiera now is unset with other patch merged | 18:37 |
| *** charliejllewelly has quit IRC | 18:37 | |
| shardy | perhaps we can add it to the overcloud sanity tests I've been putting in tripleo.sh | 18:37 |
| EmilienM | jaosorior: so write a custom fact which gets this env, and use it as a parameter value when you use the provider | 18:37 |
| EmilienM | pradk: ok | 18:37 |
| EmilienM | shardy: yes | 18:37 |
| jaosorior | EmilienM: got any examles? haven't done that before | 18:37 |
| ansiwen | shardy: nope, I didn't fix anything... and the ovb-nonha the endpoints are still broken... doesn't matter? see here and search for '://:': http://logs.openstack.org/34/398634/20/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/0370e4e/logs/postci.txt.gz#_2017-01-17_23_36_03_000 | 18:38 |
| pradk | thx guys | 18:39 |
| *** jpena is now known as jpena|off | 18:39 | |
| shardy | ansiwen: but it's not enabled in the nonha job, is it? | 18:39 |
| *** sdake has joined #tripleo | 18:40 | |
| EmilienM | jaosorior: that's a custom fact | 18:40 |
| EmilienM | return ENV['my_env_thing'] | 18:40 |
| EmilienM | and it's done! | 18:40 |
| EmilienM | call this fact $::myfact and use this value when you call the provider (in the attributes) | 18:40 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: add a tq gate check workflow for build-test-packages https://review.openstack.org/422065 | 18:43 |
| ansiwen | shardy: no, it's not, but when I had the tester-commit before, that set OS::TripleO::Services::Ec2Api: puppet/services/ec2-api.yaml in overcloud-resource-registry-puppet.j2.yaml, the endnodes where still not set correctly and it failed because of that. is that expected? deploying locally with "-e environments/services/ec2-api.yaml" did work though, and the endpoint were correct. so verything is fine? | 18:44 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates: Add Octavia API service definitions https://review.openstack.org/411872 | 18:45 |
| jaosorior | EmilienM: uhm... what if the environment variable I need always has the same value. Only thing I need is that the value is set and available for that process, and that's it. Would it be the same thing? | 18:45 |
| jaosorior | through a custom fact | 18:45 |
| *** bana_k has joined #tripleo | 18:46 | |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: ntp: make sure chrony is purged before https://review.openstack.org/422164 | 18:47 |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: ntp: make sure chrony is purged before https://review.openstack.org/422164 | 18:48 |
| EmilienM | jaosorior: I'm not sure to understand tbh | 18:49 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates: Introduce Octavia implememtation services https://review.openstack.org/411874 | 18:49 |
| *** karimb has joined #tripleo | 18:50 | |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-heat-templates: Add registry and role service list entries for Octavia https://review.openstack.org/418596 | 18:50 |
| jaosorior | EmilienM: so, if I set KRB5_CLIENT_KTNAME to /etc/krb5.keytab the getcert call from certmonger will use that keytab to authenticate. So all I need is that environment variable to have that value. | 18:50 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add IP to provisioning interface for FreeIPA if requested https://review.openstack.org/420882 | 18:51 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: TESTING: ping FreeIPA server https://review.openstack.org/421393 | 18:51 |
| shardy | ansiwen: I'll take a closer look later, just dropping off for a while now - basically I think the EndpointMap output shouldn't contain services which aren't enabled, but it does because we don't filter the endpoints | 18:52 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Composable roles within services - NTP https://review.openstack.org/310725 | 18:52 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: TLS everywhere job introduction https://review.openstack.org/391738 | 18:52 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: DO NOT MERGE - testing TLS everywhere job https://review.openstack.org/419529 | 18:52 |
| shardy | provided it looks OK in your patch where the service is enabled, I'd probably just ignore it for now, and perhaps raise a bug | 18:52 |
| shardy | I don't understand why you can't reproduce the same locally tho | 18:52 |
| *** shardy is now known as shardy_afk | 18:52 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Implement NTP profile https://review.openstack.org/310725 | 18:54 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: add a tq gate check workflow for build-test-packages https://review.openstack.org/422065 | 18:54 |
| *** jprovazn has quit IRC | 18:55 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Deploy NTP with puppet-tripleo profile https://review.openstack.org/422167 | 18:55 |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: ntp: make sure chrony is purged before https://review.openstack.org/422164 | 18:56 |
| bnemec | Ruh-roh: http://logs.openstack.org/98/416298/4/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/f09dc95/logs/postci.txt.gz#_2017-01-18_18_11_32_000 | 19:02 |
| bnemec | I think we have a puppet problem in jobs that use ceph. | 19:02 |
| bnemec | EmilienM: ^ | 19:02 |
| EmilienM | can we stop for the bugs today? | 19:04 |
| *** tosky_ has quit IRC | 19:04 | |
| EmilienM | bnemec: this code was written by gfidente | 19:05 |
| EmilienM | gfidente: do you mind to take a look please? | 19:06 |
| bnemec | It _just_ started showing up, so I'm betting something released recently. | 19:06 |
| EmilienM | selinux or ceph package | 19:06 |
| EmilienM | let's see rpm -qa | 19:06 |
| dprince | EmilienM: you okay to put this bit back https://review.openstack.org/#/c/421999/1 | 19:09 |
| *** jprovazn has joined #tripleo | 19:09 | |
| *** jcoufal has joined #tripleo | 19:09 | |
| EmilienM | dprince: if it can help you, ok... | 19:10 |
| *** sdake_ has joined #tripleo | 19:10 | |
| dprince | EmilienM: our containers have more recent RPMs that requires the nova cells DB for setup | 19:10 |
| dprince | EmilienM: only creating the database doesn't hurt while Nova fixes the core issues | 19:10 |
| EmilienM | bnemec: https://www.diffchecker.com/npr5vXS7 | 19:11 |
| EmilienM | dprince: cool man | 19:11 |
| EmilienM | bnemec: could be policycoreutils | 19:12 |
| bnemec | EmilienM: I was literally about to do the same thing. :-) | 19:12 |
| bnemec | There are a lot of differences. :-/ | 19:12 |
| EmilienM | or libsemanage | 19:12 |
| bnemec | It is an selinux failure. | 19:13 |
| EmilienM | bnemec: maybe https://github.com/SELinuxProject/selinux/commit/5b0ad2f00ec730b86eb871d30cb98661dc7a6554 | 19:13 |
| bnemec | And there's a new selinux-policy. | 19:13 |
| *** sdake has quit IRC | 19:14 | |
| bnemec | EmilienM: Mmm, does look suspicious. Especially considering Notice: /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[fcontext_/srv/data]/returns: ValueError: Type ceph_var_lib_t is invalid, must be a file or device type | 19:14 |
| gfidente | EmilienM bnemec which code? | 19:15 |
| bnemec | Heh, "overcloud delete does not actually delete the overcloud" | 19:15 |
| bnemec | Working as intended. :-D | 19:15 |
| EmilienM | gfidente: in puppet-ceph | 19:15 |
| bnemec | I'm opening a bug. One minute. | 19:15 |
| EmilienM | gfidente: https://github.com/openstack/puppet-ceph/commit/f13493abc38cb13eec94bf203f15ec1d26d7ad28 | 19:15 |
| gfidente | but that's been there from quite a while | 19:16 |
| EmilienM | gfidente: yes, but now it fails | 19:16 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart-extras: Ensure sphinx build output hits stdout https://review.openstack.org/422178 | 19:16 |
| EmilienM | gfidente: see http://logs.openstack.org/98/416298/4/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/f09dc95/logs/postci.txt.gz#_2017-01-18_18_11_32_000 | 19:16 |
| EmilienM | Error: semanage fcontext -a -t ceph_var_lib_t '/srv/data(/.*)?' && restorecon -R /srv/data returned 1 | 19:17 |
| *** absubram has quit IRC | 19:17 | |
| EmilienM | also: | 19:17 |
| EmilienM | /Stage[main]/Ceph::Osds/Ceph::Osd[/srv/data]/Exec[fcontext_/srv/data]/returns: ValueError: Type ceph_var_lib_t is invalid, must be a file or device type | 19:17 |
| EmilienM | gfidente: is it something that we take care of ^ ? | 19:17 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart-extras: Ensure sphinx build output hits stdout https://review.openstack.org/422178 | 19:17 |
| gfidente | sounds like the selinux file type is not available | 19:18 |
| gfidente | as if ceph-selinux was not installed? | 19:18 |
| EmilienM | gfidente: see https://www.diffchecker.com/npr5vXS7 | 19:18 |
| EmilienM | ceph-selinux is here | 19:18 |
| EmilienM | ceph-selinux-10.2.2-0.el7.x86_64 | 19:19 |
| bnemec | https://bugs.launchpad.net/tripleo/+bug/1657562 | 19:19 |
| openstack | Launchpad bug 1657562 in tripleo "Ceph deployments failing on selinux error" [Critical,Triaged] | 19:19 |
| openstackgerrit | Dirk Mueller proposed openstack/os-refresh-config: Adjust test suite for new psutil versions https://review.openstack.org/422180 | 19:19 |
| openstackgerrit | Dirk Mueller proposed openstack/os-refresh-config: Updated from global requirements https://review.openstack.org/420924 | 19:20 |
| EmilienM | pradk: can you take https://bugs.launchpad.net/tripleo/+bug/1657555 please? | 19:20 |
| openstack | Launchpad bug 1657555 in tripleo "CeilometerStoreEvents set to false, but Ceilometer still stores events" [Undecided,New] | 19:20 |
| pradk | EmilienM, yep https://review.openstack.org/#/c/422154/ | 19:21 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Add docker nova services https://review.openstack.org/420921 | 19:21 |
| EmilienM | pradk: Closes-Bug please | 19:21 |
| pradk | EmilienM, i'll refer the bug.. dint realize there was a bug | 19:22 |
| EmilienM | np | 19:22 |
| *** absubram has joined #tripleo | 19:22 | |
| *** sdake_ is now known as sdkae | 19:23 | |
| *** sdkae is now known as sdake | 19:23 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Remove CeilometerStoreEvents parameter https://review.openstack.org/422154 | 19:24 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Deploy network envs appropriate for the job https://review.openstack.org/347503 | 19:24 |
| *** jerrygb has joined #tripleo | 19:24 | |
| jaosorior | adarazs: still around? | 19:24 |
| bnemec | Heh, updates job is now voting. I wondered by all of a sudden all of the jobs were showing queued. | 19:24 |
| bnemec | s/by/why/ | 19:24 |
| bnemec | Weird typo o.O | 19:25 |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: Remove Glance Registry from undercloud https://review.openstack.org/417214 | 19:25 |
| EmilienM | doing a little break | 19:25 |
| EmilienM | brb | 19:25 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Rename updates-nv to just updates https://review.openstack.org/416308 | 19:26 |
| gfidente | EmilienM interesing policycoreutils changed but not ceph-selinux | 19:26 |
| *** abregman_ has joined #tripleo | 19:26 | |
| *** yamahata has joined #tripleo | 19:26 | |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart-extras: Add options to set the overcloud's FQDNs and domain https://review.openstack.org/421589 | 19:27 |
| jidar | anybody know how to get diskimage-builder to not include "pip-and-virtualenv" when building a image for SoftwareDeployments? | 19:27 |
| jtomasek | lblanchard: could you please link me to nodes assignment / tagging split wireframes? | 19:28 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-refresh-config: Updated from global requirements https://review.openstack.org/420924 | 19:28 |
| *** jmelvin has quit IRC | 19:29 | |
| *** jerrygb has quit IRC | 19:29 | |
| *** rasca has quit IRC | 19:29 | |
| bnemec | Oh, ick. Apparently moving updates to voting restarted all of those jobs. | 19:33 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: TLS everywhere job introduction https://review.openstack.org/391738 | 19:33 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: DO NOT MERGE - testing TLS everywhere job https://review.openstack.org/419529 | 19:33 |
| bnemec | [heat-admin@overcloud-controller-0 ~]$ heat stack-list | grep baremetal | wc -l | 19:33 |
| bnemec | 80 | 19:33 |
| dtrainor | Anyone know how I can reset the state of validations? Ultimately I want to mimic what would happen with 'prep' validations immediately after I log in to UI for the first time. | 19:34 |
| gfidente | I just tried with the updated versions of policycoreutils and libsemanage didn't reproduce | 19:34 |
| *** rbrady-afk is now known as rbrady | 19:36 | |
| lblanchard | jtomasek: here is what I have currently: https://openstack.invisionapp.com/share/247VLM4SB#/164572244_2016-6-6_TripleO_UI18 | 19:37 |
| *** trown is now known as trown|mtg | 19:37 | |
| EmilienM | ok back | 19:37 |
| lblanchard | jtomasek: using the "Assign Nodes" link which shows the modal where the user can assign or tag. We probably do need the spinner or text box on the main deployment plan page to allow "Auto assign" | 19:38 |
| openstackgerrit | Pradeep Kilambi proposed openstack/instack-undercloud: Remove store events setting https://review.openstack.org/422184 | 19:44 |
| gfidente | bnemec EmilienM so this is not using cached images right? | 19:44 |
| *** fultonj has quit IRC | 19:45 | |
| jtomasek | lblanchard: thanks | 19:45 |
| lblanchard | jtomasek: no problem | 19:45 |
| dprince | EmilienM: per your recent work I'm adding it to Kolla as well here https://review.openstack.org/#/c/422185/ (Nova placement API) | 19:45 |
| dprince | sdake, Slower: ^^ | 19:45 |
| sdake | dprince sup | 19:46 |
| bnemec | gfidente: It might be | 19:46 |
| EmilienM | dprince: cool man! | 19:46 |
| sdake | dprince need a review? | 19:46 |
| dprince | sdake: new placement API package added to the nova-api Kolla container | 19:46 |
| gfidente | bnemec I am trying to isolate what could be causing it from the base image | 19:46 |
| gfidente | to reproduce it locally | 19:46 |
| dprince | sdake: https://review.openstack.org/#/c/422185/ | 19:47 |
| gfidente | upgrading libsemanage and policycoreutils didn't reproduce it | 19:47 |
| sdake | dprince cool - no idea what a placement api is :) | 19:47 |
| sdake | i am looking at the patch now | 19:47 |
| sdake | needs a bug or blueprint | 19:47 |
| dprince | sdake: might need to file a BP or bug for tracking but it is a simple thing I think | 19:47 |
| sdake | other then that looks fine | 19:47 |
| dprince | sdake: I'll add a quick BP then, thanks | 19:47 |
| sdake | dprince i also wonder about src deployments | 19:47 |
| sdake | dprince is the nova placement api a separate nova deliverable | 19:48 |
| sdake | or same tarball? | 19:48 |
| Slower | is that going to mean another process? | 19:50 |
| dprince | sdake: same tarball, so source should have the bits. Its primarily a wsgi script that I need | 19:53 |
| sdake | dprince wfm | 19:54 |
| sdake | dprince typically the wy you would run another process is via a separate microcontainer | 19:54 |
| sdake | such as nova-api-placement | 19:54 |
| sdake | however I don't understand enough about how nova works to know if that i the correct approach | 19:54 |
| sdake | rather microservice | 19:55 |
| sdake | too man ynouns, verbs, numbers, names :) | 19:55 |
| EmilienM | gfidente: did you try to update everything? | 19:56 |
| dprince | sdake: Added the BP link into the patch here https://review.openstack.org/#/c/422185/ | 19:57 |
| gfidente | EmilienM I tried the policy-targed | 19:59 |
| gfidente | trying a full upgrade now | 19:59 |
| *** jmelvin has joined #tripleo | 19:59 | |
| EmilienM | gfidente: thanks for helping | 20:00 |
| rook | i think the other bit was i had to restart the computes bnemec | 20:00 |
| *** bailing-wire has joined #tripleo | 20:00 | |
| openstackgerrit | Merged openstack/puppet-tripleo: Remove legacy flag and use composable interface https://review.openstack.org/420081 | 20:00 |
| sdake | dprince your welcome to join the drivers team of kolla (we have a pretty open policy on this) to fix bugs, approve blueprints that other people author, etc | 20:01 |
| sdake | dprince our general policy is anyone that shows interest in the project is permitted into the drivers team | 20:01 |
| sdake | the drivers team is different from the core review tema of course | 20:02 |
| dprince | sdake: ack, thanks I will | 20:02 |
| sdake | dprince if interested ping me | 20:02 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: do not merge: purge firewall rules by default https://review.openstack.org/422203 | 20:02 |
| sdake | dprince ack you wil ping me or join? :) | 20:02 |
| dprince | sdake: I'm interested :) | 20:02 |
| sdake | dprince i htink we had a bit of overlap lag :) | 20:02 |
| sdake | dprince adding - whats your launchpad id | 20:02 |
| *** bailing-wire is now known as david-lyle | 20:02 | |
| sdake | dprince which of the 3 kolla deliverables, all 3, or just the kolla deliverlabe? | 20:02 |
| sdake | dprince note we have only one rule, you can't approve your own blueprints you write | 20:03 |
| sdake | you can schedule them as you please | 20:03 |
| sdake | if we kick them out of a release, its because they aren't ready | 20:03 |
| sdake | note policies her ecould always change in the future - howeve rwe have operated this way for 3 years with no ill effects | 20:04 |
| gfidente | EmilienM though I did an OS upgrade and the command still succeeded | 20:04 |
| gfidente | it seems to be something related to the policy in-memory | 20:04 |
| dprince | sdake: ack, I'll try to keep things in sync with this. No self approvals :) | 20:04 |
| *** dsneddon is now known as dsneddon_afk | 20:04 | |
| *** jkilpatr has quit IRC | 20:05 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci: [WIP] DO NOT MERGE - test tempest job using quickstart https://review.openstack.org/421272 | 20:05 |
| *** jerrygb has joined #tripleo | 20:05 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci: OOQ/OVB: Add configurations for tempest job https://review.openstack.org/420620 | 20:05 |
| sdake | dprince i added "dprince" as your launchpad id - can you confirm this is you? | 20:05 |
| *** jkilpatr has joined #tripleo | 20:06 | |
| sdake | dprince it says "DPrince" in launchpad, which I think may not be you | 20:07 |
| sdake | dprince i'm going ot temporarily remove dprince until i get confirmation of your launchpad id | 20:07 |
| sdake | dprince some people get added inadvertently then wonder what is going on wit hthat | 20:07 |
| sdake | dprince there are a few dan princes in there | 20:08 |
| *** ooolpbot has joined #tripleo | 20:11 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 20:11 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1657562 | 20:11 |
| *** ooolpbot has quit IRC | 20:11 | |
| openstack | Launchpad bug 1657562 in tripleo "Ceph deployments failing on selinux error" [Critical,Triaged] | 20:11 |
| *** jerrygb has quit IRC | 20:13 | |
| *** dsneddon has joined #tripleo | 20:15 | |
| *** trown|mtg is now known as trown | 20:15 | |
| openstackgerrit | Chris Jones proposed openstack/puppet-pacemaker: Add rsoec tests for ironic stonith fence https://review.openstack.org/422218 | 20:17 |
| openstackgerrit | Chris Jones proposed openstack/puppet-pacemaker: Add rspec tests for ironic stonith fence https://review.openstack.org/422218 | 20:17 |
| Ng | mwhahaha: ^^ the tests aren't passing yet, but that's what I have so far for the follow-on for https://review.openstack.org/#/c/390396/ | 20:18 |
| mwhahaha | Ng: ok thanks | 20:18 |
| *** Goneri has joined #tripleo | 20:18 | |
| mwhahaha | i'll look in a min | 20:18 |
| bandini | EmilienM: just for the record I think I spotted why purge is not working as expected. I added a comment https://bugs.launchpad.net/tripleo/+bug/1657108/comments/3 | 20:19 |
| openstack | Launchpad bug 1657108 in tripleo "tripleo::firewall does not work as intended if the image has prepopulated firewall rules" [High,Triaged] | 20:19 |
| EmilienM | bandini: oh man, nice catch | 20:20 |
| openstackgerrit | Merged openstack/tripleo-quickstart: ensure the volume pool can be marked autostart in all cases https://review.openstack.org/421467 | 20:20 |
| EmilienM | bandini: I don't get your E) | 20:21 |
| gfidente | bnemec I would like to try adding a command to postci | 20:21 |
| gfidente | if I run a yum install in there will it work? | 20:21 |
| bandini | EmilienM: so if we enable purge (which by default is set to false in THT), we still need to fix the bug mentioned for it to work | 20:21 |
| EmilienM | bandini: you mean changing Default to True in TripleO? | 20:21 |
| EmilienM | bandini: right | 20:21 |
| bandini | EmilienM: yeah, set purge to true by default in tripleo *and* fix the above bug | 20:21 |
| EmilienM | bandini: bug has been open for long time :/ | 20:22 |
| bandini | yeah, kinda sucks | 20:22 |
| panda|bbl | if we don't merge https://review.openstack.org/421822 we won't get promotions (yes I should have got this right the first time :/ ) | 20:22 |
| EmilienM | panda|bbl: done | 20:23 |
| EmilienM | bandini: I can look at the module | 20:23 |
| panda|bbl | EmilienM: thanks! | 20:23 |
| EmilienM | bandini: I have a workaround for tripleo also | 20:23 |
| EmilienM | bandini: i'll send a patch and see how it works | 20:23 |
| bandini | EmilienM: oh, what's your workaround? | 20:24 |
| EmilienM | bandini: but we can make sure iptables is started before managing any rule | 20:24 |
| EmilienM | I mean before purging | 20:24 |
| EmilienM | with some resource ordering | 20:24 |
| bandini | ah right that would work around this, true | 20:24 |
| bnemec | gfidente: Yeah, a yum install should work fine. | 20:24 |
| bnemec | gfidente: So far it looks like the failing jobs are the ones using cached images, BTW. | 20:25 |
| *** abregman_ has quit IRC | 20:28 | |
| gfidente | bnemec so it seems the ceph module is not loaded into selinux policy to me | 20:28 |
| *** panda|bbl is now known as panda | 20:29 | |
| gfidente | but that is something supposedly happening in a postinstall script of ceph-selinux | 20:29 |
| gfidente | this /usr/sbin/semodule -i /usr/share/selinux/packages/ceph.pp | 20:29 |
| gfidente | gets ceph_var_lib_t into the policy | 20:29 |
| *** akuznetsov has joined #tripleo | 20:31 | |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/tripleo-quickstart: ensure that tripleo-quickstart volume pool is defined https://review.openstack.org/422224 | 20:31 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart-extras: Ensure sphinx build output is collected https://review.openstack.org/422178 | 20:31 |
| EmilienM | bnemec: this problem is tricky | 20:33 |
| *** karimb has quit IRC | 20:35 | |
| EmilienM | bandini: I meant to ping you, the problem is super tricky. I wonder if we should just empty the file :/ | 20:37 |
| openstackgerrit | Giulio Fidente proposed openstack-infra/tripleo-ci: [NO MERGE] Meant to see if the ceph-selinux moduled is loaded https://review.openstack.org/422226 | 20:39 |
| EmilienM | gfidente: I wondr if Puppet CI is also red on ceph job | 20:40 |
| EmilienM | gfidente: let me try it | 20:40 |
| bandini | EmilienM: I am slowly starting to think that indeed emptying the file is the easiest way :/ I tried so many different approaches (also with gfidente's help) | 20:40 |
| *** sshnaidm|bbl is now known as sshnaidm | 20:40 | |
| openstackgerrit | Giulio Fidente proposed openstack-infra/tripleo-ci: [NO MERGE] Meant to see if the ceph-selinux moduled is loaded https://review.openstack.org/422226 | 20:43 |
| *** absubram has quit IRC | 20:43 | |
| larsks | bandini, for the record, I disagree with the analysis in https://tickets.puppetlabs.com/browse/MODULES-3184 | 20:43 |
| EmilienM | larsks: why? | 20:44 |
| larsks | The module right now is pretty os-agnostic, so it doesn't "know" where firewall state is persisted on disk, as far as I can tell. So "fixing" the module to know about /etc/sysconfig/iptables seems wrong. | 20:44 |
| larsks | Or does it? /me takes a second look... | 20:45 |
| larsks | Oh, huh, I missed that in lib/puppet/type/firewall.rb | 20:45 |
| bandini | larsks: I guess the solution can be discussed, but the problem is indeed that one. also note that the module does know about /etc/sysconfig/iptables and stores its rules there | 20:45 |
| larsks | I retract my statement. | 20:45 |
| bandini | eheh | 20:45 |
| larsks | We're hitting similar problems in some ansible work. The answer is definitely to purge everything first, because you can't really have more than one thing managing iptables without terrible things happening at some point. | 20:46 |
| bandini | larsks: exactly. how would you "purge everythin first"? emptying /etc/sysconfig/iptables? | 20:46 |
| bandini | or make the purge system in puppetlabs-firewall work in the case where iptables service is non active on boot? | 20:46 |
| *** akuznetsov has quit IRC | 20:47 | |
| *** jmelvin has quit IRC | 20:47 | |
| bandini | right now we're leaning towards just emptying /etc/sysconfig/iptables | 20:47 |
| larsks | bandini, do you care about existing active firewall rules (that may or may not have been persisted to /etc/sysconfig/iptables yet)? Or do you assume a reboot at some point? | 20:48 |
| larsks | Or I guess an explicit iptables-restore? | 20:48 |
| *** jmelvin has joined #tripleo | 20:48 | |
| *** jmelvin has quit IRC | 20:48 | |
| *** egafford has quit IRC | 20:49 | |
| *** jmelvin has joined #tripleo | 20:49 | |
| bandini | larsks: I don't think we care about the existing rules as the ones we want are preserved by puppet anyway. We basically want to kill the rules contained in the stock /etc/sysconfig/iptables | 20:49 |
| bandini | which are provided by the rpm itself (i.e. the ones opening icmp and ssh only) | 20:49 |
| EmilienM | should we purge it in the image? | 20:49 |
| bandini | EmilienM: that would be my take, yes | 20:49 |
| larsks | Otherwise, yes, emptying /etc/sysconfig/iptables and (re-)starting the "iptables" service should take care of things. | 20:50 |
| bandini | via some DIB magic (of which I have very little knowledge)? | 20:50 |
| EmilienM | bandini: it sounds very workaroundish but well | 20:51 |
| bandini | EmilienM: yeah :/ it is not ideal for sure | 20:51 |
| openstackgerrit | Dan Radez proposed openstack/diskimage-builder: Adding aarch64 support for CentOS7 https://review.openstack.org/422232 | 20:52 |
| bandini | larsks: how is your DIB fu? | 20:53 |
| larsks | bandini, pretty much does not exist :/. I've mucked about with tripleo-puppet-elements a few times, but that was pretty much it. | 20:53 |
| *** jerrygb has joined #tripleo | 20:53 | |
| *** absubram has joined #tripleo | 21:00 | |
| EmilienM | bandini: let me look at it | 21:02 |
| bandini | ack | 21:03 |
| *** lblanchard has quit IRC | 21:03 | |
| bandini | EmilienM: another idea would be to empty the file via puppet only if purge is set to true. How does that sound? | 21:04 |
| bandini | (and change the purge default to true in tht) | 21:04 |
| EmilienM | bandini: it would be hard to make it idempotent | 21:05 |
| bandini | right it would get messy rather quickly | 21:06 |
| EmilienM | we also need to think at upgrades | 21:06 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Expose instance discovery method for compute agent https://review.openstack.org/422237 | 21:06 |
| openstackgerrit | Rajini Ram proposed openstack/tripleo-heat-templates: Enables support for configuring Cinder with a ScaleIO storage backend. This change adds all relevant parameters for: - EMC ScaleIO Block Storage driver configuration https://review.openstack.org/422238 | 21:07 |
| jaosorior | pradk: was libvirt_metadata already the default? | 21:07 |
| pradk | jaosorior, it will be once the upstream patch merges | 21:08 |
| *** jkilpatr has quit IRC | 21:08 | |
| jaosorior | pradk: is there something we need to think about for upgrades? | 21:09 |
| jaosorior | pradk: or does a restart of that service with the new value just work? | 21:09 |
| pradk | jaosorior, restart should do it i think | 21:09 |
| openstackgerrit | Rajini Ram proposed openstack/tripleo-heat-templates: Enables support for configuring Cinder with a ScaleIO storage backend. https://review.openstack.org/422238 | 21:09 |
| *** ooolpbot has joined #tripleo | 21:11 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 21:11 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1657562 | 21:11 |
| *** ooolpbot has quit IRC | 21:11 | |
| openstack | Launchpad bug 1657562 in tripleo "Ceph deployments failing on selinux error" [Critical,Triaged] | 21:11 |
| openstackgerrit | Rajini Ram proposed openstack/tripleo-heat-templates: Enables support for configuring Cinder with a ScaleIO storage backend. https://review.openstack.org/422238 | 21:12 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Deploy NTP with puppet-tripleo profile https://review.openstack.org/422167 | 21:13 |
| sshnaidm | EmilienM, jaosorior, trown, could you please review/merge in your time: https://review.openstack.org/#/c/421729 it's fix for pingtest of nonha oooq job | 21:16 |
| jaosorior | sshnaidm: sure, will check it out | 21:16 |
| pradk | jaosorior, can you re-review https://review.openstack.org/#/c/418475/ .. there is a useless assert in the tests needed update | 21:16 |
| trown | EmilienM: seem infra is not very interested in the approach I took to disable non-quickstart OVB jobs on quickstart only tripleo-ci patches :( ... convinced AJaeger to remove -1, but can't convince any other infra folks to review it. wdyt of putting all those files in quickstart-extras and just having the toci script in tripleo-ci clone and kick the scripts from there | 21:16 |
| trown | EmilienM: that would solve the same thing from a different angle | 21:17 |
| EmilienM | mhh | 21:17 |
| EmilienM | why can't they let us work how we want | 21:17 |
| EmilienM | trown: where is discussion? | 21:17 |
| trown | EmilienM: in #openstack-infra | 21:18 |
| jaosorior | pradk: if you don't see much value in that assert (I don't either) you can just remove it in a subsequent commit | 21:19 |
| trown | EmilienM: asked fungi and clarkb if either would review, and they seem to really not like that skip-file thing, it is not specific to tripleo | 21:19 |
| pradk | jaosorior, yea i might just do that.. :) | 21:19 |
| *** hafe has joined #tripleo | 21:20 | |
| *** jprovazn has quit IRC | 21:20 | |
| EmilienM | trown: another angle might be fine then | 21:20 |
| trown | EmilienM: though I suspect they would be a bit more motivated to solve it if we were wasting an equivalent percentage of infra resources | 21:20 |
| panda | EmilienM: forgot a +2 in https://review.openstack.org/421822 ? | 21:20 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-quickstart-extras: Add virtualbmc_port field to scale config https://review.openstack.org/422243 | 21:20 |
| EmilienM | trown: yes | 21:21 |
| *** absubram has quit IRC | 21:21 | |
| fungi | honestly, no. i was pretty against the skip-if feature implementation in the first place | 21:21 |
| *** jkilpatr has joined #tripleo | 21:22 | |
| fungi | and the neutron dev who was pushing really hard for it disappeared from the community shortly after it came into use | 21:22 |
| fungi | there's a general view across infra that consistently running the same jobs for the same repos regardless of what files you're modifying reduces the chances that you'll deadlock your repo and waste more time submitting changes to adjust your job configuration to get back out of that mess | 21:23 |
| trown | fungi: fair enough, I think I have an idea to solve it in a different way | 21:23 |
| fungi | ci resources are cheaper than reviewers | 21:23 |
| fungi | and skip-if patches have a particularly high frequency of not being right on the first try and needing many subsequent adjustments until things are working as hoped | 21:24 |
| openstackgerrit | John Trowbridge proposed openstack-infra/tripleo-ci: DNM testing RDO dependency updates https://review.openstack.org/422244 | 21:24 |
| bandini | EmilienM: ok to merge this one in the meantime? https://review.openstack.org/#/c/418623/ | 21:25 |
| fungi | come zuul v3 when projects can configure much of this in their own repos and the job configuration becomes self-testing, the balance around that may shift significantly | 21:25 |
| EmilienM | trown: the idea you have is still heavy to implement, we'll need to pull scripts from another repo | 21:26 |
| *** leanderthal has quit IRC | 21:26 | |
| trown | EmilienM: ya it is not what I would choose as the first solution | 21:27 |
| trown | EmilienM: but it is 15+ hours of total OVB time per patch that is 100% wasted when only iterating on the quickstart stuff | 21:27 |
| EmilienM | yeah.. let's do it then | 21:28 |
| *** jayg is now known as jayg|g0n3 | 21:28 | |
| *** sdake_ has joined #tripleo | 21:29 | |
| jidar | is diskimage-builder just not for cloud images? ie: instances inside your overcloud | 21:30 |
| *** jerrygb_ has joined #tripleo | 21:30 | |
| jidar | and in addition, is OS::Heat::SoftwareDeployment just not for non-tripleO tasks? | 21:30 |
| larsks | jidar, diskimage-builder just builds disk images. You can use them for whatever you want. | 21:31 |
| larsks | OS::Heat::SoftwareDeployment works fine outside of tripleo. | 21:31 |
| jidar | https://ask.openstack.org/en/question/101659/osheatsoftwaredeployment-and-image-requirements-aka-diskimage-builder/ | 21:31 |
| openstackgerrit | Alex Schultz proposed openstack/python-tripleoclient: Run stack delete for overcloud delete https://review.openstack.org/422250 | 21:31 |
| larsks | It just requires the appropriate heat agents installed in your images. | 21:31 |
| jidar | right, so os-apply-config and os-collect-config etc etc | 21:31 |
| jidar | except one of those requires jq | 21:32 |
| jidar | from this commit here: https://git.openstack.org/cgit/openstack/diskimage-builder/commit/?id=dca44b77a278e1ece034afe29d649b86290491ba | 21:32 |
| *** sdake has quit IRC | 21:32 | |
| jidar | except jq doesn't get installed by any of the diskimage-builder tools | 21:32 |
| *** jcoufal_ has joined #tripleo | 21:33 | |
| *** jerrygb has quit IRC | 21:33 | |
| *** absubram has joined #tripleo | 21:33 | |
| *** akrivoka has quit IRC | 21:34 | |
| jidar | that's not entirely correct | 21:34 |
| *** jcoufal has quit IRC | 21:35 | |
| *** jcoufal__ has joined #tripleo | 21:35 | |
| openstackgerrit | Merged openstack/puppet-pacemaker: Add a remote class for pacemaker-remote https://review.openstack.org/418623 | 21:37 |
| *** jcoufal_ has quit IRC | 21:38 | |
| *** jcoufal has joined #tripleo | 21:38 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: enable Placement API on multinode jobs https://review.openstack.org/422258 | 21:38 |
| bnemec | jidar: That script is not part of either o-a-c or o-c-c, and in fact it should almost certainly not be in a tenant image at all. | 21:40 |
| bnemec | It also has a package install for jq, so even if it does somehow end up in the image the dep should be satisfied. | 21:40 |
| *** jcoufal__ has quit IRC | 21:40 | |
| *** jcoufal_ has joined #tripleo | 21:40 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Promotion: fix promotion check https://review.openstack.org/421822 | 21:40 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Add deployed server bootstrap to noop-ctlplane https://review.openstack.org/421586 | 21:41 |
| jidar | script was pulled from: http://docs.openstack.org/developer/heat/template_guide/software_deployment.html#software-deployment-resources | 21:41 |
| *** trown is now known as trown|outtypewww | 21:42 | |
| jidar | man all I want is to be able to write some puppet code into a heat template | 21:42 |
| jidar | this is wayyyyyyyy too involved | 21:42 |
| bnemec | jidar: None of the elements in the doc should be pulling in ironic-discoverd-ramdisk. If they are that's a huge bug. | 21:43 |
| *** jcoufal has quit IRC | 21:43 | |
| jidar | it's running this ./tripleo-image-elements/elements/os-refresh-config/os-refresh-config/post-configure.d/99-refresh-completed | 21:44 |
| *** dmacpher has quit IRC | 21:44 | |
| jidar | which is what has jq, which is what fails | 21:44 |
| jidar | is that at least by design? (not the fail part) | 21:44 |
| bnemec | jidar: That's not what you linked to above. | 21:44 |
| jidar | which one? the git commit? if so, yea sorry I did mention that's not entirely correct | 21:45 |
| bnemec | jidar: That element should be installing jq in the image too: https://github.com/openstack/tripleo-image-elements/blob/master/elements/os-refresh-config/package-installs.yaml | 21:47 |
| jidar | doesn't that require epel though? | 21:48 |
| jidar | (for cent) | 21:48 |
| openstackgerrit | John Trowbridge proposed openstack-infra/tripleo-ci: DNM testing RDO dependency updates https://review.openstack.org/422244 | 21:48 |
| bnemec | jidar: epel or rdo I suspect | 21:49 |
| bnemec | I believe we're getting it from rdo. | 21:49 |
| *** ansmith has quit IRC | 21:50 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: provide instructions on how to only use quickstart as instack-virt-setup https://review.openstack.org/422260 | 21:50 |
| jidar | https://gist.github.com/35401adf45b42e0560f30be8682f0e6a | 21:50 |
| jidar | if you don't mind, that's the yum history of it _attempting_ to install jq | 21:50 |
| *** fultonj has joined #tripleo | 21:51 | |
| bnemec | Yeah, yum is stupid about that. | 21:51 |
| bnemec | It will report success even if some packages are missing. | 21:51 |
| bnemec | I don't know what we can do about it though. We need something to process the json, and even with jq it isn't trivial. | 21:51 |
| bnemec | Probably documenting the dependency for centos/rhel is the best short-term option. | 21:52 |
| jidar | hmm | 21:52 |
| jidar | I tried to wedge a yum -y install epel-release in there but it just spit out a bunch of errors | 21:53 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: provide instructions on how to only use quickstart as instack-virt-setup https://review.openstack.org/422260 | 21:53 |
| bnemec | jidar: There is an epel element you can just include that should work. | 21:55 |
| *** dmacpher has joined #tripleo | 21:56 | |
| jidar | praise.... looking | 21:56 |
| openstackgerrit | Merged openstack/tripleo-common: User provided pwd has to be merged for get_passwords workflow https://review.openstack.org/414099 | 21:57 |
| *** jcoufal has joined #tripleo | 21:58 | |
| *** morazi has quit IRC | 21:58 | |
| EmilienM | bnemec: I wounder how https://review.openstack.org/#/c/422095/ could pass nonha | 21:58 |
| EmilienM | bnemec: the ceph thing doesn't look consistent | 21:58 |
| EmilienM | gfidente: ^ | 21:58 |
| bnemec | EmilienM: It's an i-u patch, so it rebuilds images. | 21:59 |
| EmilienM | ah | 21:59 |
| bnemec | It looks like this is only happening with cached images. | 21:59 |
| EmilienM | once day I'll learn all of that | 21:59 |
| *** dprince has quit IRC | 21:59 | |
| *** jcoufal_ has quit IRC | 22:00 | |
| *** pkovar has quit IRC | 22:01 | |
| bnemec | It's doubly confusing now because the updates job never builds images. | 22:01 |
| bnemec | But it knocks almost half an hour off our job runtimes when we can use cached images. | 22:03 |
| bnemec | (which seems excessively long to me...) | 22:03 |
| EmilienM | jaosorior: are you still around? I need your help this time | 22:04 |
| EmilienM | jaosorior: I figured that http://paste.openstack.org/show/Pge5xuIioQE8oM6hBGzT/ | 22:04 |
| EmilienM | jaosorior: this is why placement api is not working | 22:04 |
| gfidente | bnemec so I thought the chroot was a good lead | 22:04 |
| EmilienM | jaosorior: I probably messed something in THT | 22:04 |
| *** iranzo has quit IRC | 22:04 | |
| gfidente | but I am trying running the semodule -i command within a chroot and it seems to behave correctly, I see the module loaded within the chroot but not outside | 22:05 |
| gfidente | so I am back at zero | 22:05 |
| *** hafe has quit IRC | 22:06 | |
| bnemec | gfidente: EmilienM: At this point it might be simplest just to kick off a promotion job. | 22:06 |
| EmilienM | bnemec: yes | 22:07 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: TEST: DONT RECHECK or REVIEW: periodic jobs https://review.openstack.org/359215 | 22:07 |
| gfidente | what does a promotion do to help though, sorry about stupid question | 22:07 |
| bnemec | gfidente: If it passes, the cached images will be updated. | 22:07 |
| *** jcoufal_ has joined #tripleo | 22:07 | |
| gfidente | bnemec ok I see | 22:07 |
| *** jcoufal__ has joined #tripleo | 22:10 | |
| *** jcoufal has quit IRC | 22:10 | |
| *** ooolpbot has joined #tripleo | 22:11 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 22:11 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1657562 | 22:11 |
| *** ooolpbot has quit IRC | 22:11 | |
| openstack | Launchpad bug 1657562 in tripleo "Ceph deployments failing on selinux error" [Critical,Triaged] | 22:11 |
| *** jcoufal_ has quit IRC | 22:12 | |
| jaosorior | EmilienM: still around | 22:12 |
| jaosorior | EmilienM: funky, the server names seem to be wrong | 22:13 |
| jaosorior | EmilienM: and the bind seems to be incorrect, thought we should be binding to the same network as the nova osapi | 22:14 |
| gfidente | bnemec so I'd try that, so we have it available if we decide to do so | 22:14 |
| bnemec | gfidente: EmilienM pushed https://review.openstack.org/359215 so the jobs should run shortly. | 22:15 |
| bnemec | As soon as we get through the backlog of restarted updates jobs, that is. :-/ | 22:15 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Fix pingtest for nonha ovb oooq job https://review.openstack.org/421729 | 22:17 |
| Ng | hmm, how would I call a workflow that's defined in tripleo-common, from a command in tripleoclient? I can see various interactions with mistral in tripleoclient, but none of them seem to work out (so clearly I'm being dumb somehow) | 22:17 |
| *** fultonj has quit IRC | 22:17 | |
| gfidente | bnemec I think the problem is that in the cached image the ceph module isn't enabled, so the policy doesn't know about the file context | 22:17 |
| gfidente | the problem is that this should be happening with that semanage -i command at postinstall so either this is failing somehow in the initial build | 22:18 |
| gfidente | or it's not persisted somehow | 22:19 |
| jaosorior | EmilienM: dude | 22:19 |
| *** Goneri has quit IRC | 22:19 | |
| *** jcoufal__ has quit IRC | 22:19 | |
| EmilienM | jaosorior: WHAT | 22:19 |
| EmilienM | :) | 22:19 |
| EmilienM | I'm in THT now, I'm fixing a tab thing in YAML | 22:20 |
| EmilienM | but not sure it will actually help | 22:20 |
| *** social has quit IRC | 22:20 | |
| jaosorior | EmilienM: can you see that issue in CI? | 22:20 |
| EmilienM | jaosorior: the issue is service sdoesn't start | 22:20 |
| EmilienM | because endpoint in haproxy is wrong | 22:20 |
| EmilienM | the binding is on the wrong adress | 22:20 |
| gfidente | EmilienM I guess I could enforce semodule -i in the puppet module if that seems reasonable | 22:20 |
| jaosorior | EmilienM: I think I know what the issue is | 22:20 |
| jaosorior | EmilienM: putting up a patch | 22:21 |
| EmilienM | jaosorior: I hope so :D | 22:21 |
| jaosorior | just gotta verify something in t-h-t first | 22:21 |
| jaosorior | EmilienM: but basically the main premise right now is that the lack of an entry for the placement api network is the issue | 22:22 |
| jaosorior | since it doesn't autogenerate the _vip entry | 22:22 |
| jaosorior | EmilienM: can the placement API be deployed in a node different than the node that will run nova osapi? | 22:22 |
| EmilienM | jaosorior: can we create a dedicated network? | 22:22 |
| EmilienM | so it's done... | 22:22 |
| EmilienM | jaosorior: yes | 22:23 |
| EmilienM | jaosorior: let's do it now so it's done | 22:23 |
| jaosorior | I'll do it | 22:23 |
| EmilienM | jaosorior: thanks :) | 22:23 |
| EmilienM | I also found a YAML thing | 22:23 |
| *** jmelvin has quit IRC | 22:23 | |
| *** panda is now known as panda|zZ | 22:24 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add entry in serviceNetMap for Nova Placement https://review.openstack.org/422280 | 22:26 |
| EmilienM | jaosorior: second iteration will be to use the network? | 22:27 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Use network entries for nova placement https://review.openstack.org/422281 | 22:27 |
| jaosorior | EmilienM: ^^ | 22:27 |
| *** b00tcat has joined #tripleo | 22:27 | |
| EmilienM | jaosorior: and network/endpoints/endpoint_map.yaml too no? | 22:28 |
| jaosorior | EmilienM: somehow I thought we had that already | 22:28 |
| *** david-lyle has quit IRC | 22:32 | |
| EmilienM | jaosorior: yes but it uses NovaApiNetwork | 22:32 |
| jaosorior | EmilienM: ah, true, fill add that | 22:32 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add entry in serviceNetMap for Nova Placement https://review.openstack.org/422280 | 22:34 |
| jaosorior | EmilienM: done | 22:35 |
| * EmilienM sends a e-hug to jaosorior | 22:35 | |
| jaosorior | EmilienM: lets wait for CI now | 22:35 |
| EmilienM | jaosorior: I'm about to be afk for ~ hours - if you want to see if it works, you need to check /var/log/nova on controller. If you see placement logs, we're good | 22:36 |
| jaosorior | EmilienM: ack | 22:37 |
| *** sdake_ has quit IRC | 22:38 | |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: Add missing FW rules for Placement API https://review.openstack.org/422289 | 22:39 |
| EmilienM | jaosorior: that also ^ :/ | 22:39 |
| *** david-lyle has joined #tripleo | 22:40 | |
| EmilienM | that's also why placement didn't start on undercloud | 22:40 |
| EmilienM | jaosorior: I need to go now, thanks for your help | 22:41 |
| * EmilienM disappears | 22:41 | |
| *** flepied has joined #tripleo | 22:44 | |
| *** liverpoo1er has quit IRC | 22:49 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Add storage-mgmt network to compute nodes https://review.openstack.org/419049 | 22:51 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Use custom network-isolation.yaml in ipv4 net-iso jobs https://review.openstack.org/422297 | 22:51 |
| bnemec | gfidente: I think ^ will fix the storage-mgmt failures. | 22:51 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Remove CeilometerStoreEvents parameter https://review.openstack.org/422154 | 22:51 |
| gfidente | bnemec the hci env is already enabling the storagemgmt port on the compute nodes | 22:54 |
| *** pmannidi has joined #tripleo | 22:54 | |
| gfidente | bnemec are there more changes in network-isolation.yaml ? | 22:55 |
| bnemec | gfidente: No, just that one line. So if you want to merge the two changes that should also do the trick. | 22:55 |
| jaosorior | bnemec: duuuuude | 22:56 |
| jaosorior | the TLS everywhere job moved forward | 22:56 |
| jaosorior | it failed | 22:56 |
| jaosorior | but it failed in an actual issue :D | 22:56 |
| jaosorior | the nova placement code is broken | 22:56 |
| bnemec | \o/ | 22:56 |
| jaosorior | bnemec: http://logs.openstack.org/29/419529/25/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/68f9756/logs/postci.txt.gz#_2017-01-18_22_50_09_000 | 22:57 |
| jaosorior | bnemec: can we merge this https://review.openstack.org/#/c/420882/ ? :D | 22:58 |
| jaosorior | EmilienM: hahaha nova placement API also breaks in the TLS job, but hey! it points it out in the job :D | 22:59 |
| bnemec | jaosorior: Are you sure you want to say "or IP" in the comment on the parameter? Wasn't just passing an IP what broke it before? | 22:59 |
| gfidente | bnemec no I meant isn't this single line | 22:59 |
| gfidente | https://github.com/openstack/tripleo-heat-templates/blob/master/environments/hyperconverged-ceph.yaml#L4 | 22:59 |
| gfidente | all we needed to add in network-isolation.yaml ? | 22:59 |
| *** flepied has quit IRC | 22:59 | |
| jaosorior | bnemec: mind if I fix that comment in that subsequent commit? so I don't have to rebase the TLS patch | 23:00 |
| bnemec | gfidente: Right, that's it. But the nic config change won't merge unless we also use the modified resource registry entry. | 23:00 |
| bnemec | jaosorior: Yeah, makes sense. | 23:00 |
| gfidente | bnemec oh right the problem is we use the modified nic templates for all jobs | 23:01 |
| gfidente | so all of them must be enabling storagemgmt port | 23:01 |
| gfidente | not just hci? | 23:01 |
| openstackgerrit | Merged openstack/instack-undercloud: Initialize gnocchi::keystone::auth on undercloud https://review.openstack.org/422095 | 23:01 |
| bnemec | gfidente: Well, no. Just the ha job is going to use the ipv4 nic-configs. | 23:01 |
| openstackgerrit | Merged openstack/instack-undercloud: Add panko service to undercloud https://review.openstack.org/418475 | 23:01 |
| bnemec | But there's a circular dependency between the nic-configs and the hci patch. | 23:01 |
| *** [1]cdearborn has quit IRC | 23:01 | |
| bnemec | One won't work without the other, unless we do https://review.openstack.org/422297 first. | 23:02 |
| gfidente | bnemec ah and they are in different repos | 23:02 |
| bnemec | Oh, except hci will only run against master for now, right? | 23:02 |
| bnemec | gfidente: Ah, I didn't even notice that. Yeah, that's also a problem. | 23:02 |
| *** dmacpher has quit IRC | 23:02 | |
| gfidente | bnemec ack, tks | 23:03 |
| bnemec | In that case I think we're okay. We can merge my two patches and then hci should work. | 23:03 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Fix comment in freeipa_setup.sh script https://review.openstack.org/422300 | 23:03 |
| jaosorior | bnemec: ^^ | 23:03 |
| *** Goneri has joined #tripleo | 23:04 | |
| jaosorior | gfidente: :P | 23:08 |
| gfidente | jaosorior how's going? | 23:09 |
| *** absubram has quit IRC | 23:09 | |
| jaosorior | gfidente: it's all good dude, greetings from Mexico | 23:09 |
| jaosorior | gfidente: what about you? | 23:09 |
| gfidente | oh you're there? | 23:09 |
| jaosorior | gfidente: yep, in Guadalajara currently | 23:09 |
| gfidente | like escaping the cold finland? | 23:09 |
| jaosorior | gfidente: hahaha kind of | 23:10 |
| jaosorior | gfidente: my girlfriend is learning spanish, so I'm here still until the middle of february | 23:10 |
| jaosorior | gfidente: arrived here before new year | 23:10 |
| jaosorior | it's been good :) | 23:10 |
| jaosorior | LOTS of food :P | 23:10 |
| gfidente | ahahha | 23:10 |
| *** sdake has joined #tripleo | 23:10 | |
| * jaosorior missed tacos | 23:11 | |
| *** ooolpbot has joined #tripleo | 23:11 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 23:11 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1657562 | 23:11 |
| *** ooolpbot has quit IRC | 23:11 | |
| openstack | Launchpad bug 1657562 in tripleo "Ceph deployments failing on selinux error" [Critical,Triaged] | 23:11 |
| gfidente | you remember I came to austin the only thing I had on my mind was to have some tacos | 23:11 |
| gfidente | well, after tripleo obviously | 23:11 |
| jaosorior | hahahaha I do :P | 23:11 |
| gfidente | so anyway looking back at the above bug | 23:11 |
| gfidente | :) | 23:12 |
| gfidente | which isn't funny anyway | 23:12 |
| *** saneax-_-|AFK is now known as saneax | 23:15 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Use network entries for nova placement https://review.openstack.org/422281 | 23:16 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras: Add options to set the overcloud's FQDNs and domain https://review.openstack.org/421589 | 23:19 |
| *** sdake has quit IRC | 23:24 | |
| *** yamahata has quit IRC | 23:24 | |
| *** dmarlin_ has quit IRC | 23:24 | |
| *** fungi has quit IRC | 23:24 | |
| *** apetrich has quit IRC | 23:24 | |
| *** fpan has quit IRC | 23:24 | |
| *** brault has quit IRC | 23:24 | |
| *** sdake has joined #tripleo | 23:24 | |
| *** yamahata has joined #tripleo | 23:24 | |
| *** dmarlin_ has joined #tripleo | 23:24 | |
| *** fungi has joined #tripleo | 23:24 | |
| *** apetrich has joined #tripleo | 23:24 | |
| *** fpan has joined #tripleo | 23:24 | |
| *** brault has joined #tripleo | 23:24 | |
| *** sdake has quit IRC | 23:25 | |
| *** sdake has joined #tripleo | 23:25 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-quickstart-extras: Enable novajoin/TLS-everywhere overcloud deploy arguments https://review.openstack.org/422310 | 23:26 |
| openstackgerrit | Michele Baldessari proposed openstack/tripleo-common: Add a PacemakerRemoteAuthkey to the default passwords https://review.openstack.org/420459 | 23:35 |
| bandini | jaosorior: yummmy tacoooos! (hopefully I will go Mexico this summer ;) | 23:35 |
| gfidente | bandini ^^ how was that generated before? | 23:35 |
| *** jkilpatr has quit IRC | 23:35 | |
| bandini | gfidente: it hasn't it is a new key and the patches using it have not landed yet | 23:36 |
| gfidente | bandini some randomstring in the template? | 23:36 |
| bandini | gfidente: nope it never existed before | 23:36 |
| gfidente | ah pacemaker remote, I confused it with the cluster user password | 23:36 |
| bandini | I want to land the review ^ above and then land pacemaker remote, so it should be all correct | 23:36 |
| bandini | yep :) | 23:36 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Use network entries for nova placement https://review.openstack.org/422281 | 23:37 |
| jaosorior | bandini: :D | 23:37 |
| jaosorior | bandini: where to Mexico? | 23:37 |
| bandini | jaosorior: well relatives are in DF. Some are in Valle de Bravo and some in Acapulco. Hopefully we hit all three places ;) | 23:38 |
| rbrady | Ng: did you find your answer about calling mistral workflows? | 23:39 |
| jaosorior | bandini: cool! | 23:39 |
| bandini | jaosorior: whereabout are you located when you stay in .mx? | 23:40 |
| *** StevenK_ is now known as StevenK | 23:40 | |
| jaosorior | bandini: right now I'm in Guadalajara | 23:40 |
| gfidente | you could at least bring me some tacos | 23:40 |
| jaosorior | bandini: in this trip, so far, I've been to Mexico city, Queretaro, Tequisquiapan, Guanajuato and here. Going to Monterrey and Saltillo on the weekend | 23:41 |
| bandini | jaosorior: sweet! I should expand my visiting area when I am in Mexico. We always end up stuck at relatives' ;) | 23:41 |
| *** pradk has quit IRC | 23:42 | |
| *** jerrygb_ has quit IRC | 23:42 | |
| jaosorior | bandini: haha same happens to me usually | 23:42 |
| bandini | and f00d..*SO MUCH* food! | 23:42 |
| * bandini can't wait ;) | 23:42 | |
| jaosorior | hahaha | 23:42 |
| gfidente | can you guys please stop | 23:43 |
| gfidente | this is very ot | 23:43 |
| *** sanagikoki has quit IRC | 23:43 | |
| bandini | gfidente: I will send you selfies while stuffing my face :P | 23:43 |
| gfidente | you know the netiquette and all that stuff | 23:43 |
| bandini | lol. anyways. time to hit the sack here. take care! | 23:43 |
| gfidente | you don't attach pictures because the others can't waste their bw | 23:43 |
| gfidente | see netiquette | 23:44 |
| Ng | rbrady: I did, thanks :) | 23:44 |
| rbrady | Ng: ack | 23:44 |
| Ng | rbrady: I should always just push at something for 30mins more before asking :) | 23:45 |
| *** jerrygb has joined #tripleo | 23:48 | |
| *** Goneri has quit IRC | 23:48 | |
| jaosorior | I gotta go | 23:48 |
| jaosorior | have a good rest of the day everyone! | 23:48 |
| *** jaosorior has quit IRC | 23:49 | |
| openstackgerrit | Giulio Fidente proposed openstack-infra/tripleo-ci: [NO MERGE] Meant to see if the ceph-selinux moduled is loaded https://review.openstack.org/422226 | 23:51 |
| *** dmacpher has joined #tripleo | 23:55 | |
| gfidente | bnemec can I download the cached image somehow? | 23:58 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Add IP to provisioning interface for FreeIPA if requested https://review.openstack.org/420882 | 23:59 |
| gfidente | Ng no that never worked for me | 23:59 |
| gfidente | I do need to ask for reviews to get to the next step | 23:59 |
| gfidente | it's not like I spend 30 or 60 minutes more and I find it myself :) | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!