| *** bfournie has joined #tripleo | 00:05 | |
| *** absubram has quit IRC | 00:13 | |
| openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart: Devmode: reposition convert and update https://review.openstack.org/397999 | 00:19 |
|---|---|---|
| openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart: Add configuration of HA IPv6 deployments https://review.openstack.org/380358 | 00:21 |
| *** rhallisey has joined #tripleo | 00:22 | |
| beagles | slagle, if you get a sec: https://review.openstack.org/#/c/395795/ | 00:29 |
| *** limao has joined #tripleo | 00:32 | |
| *** newmember has quit IRC | 00:33 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Revert "Adjust MTU to compensate for VLAN tag issue" https://review.openstack.org/392542 | 00:38 |
| openstackgerrit | Merged openstack/tripleo-common: Create new docker command hook https://review.openstack.org/312723 | 00:38 |
| openstackgerrit | Merged openstack/tripleo-puppet-elements: Remove redundant 'the' https://review.openstack.org/389652 | 00:38 |
| openstackgerrit | Merged openstack/os-collect-config: Updated from global requirements https://review.openstack.org/381142 | 00:38 |
| openstackgerrit | Merged openstack/os-cloud-config: Updated from global requirements https://review.openstack.org/381141 | 00:39 |
| openstackgerrit | Merged openstack/os-refresh-config: Updated from global requirements https://review.openstack.org/381143 | 00:39 |
| openstackgerrit | Merged openstack/instack-undercloud: apache: Remove product informations https://review.openstack.org/389255 | 00:39 |
| openstackgerrit | Merged openstack/python-tripleoclient: Enable releasenotes translation https://review.openstack.org/391698 | 00:39 |
| *** ecerquei has joined #tripleo | 00:41 | |
| *** jcoufal has quit IRC | 00:50 | |
| beagles | mmm.. okay that's weird, dsneddon, bnemec, you still around | 00:51 |
| *** morazi has quit IRC | 00:53 | |
| dsneddon | beagles, What's up? | 00:54 |
| *** owalsh_ has joined #tripleo | 00:55 | |
| beagles | dsneddon: I approved a patch with a recheck figuring it would run all the jobs to go green before merging the patch | 00:55 |
| beagles | dsneddon: https://review.openstack.org/#/c/392542/ .. but it seems that it didn't | 00:55 |
| dsneddon | beagles, Nope, it doesn't do that. | 00:55 |
| dsneddon | beagles, Surprise! | 00:55 |
| beagles | dsneddon: crap | 00:55 |
| *** owalsh has quit IRC | 00:56 | |
| beagles | dsneddon: not that it should be a problem but that was not what I intended | 00:56 |
| dsneddon | beagles, I think you can be forgiven. At least you didn't do it on a patch that was legitimately failing. | 00:56 |
| beagles | dsneddon: heh, I guess sticking your finger into the proverbial socket as a learning method is still alive and well... | 00:57 |
| *** egafford has joined #tripleo | 00:57 | |
| beagles | at least for me | 00:57 |
| dsneddon | beagles, With that ovb_ha job being flaky last and this week, I've been worried we might have to make it non-voting or merge in spite of it. | 00:57 |
| beagles | dsneddon: yeah, it's brutal | 00:57 |
| *** dmarlin has left #tripleo | 00:59 | |
| *** egafford has quit IRC | 01:03 | |
| *** kjw3 has quit IRC | 01:09 | |
| *** rajinir has quit IRC | 01:16 | |
| dsneddon | beagles, bnemec: The https://review.openstack.org/#/c/396446 patch is needed pretty desparately, can we just merge it? | 01:19 |
| beagles | dsneddon: yup :) | 01:20 |
| beagles | dsneddon: as far as I'm concerned anyways | 01:20 |
| dsneddon | beagles, Muchos gracias, amigo. | 01:21 |
| * dsneddon wonders is it muchas? | 01:21 | |
| * beagles has no idea :) | 01:21 | |
| dtrainor | take note, this is the first time in modern history that dsneddon is not certain of an answer | 01:22 |
| beagles | dsneddon: I agree fwiw... I was like "wait a sec, this is just doing X - lighten up dude" :) | 01:22 |
| dsneddon | beagles, Eh, it's usually me being the perfectionist. | 01:22 |
| *** egafford has joined #tripleo | 01:23 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates: Do not manage overcloud repositories when using external Ceph https://review.openstack.org/397819 | 01:24 |
| dtrainor | sorry in advance for the dumb question, but have we been using jinja for overcloud.yaml, for a while? | 01:24 |
| dsneddon | Only since Newton-3 or something. | 01:25 |
| dtrainor | there's an issue with the ui where some veritification of the tarball containing heat templates must not be accommodating for using jinja in these templates, trying to find a date around when this changed so i can research and write a smarter bug | 01:25 |
| dsneddon | I can tell you that, hang on a sec | 01:26 |
| dsneddon | Merged September 10: https://review.openstack.org/#/c/315679/ | 01:27 |
| dtrainor | now i'd like to ask you how you came to that answer :) | 01:27 |
| dsneddon | I went to review.openstack.org, and in the search field entered "project:openstack/tripleo-heat-templates jinja" and looked at the commit titles. | 01:28 |
| dsneddon | If you start typing "project:" it will auto-complete. You can also use "status: [open|merged]" or "owner: <author". | 01:29 |
| dtrainor | adding that to the list of things to study, thanks | 01:29 |
| dsneddon | Now, if you click on a review, there is a section below the commit message with Author, Committer, Commit, Parent, etc. If you click the (gitweb) link, it takes you to the code tree. | 01:30 |
| *** owalsh_ is now known as owalsh | 01:30 | |
| dtrainor | right | 01:30 |
| dtrainor | cool | 01:31 |
| dsneddon | But the one thing I haven't figured out how to do in the web interface, is search through the code for a string, so I still end up checking out all the projects and using git. | 01:31 |
| dsneddon | grep, I mean. both, actually. | 01:31 |
| dtrainor | heh | 01:33 |
| dsneddon | But you can do a search in the gitweb for strings in commit messages, which is somewhat handy. | 01:33 |
| dtrainor | i'm just used to giving up on launchpad's search, i should have dug deeper in gerrit's search. | 01:34 |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates: Do not manage overcloud repositories when using external Ceph https://review.openstack.org/397819 | 01:36 |
| *** fultonj has quit IRC | 01:38 | |
| *** links has joined #tripleo | 01:55 | |
| *** tiswanso has joined #tripleo | 01:57 | |
| *** rhallisey has quit IRC | 02:01 | |
| *** tiswanso has quit IRC | 02:02 | |
| openstackgerrit | Cao Xuan Hoang proposed openstack/instack: Replace 'assertTrue(a in b)' with 'assertIn(a, b)' https://review.openstack.org/398029 | 02:07 |
| *** links has quit IRC | 02:07 | |
| *** coolsvap has joined #tripleo | 02:19 | |
| *** tiswanso has joined #tripleo | 02:27 | |
| *** owalsh has quit IRC | 02:29 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Fix external Load Balancer deployment https://review.openstack.org/397658 | 02:30 |
| openstackgerrit | James Slagle proposed openstack/tripleo-heat-templates: Fix external Load Balancer deployment https://review.openstack.org/398040 | 02:30 |
| *** owalsh has joined #tripleo | 02:31 | |
| slagle | bandini: fyi, newton backport ^, if you can try to get it merged when you get in. thx | 02:32 |
| *** ecerquei has quit IRC | 02:34 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Prepare 6.0.0 (ocata-1) https://review.openstack.org/398046 | 02:38 |
| *** tiswanso has quit IRC | 02:39 | |
| *** tiswanso has joined #tripleo | 02:40 | |
| bfournie | beagles: thanks for the +2, looks like that ovb_ha job failed though, I'll recheck | 02:40 |
| beagles | bfournie: good luck :) that bloody ovb-ha job is being a nightmare | 02:41 |
| *** fpan has quit IRC | 02:44 | |
| *** fpan has joined #tripleo | 02:45 | |
| *** achadha_ has quit IRC | 02:49 | |
| *** fpan has quit IRC | 02:49 | |
| *** achadha has joined #tripleo | 02:50 | |
| *** fpan has joined #tripleo | 02:51 | |
| bfournie | beagles: yep :-( | 02:51 |
| *** achadha has quit IRC | 02:51 | |
| *** achadha has joined #tripleo | 02:52 | |
| *** achadha has quit IRC | 02:56 | |
| *** fzdarsky_ has joined #tripleo | 02:58 | |
| *** achadha has joined #tripleo | 03:00 | |
| *** fzdarsky|afk has quit IRC | 03:02 | |
| *** achadha has quit IRC | 03:05 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-puppet-elements: Disable old hiera.yaml o-a-c script https://review.openstack.org/392615 | 03:07 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Nova base cleanups for hiera json hook https://review.openstack.org/397974 | 03:08 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Horizon service cleanups for hiera json hook https://review.openstack.org/393258 | 03:08 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Hiera optimization: use a new hiera hook https://review.openstack.org/384757 | 03:09 |
| *** fpan has quit IRC | 03:23 | |
| *** udesale has joined #tripleo | 03:25 | |
| *** fpan has joined #tripleo | 03:25 | |
| *** achadha has joined #tripleo | 03:28 | |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates: GATE TEST, please ignore https://review.openstack.org/360618 | 03:30 |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates: GATE TEST, please ignore https://review.openstack.org/365449 | 03:30 |
| *** egafford has quit IRC | 03:30 | |
| *** achadha has quit IRC | 03:32 | |
| *** ctayal has joined #tripleo | 03:32 | |
| *** fragatin_ has joined #tripleo | 03:47 | |
| *** fragatina has quit IRC | 03:51 | |
| *** fragatin_ has quit IRC | 03:52 | |
| *** cwolferh has quit IRC | 04:04 | |
| *** TSCHAK has joined #tripleo | 04:06 | |
| *** TSCHAK has quit IRC | 04:08 | |
| *** sudipto has joined #tripleo | 04:24 | |
| *** yamahata has quit IRC | 04:32 | |
| *** cwolferh has joined #tripleo | 04:33 | |
| *** newmember has joined #tripleo | 04:47 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Horizon service cleanups for hiera json hook https://review.openstack.org/393258 | 04:48 |
| *** tiswanso has quit IRC | 04:53 | |
| *** sudipto has quit IRC | 04:53 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-tripleoclient: Updated from global requirements https://review.openstack.org/395379 | 04:53 |
| *** achadha_ has joined #tripleo | 04:54 | |
| *** jprovazn has joined #tripleo | 05:07 | |
| *** ctayal has quit IRC | 05:08 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/tripleo-common: Remove discover from test-requirements https://review.openstack.org/345225 | 05:09 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/os-collect-config: Remove discover from test-requirements https://review.openstack.org/345852 | 05:11 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/os-refresh-config: Remove discover from test-requirements https://review.openstack.org/345849 | 05:12 |
| *** Vijayendra has joined #tripleo | 05:16 | |
| *** masco has joined #tripleo | 05:21 | |
| openstackgerrit | Noam Angel proposed openstack/diskimage-builder: redhat-common add option to select networking service NetworkManager/network https://review.openstack.org/392170 | 05:26 |
| *** rlandy has quit IRC | 05:38 | |
| *** oshvartz has quit IRC | 05:39 | |
| *** achadha_ has quit IRC | 05:40 | |
| *** achadha has joined #tripleo | 05:40 | |
| *** bana_k has joined #tripleo | 05:48 | |
| openstackgerrit | Noam Angel proposed openstack/diskimage-builder: add option to configure cloud-init to allow password authentication https://review.openstack.org/391765 | 05:51 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Fix external Load Balancer deployment https://review.openstack.org/398040 | 05:51 |
| openstackgerrit | Noam Angel proposed openstack/diskimage-builder: add option to configure cloud-init to allow password authentication https://review.openstack.org/391765 | 05:52 |
| openstackgerrit | Noam Angel proposed openstack/diskimage-builder: add option to configure cloud-init to allow password authentication https://review.openstack.org/391765 | 05:54 |
| *** anshul has joined #tripleo | 06:02 | |
| *** pkovar has joined #tripleo | 06:12 | |
| *** jaosorior has joined #tripleo | 06:16 | |
| *** bana_k has quit IRC | 06:22 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Add Heat to scenario004 deployment https://review.openstack.org/398124 | 06:27 |
| *** anshul has quit IRC | 06:30 | |
| *** jprovazn has quit IRC | 06:32 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Use keystone profile parameter to pass heat password https://review.openstack.org/398127 | 06:33 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Remove explicit hiera calls for heat in keystone profile https://review.openstack.org/398128 | 06:35 |
| *** jaosorior has quit IRC | 06:41 | |
| *** jaosorior has joined #tripleo | 06:42 | |
| *** anshul has joined #tripleo | 06:44 | |
| *** cwolferh has quit IRC | 06:48 | |
| *** lmiccini has joined #tripleo | 06:50 | |
| *** dsariel has joined #tripleo | 06:51 | |
| *** mhenkel has joined #tripleo | 06:54 | |
| *** dmacpher is now known as dmacpher-afk | 06:57 | |
| *** oshvartz has joined #tripleo | 06:58 | |
| *** chandankumar has joined #tripleo | 07:00 | |
| *** iranzo has joined #tripleo | 07:03 | |
| *** iranzo has joined #tripleo | 07:03 | |
| *** pgadiya has joined #tripleo | 07:04 | |
| *** nyechiel has joined #tripleo | 07:09 | |
| *** aufi has joined #tripleo | 07:13 | |
| *** rasca has joined #tripleo | 07:15 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/python-tripleoclient: Enable creating images with Glance v2 https://review.openstack.org/387430 | 07:16 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Remove explicit hiera calls for heat in keystone profile https://review.openstack.org/398128 | 07:17 |
| *** ebarrera has joined #tripleo | 07:18 | |
| *** achadha has quit IRC | 07:22 | |
| *** abregman has joined #tripleo | 07:22 | |
| *** florianf has joined #tripleo | 07:23 | |
| *** achadha_ has joined #tripleo | 07:29 | |
| *** nyechiel has quit IRC | 07:31 | |
| *** cwolferh has joined #tripleo | 07:32 | |
| *** jaosorior has quit IRC | 07:34 | |
| *** jprovazn has joined #tripleo | 07:35 | |
| *** jlinkes has joined #tripleo | 07:40 | |
| *** pcaruana has joined #tripleo | 07:41 | |
| *** jbadiapa has joined #tripleo | 07:43 | |
| *** achadha_ has quit IRC | 07:49 | |
| openstackgerrit | Noam Angel proposed openstack/diskimage-builder: add option to configure cloud-init to allow password authentication https://review.openstack.org/391765 | 07:51 |
| *** dmacpher-afk is now known as dmacpher | 07:52 | |
| *** jaosorior has joined #tripleo | 07:54 | |
| *** shardy has joined #tripleo | 07:56 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Fix scenario004 deployment https://review.openstack.org/398124 | 08:00 |
| *** dbecker has joined #tripleo | 08:05 | |
| *** skramaja_ has joined #tripleo | 08:05 | |
| *** skramaja has quit IRC | 08:06 | |
| *** karthiks has quit IRC | 08:07 | |
| *** ccamacho has joined #tripleo | 08:07 | |
| openstackgerrit | Merged openstack/tripleo-validations: Fix the DHCP validations https://review.openstack.org/397727 | 08:10 |
| openstackgerrit | Flavio Percoco proposed openstack/tripleo-heat-templates: Correct AllNodesDeploySteps depends_on https://review.openstack.org/397983 | 08:12 |
| openstackgerrit | Flavio Percoco proposed openstack/tripleo-heat-templates: Containerized Services for Composable Roles https://review.openstack.org/330659 | 08:12 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-specs: Composable Service Upgrades https://review.openstack.org/392116 | 08:15 |
| *** cylopez has joined #tripleo | 08:16 | |
| *** amoralej|off is now known as amoralej | 08:20 | |
| *** mhenkel has quit IRC | 08:21 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add necessary parameters for encrypted volumes support https://review.openstack.org/398177 | 08:21 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add necessary parameters for encrypted volumes support https://review.openstack.org/398177 | 08:22 |
| *** karthiks has joined #tripleo | 08:22 | |
| ccamacho | morning guys! | 08:23 |
| *** fzdarsky_ has quit IRC | 08:26 | |
| jaosorior | morning | 08:26 |
| marios | \o ooo | 08:27 |
| *** hewbrocca_afk is now known as hewbrocca | 08:29 | |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: Create directories with root https://review.openstack.org/384892 | 08:29 |
| *** akrivoka has joined #tripleo | 08:29 | |
| *** jpich has joined #tripleo | 08:31 | |
| hewbrocca | marios: good morning Mr Bug Squasher | 08:33 |
| hewbrocca | Maybe we should just call you Sasquash | 08:33 |
| *** jpena|off is now known as jpena | 08:34 | |
| *** saneax-_-|AFK is now known as saneax | 08:36 | |
| *** hjensas has joined #tripleo | 08:36 | |
| *** tremble has joined #tripleo | 08:36 | |
| *** arxcruz has joined #tripleo | 08:38 | |
| *** owalsh has quit IRC | 08:38 | |
| *** owalsh has joined #tripleo | 08:39 | |
| *** fzdarsky has joined #tripleo | 08:39 | |
| marios | hewbrocca: o/ please don't jinx it ... also wasn't me... whole team has been fighting hard and honestly we aren't _done_ _done_ yet I mean as in the status, there are lots of things to go through QA still | 08:42 |
| marios | cautiously optimisti | 08:42 |
| marios | c | 08:43 |
| marios | hewbrocca: all the exciting new ways that sasha will find to break it between now and then :) | 08:43 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: Create directories with root https://review.openstack.org/384892 | 08:44 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/instack-undercloud: Stop pinning Glance API https://review.openstack.org/387432 | 08:47 |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-quickstart: WIP: Update OOOQ to ansible 2.2 https://review.openstack.org/398194 | 08:49 |
| *** mcornea has joined #tripleo | 08:53 | |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: Create directories with root https://review.openstack.org/384892 | 08:54 |
| jaosorior | mcornea: do you happen to have an SSL deployment lying around? | 08:55 |
| *** saneax is now known as saneax-_-|AFK | 08:55 | |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Format the nodes list in openstack overcloud delete node https://review.openstack.org/395083 | 08:56 |
| *** anshul has quit IRC | 08:56 | |
| *** athomas has joined #tripleo | 08:56 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 08:57 |
| openstackgerrit | Noam Angel proposed openstack/diskimage-builder: add option to configure cloud-init to allow password authentication https://review.openstack.org/391765 | 08:57 |
| *** ealcaniz has joined #tripleo | 08:58 | |
| *** saneax-_-|AFK is now known as saneax | 09:00 | |
| jaosorior | ccamacho: got time for a review? https://review.openstack.org/#/c/398127/1 :D | 09:00 |
| ccamacho | jaosorior, sure man, added to the queue :) | 09:02 |
| *** dtantsur|afk is now known as dtantsur | 09:03 | |
| *** skramaja has joined #tripleo | 09:06 | |
| *** skramaja_ has quit IRC | 09:06 | |
| *** karthiks has quit IRC | 09:06 | |
| openstackgerrit | Merged openstack/python-tripleoclient: Updated from global requirements https://review.openstack.org/395379 | 09:08 |
| *** anshul has joined #tripleo | 09:11 | |
| jaosorior | shardy: I accidentally messed up the scenario004, could you check out the fix? https://review.openstack.org/#/c/398124/ | 09:12 |
| *** ohamada has joined #tripleo | 09:13 | |
| *** saneax is now known as saneax-_-|AFK | 09:13 | |
| openstackgerrit | chenyingnan proposed openstack/instack: Don't include openstack/common in flake8 exclude list https://review.openstack.org/398216 | 09:18 |
| *** dsariel has quit IRC | 09:18 | |
| *** karthiks has joined #tripleo | 09:19 | |
| shardy | jaosorior: lgtm but I suspect you don't need the Cfn or Cloudwatch services | 09:19 |
| *** numans has joined #tripleo | 09:19 | |
| *** arxcruz has quit IRC | 09:21 | |
| shardy | jpich: Good morning - re https://blueprints.launchpad.net/tripleo/+spec/tripleo-ui-basic-ci-check | 09:23 |
| shardy | https://review.openstack.org/#/c/392215/ landed, can we mark that implemented? | 09:23 |
| *** rbowen has joined #tripleo | 09:24 | |
| shardy | Ah https://review.openstack.org/#/c/390845/ is still pending | 09:24 |
| jaosorior | shardy: I don't> | 09:24 |
| jaosorior | ? | 09:24 |
| *** arxcruz has joined #tripleo | 09:24 | |
| jaosorior | uhm... alright, I'll remove them and lets see how that goes | 09:24 |
| shardy | jaosorior: pingtest doesn't use those services AFAIK | 09:24 |
| shardy | so if you want to be super minimal you can probably remove them | 09:25 |
| jaosorior | so I shall | 09:25 |
| jaosorior | thanks | 09:25 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: Create directories with root https://review.openstack.org/384892 | 09:26 |
| jpich | shardy: Yeah the actual check at https://review.openstack.org/#/c/390845/ isn't merged yet, only one +2. Sorry this wasn't clear, if you have tips on how one may highlight pending changes better in blueprints I'll arrange them differently in the future :) | 09:26 |
| shardy | jpich: Cool, thanks I just missed it - now approved | 09:26 |
| shardy | when that lands can we call this complete? | 09:26 |
| jpich | shardy: Indeed | 09:26 |
| shardy | great, thanks! | 09:26 |
| jpich | shardy: Thanks! | 09:27 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add necessary parameters for encrypted volumes support https://review.openstack.org/398177 | 09:28 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Fix scenario004 deployment https://review.openstack.org/398124 | 09:29 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci: Use encrypted volume in scenario004 https://review.openstack.org/398221 | 09:29 |
| b00tcat | hey, can I get +workflow here? https://review.openstack.org/#/c/394889/ | 09:30 |
| *** lucas-afk is now known as lucasagomes | 09:32 | |
| *** gfidente has joined #tripleo | 09:32 | |
| shardy | b00tcat: Hi! lgtm, done | 09:32 |
| *** dtantsur is now known as dtantsur|afk | 09:33 | |
| b00tcat | ;) | 09:33 |
| *** paramite has joined #tripleo | 09:33 | |
| *** dtantsur|afk is now known as dtantsur|bbl | 09:33 | |
| jaosorior | couple of easy commits if someone has time https://review.openstack.org/#/c/398128/ https://review.openstack.org/#/c/398127/ | 09:36 |
| *** sshnaidm|away is now known as sshnaidm | 09:37 | |
| *** saneax-_-|AFK is now known as saneax | 09:38 | |
| shardy | panda: Hi! What's the status of https://bugs.launchpad.net/tripleo/+bug/1638350 ? | 09:38 |
| openstack | Launchpad bug 1638350 in tripleo "pingtest failing on OVB jobs to create Cinder volume and Nova server with 504 error" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 09:38 |
| openstackgerrit | Martin Mágr proposed openstack/python-tripleoclient: Use correct region value https://review.openstack.org/398224 | 09:38 |
| shardy | I see several related fixes, but I need to either close it or defer to ocata-2 | 09:38 |
| openstackgerrit | Adriano Petrich proposed openstack/python-tripleoclient: Give better output on scale failures https://review.openstack.org/398226 | 09:40 |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: Create directories with root https://review.openstack.org/384892 | 09:41 |
| *** derekh has joined #tripleo | 09:41 | |
| sshnaidm | derekh, hi | 09:47 |
| sshnaidm | derekh, do you know if we have dns name for mirror server? | 09:47 |
| derekh | sshnaidm: none that I'm aware of | 09:48 |
| sshnaidm | ok | 09:48 |
| derekh | sshnaidm: dprince controls tripleo.org , if we need a DNS entry for it then he may be able to add a subdomain | 09:48 |
| sshnaidm | derekh, ok, thanks | 09:49 |
| openstackgerrit | Merged openstack/os-net-config: Add support for name replacement in OVS_EXTRA https://review.openstack.org/395795 | 09:49 |
| openstackgerrit | Merged openstack/os-net-config: Add ovs_fail_mode option for OVS bridges https://review.openstack.org/397405 | 09:49 |
| *** bogdando has quit IRC | 09:50 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: DONT MERGE: Test quickstart experimental job https://review.openstack.org/397609 | 09:50 |
| *** bogdando has joined #tripleo | 09:52 | |
| openstackgerrit | Martin Mágr proposed openstack/python-tripleoclient: Use correct region value https://review.openstack.org/398234 | 09:52 |
| *** yamahata has joined #tripleo | 09:54 | |
| jaosorior | d0ugal: hey, could you check this out? https://review.openstack.org/#/c/387430/ | 09:55 |
| d0ugal | jaosorior: sure | 09:56 |
| *** limao has quit IRC | 09:57 | |
| panda | shardy: no progress, we're seeing this in rdo ci too, but all searches are inconclusive, logs don't show any evidence of error. It doesn't happen often AFAIK, so it's even difficult to reproduce. | 09:57 |
| *** saneax is now known as saneax-_-|AFK | 09:59 | |
| shardy | panda: Ok - could you add a note with the latest status to the bug and defer to ocata-2 please? | 10:01 |
| shardy | I'm trying to clean all ocata-1 stuff so I can propose the release today | 10:02 |
| *** milan has joined #tripleo | 10:02 | |
| *** dbecker has quit IRC | 10:04 | |
| panda | shardy: sure | 10:04 |
| *** yamahata has quit IRC | 10:07 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 10:07 |
| openstackgerrit | Marios Andreou proposed openstack/os-net-config: Add support for name replacement in OVS_EXTRA https://review.openstack.org/398242 | 10:08 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-docs: Add documentation on how to use an external Ceph cluster https://review.openstack.org/397825 | 10:10 |
| *** hewbrocca is now known as hewbrocca_afk | 10:10 | |
| openstackgerrit | Marios Andreou proposed openstack/os-net-config: Add ovs_fail_mode option for OVS bridges https://review.openstack.org/398245 | 10:13 |
| *** dsariel has joined #tripleo | 10:24 | |
| *** udesale has quit IRC | 10:26 | |
| shardy | bandini: Hi, good morning! Can we close https://bugs.launchpad.net/tripleo/+bug/1628998 or should it be deferred to ocata-2 ? | 10:27 |
| openstack | Launchpad bug 1628998 in tripleo "Change rabbitmq queues HA mode from ha-all to ha-exactly" [High,In progress] - Assigned to Michele Baldessari (michele) | 10:27 |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 10:32 |
| bandini | shardy: if we could land https://review.openstack.org/#/c/389309/ then it can be closed | 10:38 |
| shardy | marios: Hey when you get a moment can I please get a review/+A on https://review.openstack.org/#/c/397696/ | 10:38 |
| shardy | final patch to close that bug for ocata-1 I think | 10:38 |
| shardy | marios: ^^ also the patch from bandini please as I've already +2'd it | 10:39 |
| bandini | thanks! | 10:39 |
| shardy | bandini: ack, thanks, lets see if we can land those two | 10:39 |
| bandini | aye | 10:39 |
| shardy | bandini: I had a question regarding composable-ha | 10:40 |
| shardy | what will upgrades look like in that case? | 10:40 |
| openstackgerrit | Merged openstack/tripleo-puppet-elements: Enable MidoNet element to install MEM https://review.openstack.org/394889 | 10:40 |
| marios | beagles: heh looks like we're updating that BZ tandem style | 10:40 |
| marios | shardy: ack | 10:40 |
| shardy | e.g for https://review.openstack.org/#/c/393448/ I'm planning to simply run the current pcs stuff on the node running pacemaker | 10:40 |
| beagles | marios: lol yup.. I'll let you have it. I haven't had any coffee yet so I'm high risk :) | 10:41 |
| *** brault has quit IRC | 10:41 | |
| shardy | but I'm not quite clear what it will look like when we've got composable HA in place e.g the O->P upgrade | 10:41 |
| marios | beagles: no its done it was just 'opendaylight' gerrit (the number of times i've done that... :/) | 10:41 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Do not manage overcloud repositories when using external Ceph https://review.openstack.org/397819 | 10:42 |
| bandini | shardy: I think conceptually it will be very similar to what we do today. Aka we can drive all pcs calls from a single node (bootstrap one) | 10:42 |
| beagles | marios: lol.. see what I mean :) | 10:42 |
| *** rbowen has quit IRC | 10:42 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Do not manage overcloud repositories when using external Ceph https://review.openstack.org/397819 | 10:42 |
| bandini | shardy: although I still have to look at your WIP properly. Just did not get around doing it yet | 10:42 |
| shardy | bandini: Ok, so we can probably just wire in the logic e.g via puppet/services/pacemaker.yaml | 10:43 |
| bandini | shardy: I believo so, yes | 10:43 |
| shardy | and have it run either on all the nodes in the cluster, or just the bootstrap node | 10:43 |
| bandini | yeah | 10:43 |
| shardy | bandini: Ok thanks, that fits pretty well with my prototype | 10:43 |
| bandini | shardy: great :) | 10:43 |
| *** brault has joined #tripleo | 10:45 | |
| shardy | all: please update the delivery status of blueprints targetted at ocata-2 | 10:45 |
| shardy | https://launchpad.net/tripleo/+milestone/ocata-2 | 10:45 |
| shardy | we have 23 incomplete blueprints, nearly all "unknown" status, and only a month to land them :-O | 10:46 |
| *** owalsh has quit IRC | 10:46 | |
| d0ugal | yikes :) | 10:47 |
| *** dbecker has joined #tripleo | 10:50 | |
| *** abregman is now known as abregman|afk | 10:51 | |
| *** adarazs is now known as adarazs_lunch | 10:56 | |
| jpich | shardy: Looks like https://blueprints.launchpad.net/tripleo/+spec/enable-communication-ui-undercloud actually merged in ocata-1, I'll update appropriately | 10:56 |
| shardy | jpich: ack, sounds good, thanks! | 10:56 |
| *** tosky has joined #tripleo | 10:57 | |
| shardy | 1 down, 39 to go (there's also 17 targetted at ocata-3) | 10:57 |
| jpich | ouch | 10:58 |
| jpich | florianf: Are you ok assigning yourself to https://blueprints.launchpad.net/tripleo/+spec/tripleo-ui-i18n-support-for-js and keeping the bp up to date, based on our last chat about it? I think the status can be set to "Started" at least? | 11:00 |
| shadower | shardy: what about blueprints that still have open specs? I've a couple of validation improvements I'd like to add that haven't merged yet | 11:00 |
| shadower | I've created the blueprints but left them intentionally empty for now | 11:01 |
| shardy | shadower: are the blueprints correctly targeted to ocata-* ? | 11:01 |
| shadower | let me check | 11:01 |
| shardy | shadower: in the meeting this week I suggested we excercise some discretion with open specs - the review rate has been poor, so I don't think we can enforce a spec freeze as originally planned | 11:02 |
| shardy | but I think it would be reasonable to refuse any which aren't already proposed at this point | 11:02 |
| shardy | the backlog of work we have already is highly likely to be too much for this cycle IMO | 11:03 |
| shadower | shardy: the blueprints aren't targeted yet: https://blueprints.launchpad.net/tripleo/+spec/validation-sources and https://blueprints.launchpad.net/tripleo/+spec/validations-in-workflows | 11:03 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates: Use default Sensu redact https://review.openstack.org/398281 | 11:03 |
| shadower | shardy: I think I left them as they were waiting for spec feedback | 11:03 |
| jpich | honza: I don't see https://blueprints.launchpad.net/tripleo/+spec/websocket-logging in the list of blueprints for ocata (e.g. https://launchpad.net/tripleo/+milestone/ocata-2 https://launchpad.net/tripleo/+milestone/ocata-3 ), if we're still planning to land this in Ocata the milestone field should be set, and the status/assignee other fields as needed too :) (see ^) | 11:03 |
| florianf | jpich: sure, done. thanks for reminding me! | 11:04 |
| jpich | florianf: Thanks! :) | 11:05 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-heat-templates: Fix typo in Keystone Sensu subscription https://review.openstack.org/398285 | 11:06 |
| *** hewbrocca_afk is now known as hewbrocca | 11:06 | |
| *** flepied has quit IRC | 11:09 | |
| openstackgerrit | Adriano Petrich proposed openstack/python-tripleoclient: Use stack name or id for backwards compatibility https://review.openstack.org/398289 | 11:11 |
| openstackgerrit | Brent Eagles proposed openstack/os-net-config: Add support for enabling hotplug on interfaces https://review.openstack.org/394660 | 11:11 |
| shardy | shadower: Ok I'll target them, added some comments to https://review.openstack.org/#/c/393775 | 11:12 |
| shardy | shadower: I'd suggest adding basic description to the BPs at least | 11:12 |
| shadower | shardy: thanks | 11:13 |
| *** hewbrocca is now known as hewbrocca_afk | 11:17 | |
| *** hewbrocca_afk is now known as hewbrocca | 11:17 | |
| *** athomas has quit IRC | 11:18 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Nova base cleanups for hiera json hook https://review.openstack.org/397974 | 11:20 |
| openstackgerrit | Adriano Petrich proposed openstack/python-tripleoclient: Use stack name or id for backwards compatibility https://review.openstack.org/398289 | 11:20 |
| *** athomas has joined #tripleo | 11:22 | |
| jaosorior | shardy: can you revisit this? https://review.openstack.org/#/c/398124/ | 11:23 |
| shardy | jaosorior: done | 11:24 |
| jaosorior | thanks | 11:25 |
| *** jaosorior is now known as jaosorior_lunch | 11:25 | |
| *** abregman|afk is now known as abregman | 11:29 | |
| *** jkilpatr has quit IRC | 11:33 | |
| openstackgerrit | Joe Talerico proposed openstack/tripleo-heat-templates: Disable Neutron agents with OVN. https://review.openstack.org/388088 | 11:34 |
| d0ugal | Is ovb-ha broken? | 11:36 |
| d0ugal | seems a few did pass earlier | 11:37 |
| d0ugal | trying to figure out: http://logs.openstack.org/83/395083/6/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/744e37a/console.html#_2016-11-16_11_16_53_182479 | 11:37 |
| d0ugal | Any ideas? | 11:37 |
| panda | d0ugal: http://logs.openstack.org/83/395083/6/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/744e37a/logs/postci.txt.gz#_2016-11-16_11_20_17_000 | 11:38 |
| panda | d0ugal: 2016-11-16 11:20:17.000 | mError: /Stage[main]/Apache::Service/Service[httpd]: Failed to call refresh: Could not restart Service[httpd]: Execution of '/usr/bin/systemctl restart httpd' returned 1: Job for httpd.service failed because a timeout was exceeded. See "systemctl status httpd.service" and "journalctl -xe" for details.m | 11:39 |
| panda | 2016-11-16 11:20:17.000 | mError: /Stage[main]/Apache::Service/Service[httpd]: Could not restart Service[httpd]: Execution of '/usr/bin/systemctl restart httpd' returned 1: Job for httpd.service failed because a timeout was exceeded. See "systemctl status httpd.service" and "journalctl -xe" for details.m | 11:39 |
| shardy | Yeah you need to look in the postci log, httpd couldn't start | 11:39 |
| panda | 2 | 11:39 |
| panda | d0ugal: I think you hit this https://bugs.launchpad.net/tripleo/+bug/1640879 | 11:40 |
| openstack | Launchpad bug 1640879 in tripleo "CI: apache fails to start on overcloud controller" [High,Triaged] | 11:40 |
| d0ugal | panda: thanks | 11:42 |
| *** dtantsur|bbl is now known as dtantsur | 11:44 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 11:46 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Stop using documentation network range https://review.openstack.org/395529 | 11:51 |
| *** flepied has joined #tripleo | 11:51 | |
| honza | jpich: the gui logging spec is up for review but hasn't been approved yet https://review.openstack.org/#/c/395138/ | 11:52 |
| *** masco has quit IRC | 11:53 | |
| sshnaidm | derekh, could you please take a look at these patches in your time? It's related to changing default network range in tripleo. https://review.openstack.org/#/c/395702/ https://review.openstack.org/#/c/397115/ thanks | 11:53 |
| jpich | honza: Alright, cool! I forgot about the spec. I believe the blueprint is still be what'll be used for tracking during the release so, add a link to the spec and set the other fields as needed, especially the milestone? shardy mentioned there was a bit of a backlog in specs review | 11:55 |
| *** bogdando has quit IRC | 11:55 | |
| honza | jpich: thanks for being on top of this --- stuff like this still eludes me; i'll update it a bit and set the milestone today | 11:57 |
| *** bogdando has joined #tripleo | 11:57 | |
| *** fzdarsky is now known as fzdarsky|lunch | 11:58 | |
| sshnaidm | panda, d0ugal it happens more and more | 11:58 |
| sshnaidm | but very rarely reproducible.. | 11:58 |
| sshnaidm | mostly because it fails without logs, I suspect httpd process can not create /var/log/httpd, because of permissions or selinux | 11:59 |
| jpich | honza: Cool, thank you! I know not everyone working on the ui is necessarily super familiar with that stuff so I'm trying to poke proactively, for the bits I know :) | 11:59 |
| *** dprince has joined #tripleo | 12:01 | |
| *** owalsh has joined #tripleo | 12:03 | |
| d0ugal | sshnaidm: it seemed to happen twice in a row here: https://review.openstack.org/#/c/395083/ | 12:03 |
| *** prateek has joined #tripleo | 12:04 | |
| *** fultonj has joined #tripleo | 12:04 | |
| derekh | sshnaidm: willdo | 12:04 |
| openstackgerrit | Adriano Petrich proposed openstack/python-tripleoclient: Give better output on scale failures https://review.openstack.org/398226 | 12:04 |
| *** coolsvap is now known as coolsvap_ | 12:09 | |
| *** jaosorior_lunch is now known as jaosorior | 12:09 | |
| *** lucasagomes is now known as lucas-hungry | 12:11 | |
| *** shardy is now known as shardy_lunch | 12:11 | |
| *** jkilpatr has joined #tripleo | 12:12 | |
| jaosorior | panda, sshnaidm well, there is a funky selinux selinux failure in the logs (I've seen that bug today too): type=SERVICE_START msg=audit(1479295373.916:1715): pid=1 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:init_t:s0 msg='unit=httpd comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed' | 12:12 |
| jaosorior | http://logs.openstack.org/24/398124/3/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/5024eef/logs/overcloud-controller-1/var/log/audit/audit.txt.gz | 12:12 |
| *** psanchez has quit IRC | 12:13 | |
| panda | jaosorior: nica | 12:14 |
| panda | jaosorior: nice, even | 12:14 |
| *** pkovar has quit IRC | 12:15 | |
| *** dprince has quit IRC | 12:15 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Replace ceilometer-dbsync by ceilometer-upgrade https://review.openstack.org/397696 | 12:15 |
| *** rook-out has quit IRC | 12:15 | |
| *** psanchez has joined #tripleo | 12:15 | |
| *** dprince has joined #tripleo | 12:16 | |
| *** rook has joined #tripleo | 12:18 | |
| *** rook is now known as Guest45665 | 12:18 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Sort parameters in keystone profile alphabetically https://review.openstack.org/398323 | 12:21 |
| jaosorior | ccamacho: done ^^ (didn't wanna mess with the CI results in the previous patch) | 12:21 |
| ccamacho | awesome! | 12:22 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/puppet-tripleo: Sort parameters in keystone profile alphabetically https://review.openstack.org/398323 | 12:24 |
| jaosorior | ccamacho: forgot to fix the order in the documentation as well ^^ | 12:24 |
| openstackgerrit | Merged openstack/instack-undercloud: Fix pep8 job https://review.openstack.org/397955 | 12:24 |
| jaosorior | dprince: could you check this out? https://review.openstack.org/#/c/398221/ | 12:26 |
| jaosorior | dprince: sory, this one https://review.openstack.org/#/c/398124/3 | 12:26 |
| jaosorior | thrash: hey dude | 12:26 |
| jaosorior | thrash: I accidentally messed up scenario004, since I missed deploying heat and configuring glance's backend, here's the fix https://review.openstack.org/#/c/398124/3 | 12:27 |
| thrash | jaosorior: haha | 12:27 |
| dprince | jaosorior: ack | 12:27 |
| thrash | jaosorior: cool. thanks for that. | 12:27 |
| thrash | jaosorior: I was wondering wtf was going on with that job. | 12:27 |
| thrash | jaosorior: I'll rebase mine on top | 12:28 |
| jaosorior | thrash: or we could merge that patch :d | 12:28 |
| thrash | jaosorior: haha | 12:28 |
| thrash | jaosorior: looks like newton failed.. | 12:29 |
| jaosorior | thrash: cause ovb-ha job is flaky, the ovb jobs don't use the scenario files at all | 12:29 |
| thrash | jaosorior: true | 12:29 |
| thrash | jaosorior: looks like it's on its way to merging. | 12:30 |
| *** adarazs_lunch is now known as adarazs | 12:31 | |
| jaosorior | thrash: could you check this one out? https://review.openstack.org/#/c/387430/ | 12:31 |
| jaosorior | dprince: thanks dude | 12:32 |
| thrash | jaosorior: ack | 12:32 |
| dprince | jaosorior: np | 12:34 |
| *** owalsh has quit IRC | 12:40 | |
| *** owalsh has joined #tripleo | 12:40 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 12:42 |
| openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart: Add configuration of HA IPv6 deployments https://review.openstack.org/380358 | 12:42 |
| *** shardy_lunch is now known as shardy | 12:45 | |
| *** anshul has quit IRC | 12:45 | |
| *** owalsh has quit IRC | 12:46 | |
| *** pkovar has joined #tripleo | 12:48 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Fix up Newton->Ocata rabbitmq ha policy https://review.openstack.org/389309 | 12:49 |
| *** leanderthal|afk is now known as leanderthal | 12:52 | |
| *** pradk has joined #tripleo | 12:55 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Full quickstart gate run on OVB https://review.openstack.org/381094 | 12:56 |
| *** tzumainn has joined #tripleo | 12:56 | |
| *** anshul has joined #tripleo | 12:57 | |
| *** morazi has joined #tripleo | 13:00 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: DONT MERGE: Test quickstart experimental job https://review.openstack.org/397609 | 13:00 |
| *** ccamacho is now known as ccamacho|lunch | 13:04 | |
| *** mhenkel has joined #tripleo | 13:06 | |
| *** morazi has quit IRC | 13:07 | |
| *** morazi has joined #tripleo | 13:08 | |
| *** prateek has quit IRC | 13:10 | |
| *** trown|outtypewww is now known as trown | 13:11 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Nothing to see here https://review.openstack.org/111011 | 13:12 |
| *** pgadiya has quit IRC | 13:12 | |
| *** rhallisey has joined #tripleo | 13:15 | |
| *** brault has quit IRC | 13:17 | |
| *** brault has joined #tripleo | 13:18 | |
| *** lucas-hungry is now known as lucasagomes | 13:20 | |
| *** jayg|g0n3 is now known as jayg | 13:25 | |
| *** bkopilov_ is now known as bkopilov | 13:28 | |
| *** fzdarsky|lunch is now known as fzdarsky | 13:29 | |
| *** brault has quit IRC | 13:30 | |
| openstackgerrit | Merged openstack/tripleo-specs: oooq as the tool set for tripleo CI https://review.openstack.org/386250 | 13:32 |
| *** abregman is now known as abregman|mtg | 13:32 | |
| *** bfournie has quit IRC | 13:33 | |
| openstackgerrit | Julie Pichon proposed openstack/instack-undercloud: Increase Mistral Task Size limit https://review.openstack.org/396523 | 13:33 |
| *** brault has joined #tripleo | 13:35 | |
| *** rlandy has joined #tripleo | 13:37 | |
| *** anshul has quit IRC | 13:39 | |
| shadower | shardy: just a sanity check: if there are parameters set in parameter_defaults, are they going to show up in "heat stack-show overcloud" | 13:41 |
| shardy | shadower: not unless the overcloud.yaml happens to consume them | 13:41 |
| shardy | shadower: try openstack stack environment show overcloud | 13:41 |
| shadower | shardy: ah, thanks! | 13:42 |
| shadower | shardy: this is the extra admin ssh keys one so it's not consumed by overcloud.yaml | 13:42 |
| *** rodrigods has quit IRC | 13:42 | |
| shardy | shadower: ah, yeah you'll only see it in the stack-show for the nested stack | 13:42 |
| *** rodrigods has joined #tripleo | 13:42 | |
| shardy | where the parameter is defined | 13:42 |
| shardy | environment show should expose it tho | 13:42 |
| shadower | thanks | 13:43 |
| *** jpena is now known as jpena|lunch | 13:43 | |
| weshay | shardy, quick question.. trown and I are looking at your composable services 101 page.. I have a deployment for keystone only | 13:52 |
| *** amoralej is now known as amoralej|lunch | 13:52 | |
| weshay | seems like heat is getting pulled in | 13:52 |
| weshay | nnect failed\n\u001b[1;31mError: Could not find data item heat::keystone::domain::domain_password in any Hiera data file and no default supplied at /etc/puppet/modules/tripleo/manifests/profile/base/keystone.pp:108 on node overcloud-controller-0.localdomain\u001b[0m\n\u001b[1;31mError: Could not find data item heat::keystone::domain::domain_password in any Hiera data file and no default supplied at /etc/puppet/modules/tripleo/manifests/pro | 13:52 |
| weshay | file/base/keystone.pp:108 on node overcloud-controller-0.localdomain\u001b[0m\n | 13:52 |
| jaosorior | weshay: fixing that | 13:53 |
| weshay | jaosorior, which the composable services? | 13:53 |
| jaosorior | weshay: https://review.openstack.org/#/c/398128/ https://review.openstack.org/#/c/398127/ | 13:53 |
| jaosorior | weshay: the keystone hiera issue | 13:53 |
| trown | ah nice | 13:53 |
| weshay | jaosorior++ | 13:53 |
| weshay | rockin | 13:53 |
| jaosorior | need reviews though | 13:54 |
| trown | jaosorior: working on it :) | 13:54 |
| jaosorior | trown: can't merge the puppet-tripleo yet since it depends on the t-h-t patch | 13:54 |
| openstackgerrit | Brad P. Crochet proposed openstack/python-tripleoclient: Do not just return when calling legacy image build https://review.openstack.org/398364 | 13:54 |
| *** anshul has joined #tripleo | 13:55 | |
| shardy | weshay: yeah we got the heat->keystone dependency reversed in the puppet profiles unfortunately | 13:55 |
| shardy | thanks jaosorior for looking into it | 13:55 |
| *** fpan_ has joined #tripleo | 13:55 | |
| trown | so just need another +2 on https://review.openstack.org/#/c/398127/1 and we are good I think | 13:55 |
| *** abregman_ has joined #tripleo | 13:56 | |
| *** udesale has joined #tripleo | 13:56 | |
| shardy | trown: done | 13:56 |
| *** tbonds has joined #tripleo | 13:57 | |
| *** fpan has quit IRC | 13:57 | |
| jaosorior | weshay: that issue was in master, right? | 13:57 |
| weshay | jaosorior, yes sir | 13:57 |
| weshay | trown, I'll submit this work then | 13:57 |
| jaosorior | alright, it'll be fixed soon | 13:57 |
| trown | weshay: wouldnt that go somewhere in the new tripleo-quickstart-extras repo? | 13:58 |
| trown | might want to wait until that is fully up | 13:58 |
| weshay | trown, ya.. I have it in prep-config and added one config in config/general_config/ | 13:59 |
| *** abregman|mtg has quit IRC | 13:59 | |
| weshay | trown, ya.. just going to get a review out there so it's visible | 13:59 |
| weshay | for comment etc | 13:59 |
| trown | weshay: k, adarazs ^ | 13:59 |
| *** bfournie has joined #tripleo | 14:02 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-common: DONT REVIEW: test https://review.openstack.org/398367 | 14:02 |
| *** abregman_ has quit IRC | 14:04 | |
| *** abregman has joined #tripleo | 14:05 | |
| *** Goneri has joined #tripleo | 14:06 | |
| sshnaidm | derekh, ^^ | 14:07 |
| dtantsur | hey! may I have 2nd +2 on https://review.openstack.org/#/c/397112/ please? | 14:07 |
| trown | dtantsur: looking | 14:07 |
| *** ctayal has joined #tripleo | 14:08 | |
| *** tiswanso has joined #tripleo | 14:09 | |
| *** rlandy is now known as rlandy|mtg | 14:09 | |
| *** Guest45665 is now known as rook | 14:13 | |
| *** fpan_ has quit IRC | 14:13 | |
| *** jcoufal has joined #tripleo | 14:14 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Default $STABLE_RELEASE in bootstrap-overcloud-full.sh https://review.openstack.org/385159 | 14:14 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Add configurable mtu to --mulitnode-setup https://review.openstack.org/385162 | 14:15 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Add --setup-nodepool-files option https://review.openstack.org/385160 | 14:15 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Add 3 node multinode job https://review.openstack.org/397441 | 14:15 |
| *** prateek has joined #tripleo | 14:17 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: add a config for a keystone-only overcloud deployment https://review.openstack.org/398375 | 14:18 |
| *** jprovazn has quit IRC | 14:20 | |
| *** lblanchard has joined #tripleo | 14:21 | |
| *** fpan_ has joined #tripleo | 14:26 | |
| *** ccamacho|lunch is now known as ccamacho | 14:28 | |
| *** jeckersb_gone is now known as jeckersb | 14:28 | |
| jaosorior | derekh: hey, wouldn't this just work with all branches? https://review.openstack.org/#/c/395529/5/scripts/deploy.sh so then it's fine that he didn't make the change in tripleo.sh branch dependant | 14:30 |
| *** oshvartz has quit IRC | 14:30 | |
| *** oshvartz has joined #tripleo | 14:31 | |
| *** amoralej|lunch is now known as amoralej | 14:32 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Add necessary parameters for encrypted volumes support https://review.openstack.org/398177 | 14:33 |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 14:34 |
| openstackgerrit | Adriano Petrich proposed openstack/python-tripleoclient: Give better output on scale failures https://review.openstack.org/398226 | 14:38 |
| shardy | mwhahaha: Hey can you please confirm the release workflow for puppet-tripleo metadata.json - I bump the metadata.json, merge it, then propose the release to openstack/releases, right? | 14:40 |
| shardy | it's a bit confusing as we're tracking the version numbers in two places | 14:40 |
| shardy | afaics that's what EmilienM did for the newton releases | 14:40 |
| mwhahaha | K | 14:40 |
| *** jaosorior has quit IRC | 14:41 | |
| *** rhallisey has quit IRC | 14:41 | |
| *** jaosorior has joined #tripleo | 14:42 | |
| openstackgerrit | Steven Hardy proposed openstack/puppet-tripleo: Release 6.0.0 https://review.openstack.org/398387 | 14:43 |
| shardy | mwhahaha, slagle: ^^ | 14:43 |
| mwhahaha | shardy: yes, that is the correct order to do things | 14:44 |
| shardy | I'll propose the openstack/releases patch after https://review.openstack.org/#/c/397819 lands | 14:44 |
| shardy | and that metadata.json bump | 14:44 |
| shardy | mwhahaha: thanks for confirming | 14:44 |
| *** jaosorior has quit IRC | 14:47 | |
| *** morazi has quit IRC | 14:47 | |
| *** rlandy|mtg is now known as rlandy | 14:49 | |
| *** morazi has joined #tripleo | 14:52 | |
| matbu | panda: i remember you were fighting with an issue in CI (badstatusline), do you remember the root cause ? | 14:53 |
| openstackgerrit | Steven Hardy proposed openstack/instack-undercloud: correctly spell yaql_limit_iterators https://review.openstack.org/398396 | 14:53 |
| *** abregman has quit IRC | 14:54 | |
| panda | matbu: wasn't me, but I think there's a fix proposed for that, let me check | 14:54 |
| matbu | panda: thanks | 14:55 |
| panda | matbu: well, it looks like a complicated issue https://bugs.launchpad.net/tripleo/+bug/1638908 | 14:55 |
| openstack | Launchpad bug 1638908 in tripleo-quickstart "Overcloud deployment fails in minimal configuration with ('Connection aborted.', BadStatusLine("''",))" [Undecided,In progress] - Assigned to Alfredo Moralejo (amoralej) | 14:55 |
| matbu | panda: great thanks | 14:56 |
| *** jeckersb is now known as jeckersb_gone | 14:58 | |
| openstackgerrit | Steven Hardy proposed openstack/instack-undercloud: correctly spell yaql_limit_iterators https://review.openstack.org/398396 | 14:59 |
| *** coolsvap_ is now known as coolsvap | 14:59 | |
| *** jcoufal has quit IRC | 14:59 | |
| social | marios: just to ensure https://review.openstack.org/#/c/398245/ is going to mitaka? | 14:59 |
| marios | social: thats the idea i believe | 15:00 |
| marios | social: so it needs to go to newton first | 15:00 |
| marios | social: master landed today | 15:00 |
| *** jcoufal has joined #tripleo | 15:01 | |
| *** jpena|lunch is now known as jpena | 15:01 | |
| *** oshvartz has quit IRC | 15:01 | |
| *** jeckersb_gone is now known as jeckersb | 15:02 | |
| *** tbonds has quit IRC | 15:05 | |
| ansiwen | my gut feeling is that 80% of CI load is caused by rechecks... | 15:08 |
| bnemec | social: marios: There is no mitaka branch for os-net-config. | 15:08 |
| *** ealcaniz has quit IRC | 15:09 | |
| marios | bnemec: heh beagles that will be a problem | 15:09 |
| beagles | oof | 15:09 |
| social | bnemec: what about branching it from commit used by package build and than cherrypick? | 15:09 |
| *** abregman has joined #tripleo | 15:16 | |
| *** prateek has quit IRC | 15:18 | |
| jpich | florianf: Did you ever find a resolution for your deployments hanging with a lot of sleep statement in the heat logs, in the end? | 15:19 |
| florianf | jpich: It seems like I just was too impatient | 15:19 |
| jpich | florianf: Oh, I thought you waited ages | 15:20 |
| florianf | jpich: I re-tried the same deployment overnight and it failed at some point | 15:20 |
| jpich | florianf: I see. All my deployments seem to get stuck at the "Controller - CREATE_IN_PROGRESS" stage today... but I cancelled them all in the end so I guess I'll try the "be patient" strategy too. Thanks for the tip :-) | 15:21 |
| shardy | matbu, panda: I think the working theory is those errors are caused by some services responding so slowly they exceed the haproxy timeout | 15:21 |
| shardy | we increased the haproxy timeout to $large number to work around it, but we're still trying to figure out why some services are taking over 1minute to respond | 15:21 |
| *** bana_k has joined #tripleo | 15:22 | |
| shardy | https://review.openstack.org/#/c/397822/ is about to land to stable/newton which should, in theory, help | 15:22 |
| florianf | jpich: So for one of them I debugged on one of the controller nodes and it looked like it couldn't connect to clock.redhat.com (HA + net iso) and at some point timed out because of that | 15:22 |
| matbu | shardy: yes agree, because in my case, the increase timeout for swift doesn't fix it | 15:23 |
| matbu | shardy: it probably something else | 15:23 |
| matbu | shardy: but nothing in the log :| | 15:23 |
| jpich | florianf: Oh. Gaaah I bet you the vpn went down on that machine | 15:23 |
| shardy | matbu: the problem is that error is a pretty meaningless generic error from python | 15:23 |
| *** Goneri has quit IRC | 15:23 | |
| shardy | matbu: and last time I looked we don't enable any logs for swift at all | 15:23 |
| matbu | shardy: yep | 15:23 |
| florianf | jpich: Or some net iso misconfiguration on my side. | 15:24 |
| jpich | florianf: You are my hero and I wish I'd asked you 5h ago | 15:24 |
| shardy | https://review.rdoproject.org/etherpad/p/badstatusline-issue has some additional data FYI | 15:24 |
| shardy | matbu: ^^ | 15:25 |
| matbu | shardy: cool thanks | 15:25 |
| florianf | jpich: what? :-) what did I do? :-) | 15:25 |
| *** absubram has joined #tripleo | 15:25 | |
| florianf | jpich: But happy to help :-D | 15:25 |
| jpich | florianf: I'm 99.99% sure that's what it is, I didn't hook the host to the vpn when I brought it up this morning but still set the ntp server to an internal URL | 15:26 |
| jpich | florianf: I need to become more efficient at debugging that stuff | 15:27 |
| florianf | jpich: oh, that makes sense | 15:28 |
| bnemec | dsneddon: I believe predictable placement goes all the way back to liberty. See the passing jobs on https://review.openstack.org/#/c/378040 | 15:28 |
| *** rajinir has joined #tripleo | 15:30 | |
| *** jprovazn has joined #tripleo | 15:30 | |
| *** michapma_alt has quit IRC | 15:35 | |
| *** Goneri has joined #tripleo | 15:35 | |
| *** udesale has quit IRC | 15:35 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-common: DONT REVIEW: test https://review.openstack.org/398367 | 15:36 |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common: Make setting nodes to manageable a discrete workflow https://review.openstack.org/398422 | 15:43 |
| openstackgerrit | Brad P. Crochet proposed openstack/python-tripleoclient: Make setting nodes to manageable a discrete workflow https://review.openstack.org/398423 | 15:43 |
| *** morazi has quit IRC | 15:44 | |
| openstackgerrit | Markos Chandras proposed openstack/diskimage-builder: lib: common-functions: Fix tmpfs umounting https://review.openstack.org/392002 | 15:44 |
| *** ealcaniz has joined #tripleo | 15:46 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-common: DONT REVIEW: test https://review.openstack.org/398367 | 15:46 |
| *** coolsvap is now known as coolsvap_ | 15:46 | |
| openstackgerrit | Ronelle Landy proposed openstack/tripleo-quickstart: WIP: Modify OVB ci-script to tripleo-quickstart-extras https://review.openstack.org/398429 | 15:48 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Fix scenario004 deployment https://review.openstack.org/398124 | 15:49 |
| openstackgerrit | Merged openstack/python-tripleoclient: Enable creating images with Glance v2 https://review.openstack.org/387430 | 15:49 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-common: DONT REVIEW: test newton https://review.openstack.org/398432 | 15:50 |
| *** abregman has quit IRC | 15:55 | |
| *** morazi has joined #tripleo | 15:57 | |
| openstackgerrit | yolanda.robla proposed openstack/python-tripleoclient: Support whole disk images in TripleO https://review.openstack.org/394426 | 15:57 |
| *** coolsvap_ is now known as coolsvap | 16:00 | |
| *** leanderthal is now known as leanderthal|afk | 16:00 | |
| *** rhallisey__ has joined #tripleo | 16:00 | |
| *** abregman has joined #tripleo | 16:01 | |
| *** anshul has quit IRC | 16:01 | |
| *** jlinkes has quit IRC | 16:02 | |
| openstackgerrit | yolanda.robla proposed openstack/python-tripleoclient: Support whole disk images in TripleO https://review.openstack.org/394426 | 16:05 |
| openstackgerrit | Merged openstack/tripleo-docs: Remove usage of --compute-scale from the baremetal overcloud docs https://review.openstack.org/381454 | 16:05 |
| *** ebarrera has quit IRC | 16:05 | |
| openstackgerrit | Merged openstack/tripleo-docs: Remove usage of --control-scale from the HA docs https://review.openstack.org/381458 | 16:06 |
| shardy | trown: how do you want to handle the release of tripleo-quickstart? | 16:06 |
| shardy | https://github.com/openstack/releases/blob/master/deliverables/newton/tripleo-quickstart.yaml | 16:07 |
| *** yamahata has joined #tripleo | 16:07 | |
| shardy | it was marked as a deliverable for newton, but it's not got a release model identified, and we've not branched any stable branches yet | 16:07 |
| shardy | should it be cycle related, or in the _independent deliverables? | 16:07 |
| trown | shardy: hmm... I think it should stay branchless | 16:07 |
| trown | that is independent? | 16:08 |
| shardy | trown: Ok, do you want me to propose a release tho, bumping the major version, e.g to 2.0.0 ? | 16:08 |
| shardy | or just not propose it and let you handle it? | 16:08 |
| shardy | I'm about to propose ocata-1 for all other tripleo pieces | 16:08 |
| trown | hmm, lets hold off on releasing it... there is not really anything to release | 16:08 |
| trown | it would be good to do a release once we have the tripleo-quickstart-extras pieces in place though | 16:09 |
| *** dmarlin has joined #tripleo | 16:09 | |
| shardy | trown: ack, OK i'll mention we're planning to release it later and switch the release model to independent | 16:09 |
| trown | shardy++ thanks | 16:09 |
| dsneddon | bnemec, slagle, shardy: Can I get a review on https://review.openstack.org/#/c/396446 ? This is the situation where dhclient continues to run even after os-net-config has configured interfaces, so the same IP/routes end up on more than one interface. It's a critical bug. | 16:10 |
| openstackgerrit | Merged openstack/tripleo-docs: Remove usage of --compute-scale from the scale roles docs https://review.openstack.org/381187 | 16:11 |
| openstackgerrit | Merged openstack/tripleo-docs: Add openstack-tripleo-ui to list of included packages https://review.openstack.org/383684 | 16:12 |
| *** tremble has quit IRC | 16:15 | |
| *** achadha has joined #tripleo | 16:16 | |
| shardy | jtomasek, akrivoka: what do you want to do re the tripleo-ui release naming - shall I bump to 2.0.0 for the first ocata release? | 16:16 |
| *** ealcaniz has quit IRC | 16:16 | |
| shardy | We went with cycle-with-intermediary so I don't think we need to align with the milestone 0b1 release numbers | 16:16 |
| *** egafford has joined #tripleo | 16:17 | |
| *** rhallisey__ is now known as rhallisey | 16:17 | |
| akrivoka | jrist: ^ | 16:18 |
| shardy | shadower: likewise with tripleo-validations, although we started at 5.0.0 | 16:18 |
| shardy | https://github.com/openstack/releases/blob/master/deliverables/newton/tripleo-validations.yaml | 16:18 |
| shardy | we can just release 6.0.0 now, or adopt the milestone model this release instead | 16:19 |
| *** ramishra has quit IRC | 16:19 | |
| openstackgerrit | Merged openstack/tripleo-common: Remove discover from test-requirements https://review.openstack.org/345225 | 16:20 |
| *** ramishra has joined #tripleo | 16:20 | |
| sshnaidm | bnemec, hi | 16:21 |
| sshnaidm | bnemec, I don't see tripleo-quickstart project cloned in /opt/stack/new in image, do you know where is it configured? | 16:22 |
| sshnaidm | derekh, ^^ | 16:22 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Use keystone profile parameter to pass heat password https://review.openstack.org/398127 | 16:22 |
| *** mcornea has quit IRC | 16:22 | |
| openstackgerrit | Merged openstack/puppet-tripleo: Remove explicit hiera calls for heat in keystone profile https://review.openstack.org/398128 | 16:22 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Do not manage overcloud repositories when using external Ceph https://review.openstack.org/397819 | 16:23 |
| *** achadha has quit IRC | 16:23 | |
| *** achadha has joined #tripleo | 16:23 | |
| openstackgerrit | Adriano Petrich proposed openstack/python-tripleoclient: Use stack name or id for backwards compatibility https://review.openstack.org/398289 | 16:24 |
| derekh | sshnaidm: I think adding it to PROJECTS here does it http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/tripleo.yaml#n159 | 16:24 |
| *** numans has quit IRC | 16:25 | |
| openstackgerrit | Merged openstack/instack-undercloud: Prevent pxe_ilo driver from guessing the boot mode to use https://review.openstack.org/397112 | 16:25 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Fix typo in Keystone Sensu subscription https://review.openstack.org/398285 | 16:25 |
| sshnaidm | derekh, and then it's processed in devstack-gate, right? | 16:27 |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart: Add ability for images job to run with pending deps repo https://review.openstack.org/324008 | 16:30 |
| derekh | sshnaidm: yup, its been a long time since I looked at it but thats how I remember it working | 16:34 |
| openstackgerrit | Brad P. Crochet proposed openstack/python-tripleoclient: Make setting nodes to manageable a discrete workflow https://review.openstack.org/398423 | 16:34 |
| sshnaidm | derekh, thanks | 16:34 |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates: Do not manage overcloud repositories when using external Ceph https://review.openstack.org/398475 | 16:35 |
| *** chandankumar has quit IRC | 16:36 | |
| *** rbowen has joined #tripleo | 16:37 | |
| *** hjensas has quit IRC | 16:37 | |
| *** dmacpher is now known as dmacpher-afk | 16:37 | |
| *** abregman has quit IRC | 16:40 | |
| *** thrash is now known as thrash|eatz | 16:43 | |
| dsneddon | bnemec, I appreciate your feedback on the os-net-config dhclient bug, but you do realize that we will probably never hit the situation where dhclient is not found, right? You would have to assume that A) dhclient exists so that the dhclient pid file exists, and B) dhclient is not located in /sbin or /usr/sbin. | 16:45 |
| dsneddon | bnemec, Honestly, I don't think we will ever hit that scenario. | 16:46 |
| bnemec | dsneddon: I +2'd. We're already in complete agreement on this. :-) | 16:47 |
| dsneddon | bnemec, Oh, cool, thanks! | 16:47 |
| *** jistr is now known as jistr|biab | 16:48 | |
| *** rasca has quit IRC | 16:49 | |
| openstackgerrit | Vincent S. Cojot proposed openstack/tripleo-heat-templates: Adds http proxy support for registering RHEL overcloud nodes https://review.openstack.org/367554 | 16:50 |
| *** paramite has quit IRC | 16:51 | |
| bnemec | derekh: slagle: I wonder if giving our ovb nodes two cpus would help with stuff like https://bugs.launchpad.net/tripleo/+bug/1640879 | 16:51 |
| openstack | Launchpad bug 1640879 in tripleo "CI: apache fails to start on overcloud controller" [High,Triaged] | 16:51 |
| bnemec | And maybe improve our test times significantly. | 16:51 |
| openstackgerrit | Merged openstack/puppet-tripleo: Increase haproxy timeouts https://review.openstack.org/397822 | 16:51 |
| bnemec | When I was looking through the logs yesterday I noticed that just step 5 of the overcloud deploy is taking nearly 20 minutes in ci. | 16:52 |
| bnemec | When I run it locally it's less than 10. | 16:52 |
| slagle | bnemec: probably. i actually think we should make our flavors used by ovb match the infra defined flavors across the board | 16:52 |
| bnemec | But my local env has 2 cpus per node. | 16:52 |
| bnemec | slagle: So 8 gig/4 cpus? | 16:52 |
| *** absubram has quit IRC | 16:53 | |
| bnemec | We could probably do that. I don't think we're particularly close to running out of resources in rh1, even with it slammed like it is. | 16:53 |
| slagle | 8GB/8vcpus | 16:53 |
| slagle | we'd have to tune down workers appropriately | 16:53 |
| shardy | The multinode jobs certainly seem a lot more stable, although there's probably a few reasons for that | 16:53 |
| bnemec | slagle: We already do that though, so it should be fine. | 16:53 |
| slagle | yes | 16:53 |
| bnemec | FWIW, this is the hypervisor-stats from rh1 right now: http://paste.openstack.org/show/589480/ | 16:54 |
| bnemec | We're using just over half the memory | 16:54 |
| bnemec | cpus are closer to max, but they're not going to be used 100% of the time anyway so we should be able to afford some overcommit. | 16:55 |
| derekh | bnemec: slagle worth a try | 16:55 |
| bnemec | slagle: derekh shardy: Okay, everyone good with us going straight to 8 and 8? | 16:57 |
| *** cylopez has quit IRC | 16:57 | |
| slagle | yea let's do it | 16:57 |
| bnemec | I'm always monitoring cpu, ram, and disk on the compute nodes, so if they end up pegged I should know. | 16:57 |
| derekh | bnemec: yup, we are not using all the RAM but I always considered this a good thing, IO in our VM's isn't synced to disk its all cached in RAM, if we use more RAM up we may see IO slowing down (or we should at least keep an eye on it to make sure the effects arn't too bad) | 16:57 |
| slagle | you can bump the undercloud flavor up to 8/8 too | 16:58 |
| *** dtantsur is now known as dtantsur|afk | 16:58 | |
| *** rbowen has quit IRC | 16:58 | |
| bnemec | Okay, overcloud nodes are 8/8. | 17:00 |
| bnemec | slagle: Do we limit the workers on the undercloud? I know we talked about doing it, but I can't remember for sure. | 17:00 |
| bnemec | Note that the undercloud is already at 8/4. | 17:00 |
| slagle | i dont know that we do | 17:00 |
| *** absubram has joined #tripleo | 17:01 | |
| slagle | if we don't, i guess leave it at 8/4 for now | 17:01 |
| bnemec | I really don't think the undercloud is our limiting factor right now anyway. | 17:02 |
| *** ctayal has quit IRC | 17:02 | |
| *** dbecker has quit IRC | 17:04 | |
| *** ctayal has joined #tripleo | 17:04 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Update ovb baremetal flavor to match infra flavor https://review.openstack.org/398496 | 17:07 |
| openstackgerrit | Bob Fournier proposed openstack/os-net-config: Stop dhclient in os-net-config if interface not set for DHCP https://review.openstack.org/398498 | 17:08 |
| bnemec | derekh: Any concerns with me bumping rh2? I'm guessing we aren't close to exhausting our resources there either, but I'm not sure how many people are using it these days. | 17:08 |
| panda | bnemec: derekh slagle is adding more CPU impacting memory ? derekh what's the memory overcommitment ? | 17:08 |
| bnemec | panda: It shouldn't significantly impact memory. We limit the api workers in ci so the additional cpus won't result in additional workers. | 17:09 |
| bnemec | And we're not overcommitted on memory. We have nearly 2 TB free in rh1. | 17:09 |
| panda | bnemec: derekh slagle we see this sometimes https://bugs.launchpad.net/tripleo/+bug/1639970 | 17:09 |
| openstack | Launchpad bug 1639970 in tripleo "CI: cinder fails to allocate memory while creating volume for ping test tenant" [High,Confirmed] | 17:09 |
| bnemec | Well, 1.5, but that rounds to 2. :-) | 17:09 |
| bnemec | panda: We also bumped the memory by 2 GB, so it should help with that too. | 17:10 |
| derekh | bnemec: no concerns, we may need to bump users quota | 17:10 |
| derekh | bnemec: and nodepools for that matter | 17:10 |
| derekh | bnemec: no sure what its set to at the moment | 17:10 |
| bnemec | derekh: Yeah, I think we basically gave nodepool infinite cpus but limited memory. | 17:11 |
| bnemec | Probably do need to bump the memory now... | 17:11 |
| * bnemec looks | 17:11 | |
| *** panda is now known as panda|bbl | 17:13 | |
| derekh | bnemec: slagle Its also worth noting that doing this effectively bumps the minimum tripleo dev requirements again to 40G of RAM for a HA deployment | 17:15 |
| derekh | its not so long ago it was 26G | 17:16 |
| bnemec | Yeah :-( | 17:16 |
| * bnemec starts shopping for a second ovb node | 17:16 | |
| bnemec | And I'm not sure what we'll do once we run out of memory the next time. Which history suggests will happen sooner rather than later. | 17:17 |
| derekh | Yup, probably next week | 17:18 |
| slagle | well, the requirements are already that | 17:18 |
| slagle | we're just updating CI to reflect that :) | 17:18 |
| jrist | shardy: I think 2.0 should be fine - akrivoka do you agree? florianf ? | 17:18 |
| florianf | jrist, shardy: 2.0 sounds good to me, since other projects are also bumping major versions with new openstack releases. | 17:23 |
| florianf | akrivoka: ^ | 17:23 |
| *** florianf is now known as florianf|afk | 17:25 | |
| *** aufi has quit IRC | 17:27 | |
| jpich | honza, akrivoka: I think I remember you both worked on node assignment/tagging, you may be interested in checking out https://bugs.launchpad.net/tripleo/+bug/1642342 | 17:29 |
| openstack | Launchpad bug 1642342 in tripleo "Node assignment always updates the 'overcloud' node count, no matter the current plan" [High,Triaged] | 17:29 |
| honza | jpich: looking | 17:31 |
| jpich | honza: Cheers! | 17:31 |
| *** rbowen has joined #tripleo | 17:34 | |
| honza | jpich: and there is no error message? | 17:34 |
| openstackgerrit | Adriano Petrich proposed openstack/python-tripleoclient: Use stack name or id for backwards compatibility https://review.openstack.org/398289 | 17:35 |
| jpich | honza: Nothing on the UI, I haven't checked the console. I guess things do get updated as they should, just not in the right place? | 17:35 |
| jpich | honza: I'll try again now and look at the browser console | 17:35 |
| honza | jpich: i'm thinking more of mistral errors | 17:35 |
| *** jistr|biab is now known as jistr | 17:36 | |
| gfidente | trown is oooq_pool created by oooq/ | 17:36 |
| gfidente | ? | 17:36 |
| jpich | honza: I'll check these logs too. It's easy to reproduce though, this is happening on my recent-ish Ocata build so you can probably see it in your current dev env | 17:36 |
| honza | jpich: I remember seeing a bug where the count wasn't updated properly because the mistral action had incorrect names for the role | 17:37 |
| *** absubram has quit IRC | 17:37 | |
| openstackgerrit | yolanda.robla proposed openstack/python-tripleoclient: Support whole disk images in TripleO https://review.openstack.org/394426 | 17:37 |
| *** dsariel has quit IRC | 17:37 | |
| honza | jpich: still setting up an env right now, i can test once it's up and running | 17:37 |
| jpich | honza: Well it *is* updated properly, just in the wrong Mistral env. E.g. my CephStorageCount for 'overcloud' just got up to '2' now, even though I added a node while I had the 'test' plan activated | 17:38 |
| honza | jpich: interesting | 17:39 |
| jpich | honza: Mistral looks happy and explicitly fetches and updates the environment named 'overcloud' | 17:39 |
| openstackgerrit | John Fulton proposed openstack/tripleo-specs: Integrate TripleO with Tendrl for External Storage Deployment/Management https://review.openstack.org/387631 | 17:40 |
| *** fragatina has joined #tripleo | 17:41 | |
| trown | gfidente: ya, the volume pool for libvirt is created by oooq | 17:41 |
| trown | gfidente: that is what you are asking about ya? | 17:41 |
| gfidente | yep | 17:41 |
| *** fzdarsky is now known as fzdarsky|afk | 17:42 | |
| *** fzdarsky|afk has quit IRC | 17:43 | |
| *** iranzo has quit IRC | 17:44 | |
| openstackgerrit | Merged openstack/tripleo-puppet-elements: Update git commit sources for Midonet element https://review.openstack.org/394975 | 17:45 |
| openstackgerrit | Merged openstack/tripleo-puppet-elements: Installs more packages with the Midonet element https://review.openstack.org/394976 | 17:45 |
| *** hewbrocca is now known as hewbrocca_afk | 17:46 | |
| *** lucasagomes is now known as lucas-afk | 17:47 | |
| *** derekh has quit IRC | 17:47 | |
| *** david-lyle has quit IRC | 17:48 | |
| *** b00tcat has quit IRC | 17:48 | |
| beagles | gfidente, dsneddon: https://review.openstack.org/#/c/398242/ appears to have passed CI, so if you get a moment... | 17:48 |
| *** david-lyle has joined #tripleo | 17:48 | |
| dsneddon | beagles, Looks good, +2 | 17:49 |
| beagles | dsneddon: thx! | 17:49 |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 17:49 |
| *** lmiccini has quit IRC | 17:49 | |
| *** achadha has quit IRC | 17:50 | |
| *** lmiccini has joined #tripleo | 17:50 | |
| *** trown is now known as trown|lunch | 17:52 | |
| dsneddon | beagles, It'd be great if you could add 'hotplug: true' to the etc/os-net-config/samples/interface.* files to demonstrate usage here: https://review.openstack.org/#/c/394660 | 17:53 |
| beagles | dsneddon: agreed, will do! | 17:53 |
| *** thrash|eatz is now known as thrash | 17:53 | |
| *** milan has quit IRC | 17:58 | |
| *** ohamada has quit IRC | 18:00 | |
| *** yamahata has quit IRC | 18:01 | |
| *** fragatin_ has joined #tripleo | 18:04 | |
| *** fragatina has quit IRC | 18:08 | |
| ayoung | CAn Fernet get some love please? https://review.openstack.org/#/c/397381/ | 18:09 |
| *** florianf|afk has quit IRC | 18:09 | |
| *** absubram has joined #tripleo | 18:14 | |
| *** florianf|afk has joined #tripleo | 18:17 | |
| *** achadha has joined #tripleo | 18:19 | |
| *** tosky has quit IRC | 18:20 | |
| *** achadha has quit IRC | 18:20 | |
| *** dbecker has joined #tripleo | 18:20 | |
| *** rbowen has left #tripleo | 18:20 | |
| *** achadha has joined #tripleo | 18:21 | |
| bnemec | Hmm, we're getting quota failures, but I bumped the quota. :-( | 18:23 |
| *** bana_k has quit IRC | 18:27 | |
| *** bana_k has joined #tripleo | 18:27 | |
| *** pkovar has quit IRC | 18:28 | |
| *** dbecker has quit IRC | 18:32 | |
| jtomasek | jpich, honza: commented on bug https://bugs.launchpad.net/tripleo/+bug/1642342 | 18:32 |
| openstack | Launchpad bug 1642342 in tripleo "Node assignment always updates the 'overcloud' node count, no matter the current plan" [High,Triaged] | 18:32 |
| bnemec | Ah, you have to bounce the api service before quota changes take effect | 18:32 |
| bnemec | Hold on to your hats, we may experience a little ci turbulence while nova-api restarts... | 18:33 |
| openstackgerrit | Merged openstack/puppet-tripleo: Prepare 6.0.0 (ocata-1) https://review.openstack.org/398046 | 18:33 |
| jpich | jtomasek: Awesome, thank you! That's super helpful | 18:34 |
| jpich | honza: See ^, though I can also look in the morning if you don't have time or an environment | 18:34 |
| jpich | oh honza was in the ping to start with... I think I'll take my inability to read as my cue to go. Have a nice day/evening | 18:36 |
| *** jpich has quit IRC | 18:36 | |
| *** abregman has joined #tripleo | 18:37 | |
| *** abregman is now known as abregman|afk | 18:37 | |
| *** yamahata has joined #tripleo | 18:39 | |
| openstackgerrit | Brad P. Crochet proposed openstack/python-tripleoclient: Make setting nodes to manageable a discrete workflow https://review.openstack.org/398423 | 18:40 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Remove setting of OS_IMAGE_API_VERSION for newton and above https://review.openstack.org/387421 | 18:41 |
| *** jkilpatr_ has joined #tripleo | 18:42 | |
| *** jpena is now known as jpena|off | 18:42 | |
| *** jkilpatr has quit IRC | 18:43 | |
| *** ctayal has quit IRC | 18:45 | |
| openstackgerrit | Brent Eagles proposed openstack/os-net-config: Add support for enabling hotplug on interfaces https://review.openstack.org/394660 | 18:45 |
| *** athomas has quit IRC | 18:50 | |
| *** chlong has joined #tripleo | 18:53 | |
| *** akrivoka has quit IRC | 18:58 | |
| *** jcoufal has quit IRC | 19:01 | |
| *** shardy is now known as shardy_afk | 19:01 | |
| *** penick has joined #tripleo | 19:03 | |
| *** lmiccini has quit IRC | 19:03 | |
| openstackgerrit | Merged openstack/os-refresh-config: Remove discover from test-requirements https://review.openstack.org/345849 | 19:04 |
| *** trown|lunch is now known as trown | 19:05 | |
| *** coolsvap has quit IRC | 19:07 | |
| *** pblaho has quit IRC | 19:09 | |
| *** amoralej is now known as amoralej|off | 19:12 | |
| *** florianf|afk has quit IRC | 19:12 | |
| *** florianf|afk has joined #tripleo | 19:13 | |
| *** shardy_afk is now known as shardy | 19:16 | |
| *** jcoufal has joined #tripleo | 19:16 | |
| *** shardy is now known as shardy_afk | 19:17 | |
| sshnaidm | bnemec, slagle do you know what is this publisher supposed to do? https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/tripleo.yaml#L205-L211 | 19:25 |
| sshnaidm | or anybody that knows.. ^^ | 19:26 |
| *** absubram has quit IRC | 19:30 | |
| openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart: Add configuration of HA IPv6 deployments https://review.openstack.org/380358 | 19:31 |
| panda|bbl | sshnaidm: https://github.com/openstack-infra/project-config/commit/911aa1163e695491c9a8e988b80eae0c72412b7f | 19:33 |
| sshnaidm | pabelanger, ^^ | 19:36 |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart: Add ability for images job to run with pending deps repo https://review.openstack.org/324008 | 19:40 |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart: Update image building CI to do full deploy https://review.openstack.org/398556 | 19:40 |
| *** fragatin_ has quit IRC | 19:41 | |
| *** fragatina has joined #tripleo | 19:41 | |
| *** absubram has joined #tripleo | 19:42 | |
| *** panda|bbl is now known as panda|Zz | 19:44 | |
| *** jkilpatr_ has quit IRC | 19:44 | |
| openstackgerrit | Andy Smith proposed openstack/tripleo-specs: Enable Oslo.Messaging Dual Backends https://review.openstack.org/396740 | 19:45 |
| openstackgerrit | Merged openstack/os-apply-config: Remove discover from test-requirements https://review.openstack.org/345855 | 19:46 |
| *** pcaruana has quit IRC | 19:51 | |
| *** newmember has quit IRC | 19:55 | |
| *** jkilpatr_ has joined #tripleo | 19:57 | |
| *** achadha_ has joined #tripleo | 19:59 | |
| *** achadha_ has quit IRC | 19:59 | |
| *** achadha has quit IRC | 19:59 | |
| *** achadha has joined #tripleo | 20:00 | |
| *** jcoufal has quit IRC | 20:05 | |
| *** penick has quit IRC | 20:09 | |
| *** anshul has joined #tripleo | 20:09 | |
| *** jcoufal has joined #tripleo | 20:12 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Update ovb baremetal flavor to match infra flavor https://review.openstack.org/398496 | 20:17 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Format the nodes list in openstack overcloud delete node https://review.openstack.org/395083 | 20:21 |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Only start the deploy if the Heat stack isn't already in progress https://review.openstack.org/396567 | 20:23 |
| *** penick has joined #tripleo | 20:24 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Use kwargs to pass in data and error to Mistral Result https://review.openstack.org/375348 | 20:27 |
| bnemec | Dammit. | 20:28 |
| bnemec | Quota exceeded for ram: Requested 8192, but already used 2575360 of 2580480 ram | 20:28 |
| bnemec | But | 20:28 |
| bnemec | | RAM | 2517504 | 3000000 | | 20:28 |
| openstackgerrit | ayoung proposed openstack/tripleo-common: Fernet Key management https://review.openstack.org/397381 | 20:30 |
| ayoung | shardy_afk, you should like that better | 20:30 |
| panda|Zz | bnemec: I think I remember something like this appened for the quota, and some services needed to be rebooted. I can't be more vague that this sorry. | 20:31 |
| *** jkilpatr_ has quit IRC | 20:37 | |
| *** jkilpatr has joined #tripleo | 20:38 | |
| *** jkilpatr has quit IRC | 20:43 | |
| *** newmember has joined #tripleo | 20:53 | |
| *** jkilpatr has joined #tripleo | 20:56 | |
| *** rcernin has joined #tripleo | 20:58 | |
| *** ckyriaki has joined #tripleo | 21:01 | |
| *** ckyriaki has quit IRC | 21:07 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 21:08 |
| *** Goneri has quit IRC | 21:10 | |
| *** dprince has quit IRC | 21:13 | |
| *** morazi has quit IRC | 21:14 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/puppet-tripleo: Remove Combination alarms support https://review.openstack.org/398579 | 21:15 |
| jeckersb | am i losing my mind and missing something obvious here - https://review.openstack.org/#/c/373488 | 21:15 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Remove Combination alarms support https://review.openstack.org/398580 | 21:15 |
| jeckersb | why is that stuck in "Needs Verified" | 21:15 |
| trown | jeckersb: the second set of jobs (ovb) has not run after your "reverify" comment | 21:16 |
| *** hafe has joined #tripleo | 21:16 | |
| jeckersb | trown: hm ok | 21:18 |
| jeckersb | i think they probably should have run before anyway | 21:18 |
| jeckersb | the first set of jobs had succeeded previously | 21:18 |
| openstackgerrit | Ian Main proposed openstack/tripleo-common: Add python-memcached to agent container. https://review.openstack.org/398582 | 21:18 |
| * jeckersb shrugs | 21:18 | |
| jeckersb | i guess i will be patient for a bit? | 21:18 |
| trown | ya, the ovb jobs should complete soonish | 21:18 |
| jeckersb | oh i see it now in zuul | 21:19 |
| jeckersb | (failed) | 21:19 |
| jeckersb | >.< | 21:19 |
| *** hafe has left #tripleo | 21:21 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Add configurable mtu to --mulitnode-setup https://review.openstack.org/385162 | 21:22 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Add 3 node multinode job https://review.openstack.org/397441 | 21:22 |
| *** kjw3 has joined #tripleo | 21:23 | |
| openstackgerrit | James Slagle proposed openstack/tripleo-common: Remove network-gateway element from image build https://review.openstack.org/398583 | 21:25 |
| *** jayg is now known as jayg|g0n3 | 21:25 | |
| *** jkilpatr has quit IRC | 21:25 | |
| openstackgerrit | James Slagle proposed openstack/python-tripleoclient: Remove network-gateway element from image build https://review.openstack.org/398584 | 21:26 |
| *** gfidente has quit IRC | 21:31 | |
| *** fultonj_ has joined #tripleo | 21:33 | |
| *** fultonj_ has quit IRC | 21:33 | |
| *** abregman|afk has quit IRC | 21:35 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Deploy network envs appropriate for the job https://review.openstack.org/347503 | 21:37 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Test with bonded public interface https://review.openstack.org/385562 | 21:37 |
| bnemec | | RAM | 2608128 | 3000000 | | 21:46 |
| bnemec | Finally! | 21:46 |
| *** arxcruz has quit IRC | 21:46 | |
| bnemec | Turns out there are two quotas you have to change. | 21:47 |
| *** trown is now known as trown|outtypewww | 21:52 | |
| openstackgerrit | Alex Schultz proposed openstack/instack-undercloud: Make inspection also respect whether iPXE is enabled https://review.openstack.org/397114 | 21:52 |
| *** ayoung has quit IRC | 21:55 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-apply-config: Updated from global requirements https://review.openstack.org/378869 | 22:00 |
| *** lblanchard has quit IRC | 22:04 | |
| *** lifeless has quit IRC | 22:11 | |
| *** Goneri has joined #tripleo | 22:11 | |
| *** lifeless has joined #tripleo | 22:12 | |
| *** jprovazn has quit IRC | 22:17 | |
| *** rhallisey has quit IRC | 22:17 | |
| ansiwen | how can I determine the right number for firewall rules? do I have to look through all the files and check which one is free? | 22:19 |
| ansiwen | or can I reuse one? | 22:19 |
| ansiwen | what is this number anyways? | 22:19 |
| bnemec | ansiwen: Yeah, I think people have just been grepping for firewall rules and picking the next unused one. | 22:20 |
| bnemec | I'm not sure it particularly matters. | 22:20 |
| ansiwen | I will reuse the nova one for ec2-api, I saw they have been reused in other cases as well... | 22:21 |
| *** tiswanso has quit IRC | 22:21 | |
| *** jeckersb is now known as jeckersb_gone | 22:23 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Test with scheduler hints https://review.openstack.org/378040 | 22:23 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Add support for testing predictable placement https://review.openstack.org/378014 | 22:23 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Test hostname map https://review.openstack.org/378017 | 22:23 |
| *** anshul has quit IRC | 22:25 | |
| *** absubram has quit IRC | 22:26 | |
| *** fultonj has quit IRC | 22:27 | |
| *** penick has quit IRC | 22:29 | |
| openstackgerrit | Ben Nemec proposed openstack/diskimage-builder: Disable all repos in os-refresh-config too https://review.openstack.org/398630 | 22:29 |
| *** jcoufal has quit IRC | 22:34 | |
| *** absubram has joined #tripleo | 22:37 | |
| openstackgerrit | Sven Anderson proposed openstack/tripleo-heat-templates: Add ec2-api service https://review.openstack.org/398634 | 22:40 |
| *** thrash is now known as thrash|g0ne | 22:44 | |
| *** Goneri has quit IRC | 22:44 | |
| *** ayoung has joined #tripleo | 22:44 | |
| *** pradk has quit IRC | 22:48 | |
| *** tiswanso has joined #tripleo | 22:51 | |
| openstackgerrit | Merged openstack/os-net-config: Stop dhclient in os-net-config if interface not set for DHCP https://review.openstack.org/396446 | 22:52 |
| openstackgerrit | d.marlin proposed openstack/diskimage-builder: Fedora AArch64 (64-bit ARM) support in diskimage-builder https://review.openstack.org/398639 | 22:55 |
| *** tiswanso has quit IRC | 22:55 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 22:58 |
| *** chlong has quit IRC | 22:59 | |
| *** jkilpatr has joined #tripleo | 23:00 | |
| sshnaidm | bnemec, is there a problem with stable branches now? | 23:01 |
| bnemec | sshnaidm: Not that I know of. Why? | 23:02 |
| sshnaidm | bnemec, a few jobs failed: http://logs.openstack.org/94/381094/37/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha-newton/511c04d/console.html#_2016-11-16_20_16_16_004175 http://logs.openstack.org/94/381094/37/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha-mitaka/e47d90b/console.html#_2016-11-16_19_24_36_451925 | 23:02 |
| sshnaidm | also here: http://logs.openstack.org/26/365026/12/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha-newton/df8aa36/console.html#_2016-11-16_19_27_39_200014 | 23:04 |
| *** penick has joined #tripleo | 23:07 | |
| *** florianf|afk has quit IRC | 23:10 | |
| sshnaidm | heat feels really bad: No module named senlinclient, AMQP server on 192.0.2.10:5672 is unreachable: [Errno 111] ECONNREFUSED. | 23:11 |
| bnemec | sshnaidm: Hmm, concerning. Can you open a bug? | 23:12 |
| sshnaidm | bnemec, yeah | 23:12 |
| *** florianf|afk has joined #tripleo | 23:13 | |
| sshnaidm | bnemec, Out of memory: Kill process 12866 (beam.smp) score 29 or sacrifice chil | 23:15 |
| bnemec | sshnaidm: Really...I wonder if our worker count isn't being applied properly on the stable branches or something. | 23:16 |
| bnemec | We just bumped the memory so OOMs should not be happening anymore. | 23:16 |
| sshnaidm | bnemec, yeah, that's strange | 23:16 |
| sshnaidm | bnemec, do we still use low-memory templates? | 23:16 |
| *** chlong has joined #tripleo | 23:16 | |
| bnemec | But we did bump the cpu at the same time, so if a worker is still scaling by cpu it could be a problem. | 23:16 |
| sshnaidm | bnemec, you mean the workers amount depends on cpus amount? | 23:17 |
| bnemec | sshnaidm: Yes | 23:18 |
| sshnaidm | bnemec, I see we also enabled sahara, it eats memory like crocodile | 23:18 |
| bnemec | Hmm, the newton job is missing the low-memory env, unless I'm just missing it. | 23:18 |
| bnemec | http://logs.openstack.org/94/381094/37/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha-newton/511c04d/console.html#_2016-11-16_19_08_44_511420 | 23:18 |
| bnemec | Mitaka has it, but it's the mitaka-and-below one: http://logs.openstack.org/94/381094/37/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha-mitaka/e47d90b/console.html#_2016-11-16_18_48_05_529385 | 23:18 |
| sshnaidm | bnemec, that's some problem with this "echo".. look at a few lines before: http://logs.openstack.org/94/381094/37/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha-newton/511c04d/console.html#_2016-11-16_19_08_44_269161 | 23:19 |
| *** dsneddon_ has joined #tripleo | 23:19 | |
| bnemec | Oh wait, no. | 23:19 |
| bnemec | http://logs.openstack.org/94/381094/37/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha-mitaka/e47d90b/console.html#_2016-11-16_18_48_05_655894 | 23:19 |
| *** penick has quit IRC | 23:20 | |
| bnemec | I bet this is the deploy.env thing. Our changes in deploy.sh are getting overwritten in tripleo.sh. | 23:20 |
| * bnemec kind of wants to revert that change | 23:20 | |
| *** penick has joined #tripleo | 23:20 | |
| bnemec | This may also explain the slowness in CI lately. | 23:20 |
| *** kjw3 has quit IRC | 23:20 | |
| sshnaidm | bnemec, what did deploy.env? | 23:21 |
| bnemec | sshnaidm: There was a change recently that sources deploy.env in tripleo.sh every time it gets run: http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/tripleo.sh#n100 | 23:21 |
| bnemec | So any env changes we make in deploy.sh get overwritten. | 23:21 |
| sshnaidm | bnemec, and it runs every time tripleo.sh is executed..? | 23:22 |
| bnemec | sshnaidm: Yes | 23:23 |
| *** penick has quit IRC | 23:23 | |
| sshnaidm | bnemec, and where is this file? I don't find in the repo | 23:24 |
| bnemec | sshnaidm: It's generated by the ci run. | 23:25 |
| sshnaidm | bnemec, oh, I see | 23:25 |
| sshnaidm | bnemec, yeah, also used it for env vars | 23:25 |
| sshnaidm | bnemec, but it's not right to do so.. we change some arguments along the job | 23:26 |
| sshnaidm | and each tripleo.sh call just cancel all changes | 23:26 |
| bnemec | Yeah, it's causing some problems. | 23:26 |
| bnemec | This isn't the first one I've run into. | 23:27 |
| *** chlong has quit IRC | 23:27 | |
| sshnaidm | if I see 16 processes of nova-api, heat etc, then we don't use low-memory I think | 23:27 |
| *** bana_k has quit IRC | 23:31 | |
| bnemec | sshnaidm: Yeah, I think we should add an explicit switch to control sourcing deploy.env instead of always doing it if it exists. | 23:32 |
| sshnaidm | bnemec, we usually do it in deploy.sh, isn't it? | 23:33 |
| bnemec | sshnaidm: Right, but according to the commit message on https://github.com/openstack-infra/tripleo-ci/commit/67deb95ed15870113ebb856629cbb0a1bd4eee55 we need to do it other times in the multinode jobs. | 23:33 |
| sshnaidm | ok, then if [[ $MULTINODE == "1" ]]; then source anything.. | 23:37 |
| bnemec | sshnaidm: No, it's not quite that simple. There's no env when we first ssh to another node, so MULTINODE will never be set. | 23:38 |
| bnemec | I think we just need a --source-env option to tripleo.sh or something along those lines. | 23:38 |
| sshnaidm | bnemec, uh.. it's on different nodes | 23:38 |
| sshnaidm | bnemec, maybe for multinode jobs on other nodes to use different path of deploy.env there? | 23:39 |
| bnemec | sshnaidm: That could work too. | 23:39 |
| sshnaidm | bnemec, https://github.com/openstack-infra/tripleo-ci/blob/master/toci_instack_osinfra.sh#L95 | 23:41 |
| bnemec | sshnaidm: Yeah, we could probably rename that to be subnode.env or something like that. | 23:43 |
| openstackgerrit | Brent Eagles proposed openstack/os-net-config: Stop dhclient in os-net-config if interface not set for DHCP https://review.openstack.org/398498 | 23:44 |
| bnemec | sshnaidm: Go ahead and set the alert tag on the bug for this. We're not going to get a fix merged tonight, so we want to make sure people see it tomorrow morning. | 23:46 |
| sshnaidm | bnemec, and do we run tripleo.sh on subnodes for something else than repo-setup? | 23:46 |
| sshnaidm | bnemec, ok | 23:47 |
| bnemec | sshnaidm: It looks like it, yes. | 23:47 |
| bnemec | There's also --multinode and --bootstrap-subnodes | 23:47 |
| bnemec | Oh wait, but those aren't ssh. | 23:48 |
| bnemec | In that case it might only be --repo-setup. That's the only one changed in https://github.com/openstack-infra/tripleo-ci/commit/67deb95ed15870113ebb856629cbb0a1bd4eee55 | 23:48 |
| sshnaidm | bnemec, yep, only it | 23:50 |
| *** david-lyle_ has joined #tripleo | 23:50 | |
| sshnaidm | bnemec, hmm.. so what to write in the bug? low-memory template doesn't apply and oom killers are going wild? | 23:50 |
| bnemec | sshnaidm: Heh, works for me. Just make sure to link to a deployment log that is missing the low-memory environment from the deploy command. | 23:51 |
| *** david-lyle has quit IRC | 23:51 | |
| sshnaidm | ok | 23:52 |
| *** rcernin has quit IRC | 23:52 | |
| bfournie | beagles: thank you for updating the commit id in the cherry-pick | 23:52 |
| beagles | bfournie: np | 23:53 |
| beagles | bfournie: hopefully we'll be able to land this baby tonight | 23:53 |
| bfournie | beagles: Yes :-) Was the commit id update needed it because cherry pick was done while waiting for ovb_ha to complete on original? | 23:54 |
| bnemec | Ooh, I have a video that explains this: https://www.youtube.com/watch?v=RxDt5qVO0RM | 23:55 |
| beagles | bfournie: "needed" might be a strong word here, but basically yeah - the cherry pick in the commit is handy when reconciling fixes, etc. | 23:55 |
| sshnaidm | bnemec, hmm.. do we have low-memory.yaml only for newton and above? | 23:56 |
| bfournie | bnemec: nice, will watch, thx. | 23:57 |
| bfournie | beagles: cool, thx again | 23:57 |
| sshnaidm | bnemec, I see there test-environments/worker-config-mitaka-and-below.yaml which set engine worker to 1 | 23:57 |
| bnemec | sshnaidm: Yeah, I have an unpleasant feeling that something went wrong when migrating the new releases to the t-h-t version. | 23:58 |
| beagles | bnemec: what did you use for capturing the video? | 23:58 |
| bnemec | beagles: Just recordmydesktop | 23:59 |
| beagles | bnemec: cool | 23:59 |
| bnemec | sshnaidm: I think I screwed up in http://git.openstack.org/cgit/openstack-infra/tripleo-ci/commit/test-environments/worker-config.yaml?id=00a28469a7d01b6f76d0c12df2cc3260146a530b | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!