| *** akuznetsov has joined #tripleo | 00:01 | |
| *** pradk_ has quit IRC | 00:07 | |
| *** rdopiera_pto has quit IRC | 00:09 | |
| *** akuznetsov has quit IRC | 00:10 | |
| *** rdopiera_pto has joined #tripleo | 00:10 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Switch to libguestfs for CentOS https://review.openstack.org/323560 | 00:15 |
|---|---|---|
| *** akshai has joined #tripleo | 00:28 | |
| *** akshai_ has joined #tripleo | 00:29 | |
| *** akshai has quit IRC | 00:32 | |
| *** r-mibu has quit IRC | 00:36 | |
| *** r-mibu has joined #tripleo | 00:36 | |
| *** akshai_ has quit IRC | 00:40 | |
| *** panda has quit IRC | 00:48 | |
| *** panda has joined #tripleo | 00:49 | |
| *** xinwu has quit IRC | 00:59 | |
| *** akuznetsov has joined #tripleo | 01:07 | |
| *** fzdarsky__ has joined #tripleo | 01:12 | |
| *** akuznetsov has quit IRC | 01:12 | |
| *** fzdarsky_ has quit IRC | 01:16 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: DO NOT MERGE testing 319345 https://review.openstack.org/323369 | 01:19 |
| *** rhallisey has quit IRC | 01:19 | |
| *** yamahata has joined #tripleo | 01:23 | |
| *** julim has joined #tripleo | 01:24 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: compute: use new param for live_migration_tunnelled https://review.openstack.org/322968 | 01:26 |
| *** julim has quit IRC | 01:32 | |
| *** xinwu has joined #tripleo | 01:34 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Explode Loadbalancer role into HAproxy + keepalived https://review.openstack.org/323582 | 01:47 |
| *** xinwu has quit IRC | 01:49 | |
| *** akuznetsov has joined #tripleo | 02:08 | |
| *** colonwq has quit IRC | 02:09 | |
| *** MaxPC has quit IRC | 02:09 | |
| *** julim has joined #tripleo | 02:10 | |
| *** akuznetsov has quit IRC | 02:13 | |
| *** hogepodge has quit IRC | 02:13 | |
| *** hogepodge has joined #tripleo | 02:17 | |
| *** hogepodge has quit IRC | 02:21 | |
| *** yamahata has quit IRC | 02:43 | |
| *** tzumainn has quit IRC | 02:55 | |
| *** akuznetsov has joined #tripleo | 03:08 | |
| *** ayoung has quit IRC | 03:09 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: DO NOT MERGE testing 319345 https://review.openstack.org/323369 | 03:12 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: CI test - never merge https://review.openstack.org/323639 | 03:12 |
| *** akuznetsov has quit IRC | 03:13 | |
| *** ebalduf has joined #tripleo | 03:29 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Update default to F23 https://review.openstack.org/247296 | 03:42 |
| *** apetrich has quit IRC | 03:48 | |
| *** apetrich has joined #tripleo | 03:48 | |
| *** xinwu has joined #tripleo | 03:54 | |
| *** links has joined #tripleo | 04:00 | |
| *** akuznetsov has joined #tripleo | 04:09 | |
| *** akuznetsov has quit IRC | 04:14 | |
| *** yamahata has joined #tripleo | 04:19 | |
| *** david-lyle has quit IRC | 04:21 | |
| *** masco has joined #tripleo | 04:29 | |
| *** liverpooler has quit IRC | 04:42 | |
| *** panda has quit IRC | 04:48 | |
| *** panda has joined #tripleo | 04:49 | |
| *** milan has quit IRC | 04:55 | |
| *** rcernin has quit IRC | 05:04 | |
| *** akuznetsov has joined #tripleo | 05:10 | |
| *** akuznetsov has quit IRC | 05:14 | |
| *** saneax_AFK is now known as saneax | 05:19 | |
| openstackgerrit | Merged openstack/tripleo-ui: Update FormErrorList component https://review.openstack.org/320498 | 05:28 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Register nodes new workflow https://review.openstack.org/323665 | 05:30 |
| openstackgerrit | Merged openstack/tripleo-ui: Notifications fixes https://review.openstack.org/322022 | 05:33 |
| *** veteran has joined #tripleo | 05:35 | |
| *** rcernin has joined #tripleo | 05:43 | |
| *** ohamada has joined #tripleo | 05:47 | |
| *** akuznetsov has joined #tripleo | 05:49 | |
| *** xinwu has quit IRC | 05:51 | |
| *** akuznetsov has quit IRC | 05:53 | |
| *** oshvartz has joined #tripleo | 05:56 | |
| *** xinwu has joined #tripleo | 06:00 | |
| *** coolsvap_ has joined #tripleo | 06:02 | |
| *** milan has joined #tripleo | 06:04 | |
| *** yamahata has quit IRC | 06:05 | |
| *** apetrich has quit IRC | 06:05 | |
| *** apetrich has joined #tripleo | 06:06 | |
| *** yolanda_ has quit IRC | 06:09 | |
| *** yolanda has joined #tripleo | 06:09 | |
| *** hjensas has joined #tripleo | 06:16 | |
| *** xinwu has quit IRC | 06:18 | |
| jrist | happy birthday marios !!! | 06:20 |
| marios | thanks jrist :) | 06:21 |
| marios | and goodnight | 06:21 |
| marios | ! | 06:21 |
| *** numans has joined #tripleo | 06:21 | |
| *** yolanda_ has joined #tripleo | 06:22 | |
| *** jprovazn has joined #tripleo | 06:22 | |
| gfidente | ahahaha | 06:22 |
| *** yolanda_ has quit IRC | 06:22 | |
| gfidente | so I am not the first one | 06:22 |
| *** yolanda_ has joined #tripleo | 06:22 | |
| gfidente | too bad | 06:22 |
| gfidente | marios, I want pictures of the bike | 06:23 |
| gfidente | marios, for your bday https://www.youtube.com/watch?v=d27CwLduhis | 06:24 |
| *** yolanda has quit IRC | 06:25 | |
| *** yolanda_ has quit IRC | 06:26 | |
| *** yolanda has joined #tripleo | 06:26 | |
| *** bswartz has quit IRC | 06:27 | |
| *** hogepodge has joined #tripleo | 06:31 | |
| marios | gfidente: haha still window shipping on the bike - thanks for Ella | 06:32 |
| openstackgerrit | Babu Shanmugam proposed openstack/puppet-tripleo: Add OVN support for neutron profile https://review.openstack.org/314875 | 06:33 |
| bandini | happ birthday marios | 06:33 |
| bandini | morning * | 06:34 |
| marios | thanks bandini. "A" I booked a table "N" for tonight "J" (gimme a "A!"... gimme) | 06:38 |
| marios | a "N" ... etc | 06:38 |
| *** rlandy has quit IRC | 06:39 | |
| bandini | rotfl | 06:39 |
| *** tremble has joined #tripleo | 06:40 | |
| bandini | matbu: /win 9 | 06:41 |
| bandini | ops | 06:41 |
| bandini | matbu: do you have the list of steps you used for https://bugs.launchpad.net/tripleo/+bug/1587449 ? | 06:41 |
| openstack | Launchpad bug 1587449 in tripleo "Major Upgrade L->M failed on major controller pacemaker step 1" [Undecided,New] | 06:41 |
| *** liverpooler has joined #tripleo | 06:47 | |
| *** akuznetsov has joined #tripleo | 06:49 | |
| *** akuznetsov has quit IRC | 06:54 | |
| *** aufi has joined #tripleo | 07:02 | |
| *** tesseract has joined #tripleo | 07:03 | |
| *** slagle has quit IRC | 07:06 | |
| *** paramite has joined #tripleo | 07:06 | |
| *** ifarkas has joined #tripleo | 07:11 | |
| ccamacho | o/ Marios!!! Happy version upgrade (Hope is a minor and not a major version)! https://s-media-cache-ak0.pinimg.com/736x/7e/96/9c/7e969cd06f436e2b733a36ffa959582c.jpg | 07:15 |
| *** hjensas has quit IRC | 07:16 | |
| ccamacho | Guys we have these submissions (Redis and MongoDB) passing CI https://review.openstack.org/#/c/315051/ https://review.openstack.org/#/c/312732/ would you mind to check them!??! | 07:17 |
| marios | ccamacho: thanks :) am looking at tht for redis now | 07:20 |
| ccamacho | thanks! | 07:20 |
| *** mikelk has joined #tripleo | 07:21 | |
| *** mcornea has joined #tripleo | 07:24 | |
| *** ebarrera has joined #tripleo | 07:26 | |
| *** pcaruana has joined #tripleo | 07:26 | |
| openstackgerrit | Martin André proposed openstack/tripleo-common: Allow running validation against different plans https://review.openstack.org/318194 | 07:27 |
| openstackgerrit | Martin André proposed openstack/tripleo-common: Validations actions and workbook https://review.openstack.org/313632 | 07:27 |
| openstackgerrit | Martin André proposed openstack/instack-undercloud: Create validations user during undercloud install https://review.openstack.org/322893 | 07:31 |
| openstackgerrit | Martin André proposed openstack/instack-undercloud: Install tripleo-common sudoers files https://review.openstack.org/323692 | 07:31 |
| *** jpena|pto is now known as jpena | 07:32 | |
| mandre | shadower: ^^ all patches in order for your reviewing pleasure | 07:33 |
| openstackgerrit | Martin Mágr proposed openstack/puppet-tripleo: New composable service: Sensu https://review.openstack.org/323303 | 07:34 |
| *** anshul has joined #tripleo | 07:34 | |
| *** anshul is now known as Guest9136 | 07:34 | |
| *** dtantsur|afk is now known as dtantsur | 07:35 | |
| ccamacho | Marios quick question, the redispassword is used also in controller.yaml by ceilometer http://paste.openstack.org/show/506829/ in that case if I move Ill break it, right? | 07:36 |
| marios | ccamacho: i seek yeah i didn't know if it was being used elsewhere | 07:38 |
| marios | ccamacho: in which case it is being passed still from puppet/controller.yaml and so is get_input ... | 07:39 |
| ccamacho | nice :) | 07:39 |
| marios | ccamacho: in theory | 07:39 |
| marios | ccamacho: once ceilometer gets decomposed out of that template will may want to move that param | 07:40 |
| marios | ccamacho: i mean out of puppet/controller.yaml | 07:40 |
| ccamacho | I will add there a comment in controller.yaml | 07:41 |
| marios | ccamacho: oh no hold on | 07:42 |
| marios | ccamacho: i think the get_input is still wrong | 07:42 |
| *** samuelBartel has joined #tripleo | 07:43 | |
| samuelBartel | Hello | 07:43 |
| marios | ccamacho: cos we aren't passing the RedisPassword to redis_password anymore right? removing it. so you still may need to add it, as well, to the redis-base.yaml | 07:43 |
| ccamacho | marios, and use it as a parameter (But without removing it from controller.yaml?) | 07:44 |
| samuelBartel | a question about the nic-configs sued to deploy node for openstack with tripleo | 07:44 |
| *** cwolferh has joined #tripleo | 07:44 | |
| samuelBartel | is it possible for a same role( for example compute) to have different nic configuration between the servers? | 07:45 |
| marios | ccamacho: yeah i think that is right. added comment | 07:45 |
| ccamacho | thanks! | 07:45 |
| samuelBartel | I am trying to setup a lab environment with baremetal servers which don't have the same number of NIC | 07:45 |
| *** akuznetsov has joined #tripleo | 07:50 | |
| matbu | bandini: morning, yes, i'll comment the bug | 07:51 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Composable roles within services - Redis https://review.openstack.org/312732 | 07:52 |
| mandre | is the gate just randomly failing? -> https://review.openstack.org/#/c/322893/ | 07:52 |
| bandini | matbu: thanks, I ask because I am working on the same L->M stuff and I have all kinds of other issues hence I was wondering | 07:54 |
| *** akuznetsov has quit IRC | 07:55 | |
| *** yamahata has joined #tripleo | 07:57 | |
| openstackgerrit | Martin Mágr proposed openstack/puppet-tripleo: New composable service: Sensu https://review.openstack.org/323303 | 07:58 |
| matbu | bandini: i add the steps | 07:59 |
| matbu | bandini: what king of issue do you hit ? | 07:59 |
| bandini | matbu: thanks, well first even the undercloud upgrade breaks in two different ways (galera upgrade and something else I can't remember) | 08:00 |
| matbu | bandini: k, for the UC upgrade you need to stop some services before | 08:01 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Configure BlockStorage services via resource chains https://review.openstack.org/322858 | 08:01 |
| *** zoli_gone-proxy is now known as zoliXXL | 08:02 | |
| matbu | bandini: https://github.com/redhat-openstack/ansible-role-tripleo-overcloud-upgrade/blob/master/tasks/pre-undercloud.yml#L27 | 08:03 |
| *** fragatin_ has joined #tripleo | 08:03 | |
| bandini | oh a nice ansible role! cool | 08:03 |
| zoliXXL | good morning. | 08:04 |
| bandini | matbu: will take a look after this call, thanks! | 08:04 |
| *** zoliXXL is now known as zoli|brb | 08:05 | |
| *** coolsvap_ has quit IRC | 08:05 | |
| matbu | bandini: no pb | 08:06 |
| *** fragatina has quit IRC | 08:07 | |
| *** fragatin_ has quit IRC | 08:07 | |
| *** jaosorior has joined #tripleo | 08:08 | |
| *** fragatina has joined #tripleo | 08:08 | |
| jaosorior | marios, gfidente: Mongodb as a composable service is ready https://review.openstack.org/#/c/315051/ :D | 08:08 |
| jaosorior | that's the t-h-t patch | 08:09 |
| jaosorior | puppet-tripleo is here https://review.openstack.org/#/c/304780/ | 08:09 |
| *** olap has joined #tripleo | 08:09 | |
| ccamacho | jaosorior, yesterday night the nonha gate was getting this nova api issue, I have recheck it the patch and now is green :) | 08:10 |
| jaosorior | ccamacho: Yeah, that nova api issue is unrelated. HAve you kept seeing it around? There were several patches with the same issue | 08:12 |
| ccamacho | since 2 weeks ago +- yeahp and is unrelated and random... Maybe related with the cloud workload.. not sure | 08:13 |
| ccamacho | jaosorior ^ | 08:13 |
| ccamacho | also keystone throws a similar exception | 08:13 |
| ccamacho | sometimes.. | 08:14 |
| gfidente | jaosorior, pradk, ccamacho see comments onhttps://review.openstack.org/#/c/304780/14 | 08:14 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Take 'host' parameter from neutron manifest into use https://review.openstack.org/323297 | 08:14 |
| *** yamahata has quit IRC | 08:15 | |
| *** yamahata has joined #tripleo | 08:15 | |
| ccamacho | jaosorior, Ill apply the change in the THT submission for MongoDB | 08:17 |
| jaosorior | ccamacho: Sure | 08:17 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Take 'host' parameter from nova manifest into use https://review.openstack.org/323295 | 08:17 |
| *** hjensas has joined #tripleo | 08:20 | |
| *** hjensas has joined #tripleo | 08:20 | |
| *** lmiccini has quit IRC | 08:22 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Composable roles within services - MongoDB https://review.openstack.org/315051 | 08:23 |
| *** lmiccini has joined #tripleo | 08:31 | |
| *** dtantsur is now known as dtantsur|bbl | 08:31 | |
| *** dbecker has quit IRC | 08:35 | |
| jistr | happy birthday marios! | 08:36 |
| *** electrofelix has joined #tripleo | 08:36 | |
| *** paramite is now known as paramite|afk | 08:36 | |
| *** dmk0202 has joined #tripleo | 08:38 | |
| gfidente | jistr, we can ask for things on bday | 08:38 |
| marios | jistr: ty :) | 08:38 |
| openstackgerrit | Imre Farkas proposed openstack/tripleo-specs: Metal to tenant https://review.openstack.org/320995 | 08:39 |
| jaosorior | marios: happy birthday dude :D | 08:39 |
| jaosorior | anybody have an idea why this fails in pep8? -> nova::host: %{::fqdn} | 08:39 |
| jaosorior | it's valid hiera :/ | 08:40 |
| jaosorior | "found character '%' that cannot start any token" | 08:40 |
| gfidente | try putting it in double quotes | 08:40 |
| marios | thanks jaosorior | 08:40 |
| *** dbecker has joined #tripleo | 08:40 | |
| jistr | jaosorior: should it be in quotes maybe? | 08:41 |
| ccamacho | jaosorior, im working on the puppet-tripleo patch for mongodb (for not duplicating effort) | 08:41 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Take 'host' parameter from nova manifest into use https://review.openstack.org/323295 | 08:41 |
| jaosorior | ccamacho: Sure, no worries, I'm working in other patches | 08:41 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Take 'host' parameter from neutron manifest into use https://review.openstack.org/323297 | 08:42 |
| *** saneax is now known as saneax_AFK | 08:44 | |
| openstackgerrit | Martin André proposed openstack/tripleo-validations: Validation to detect if undercloud disks use 512e https://review.openstack.org/322179 | 08:44 |
| openstackgerrit | Martin André proposed openstack/tripleo-validations: Add documentation and dynamic inventory script https://review.openstack.org/322178 | 08:44 |
| openstackgerrit | Martin André proposed openstack/tripleo-common: Allow running validation against different plans https://review.openstack.org/318194 | 08:45 |
| openstackgerrit | Martin André proposed openstack/tripleo-common: Validations actions and workbook https://review.openstack.org/313632 | 08:45 |
| *** paramite|afk is now known as paramite | 08:46 | |
| *** derekh has joined #tripleo | 08:47 | |
| *** dmk0202 has quit IRC | 08:48 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: TEST - DO NOT MERGE https://review.openstack.org/323732 | 08:52 |
| *** paramite is now known as paramite|afk | 08:53 | |
| *** akuznetsov has joined #tripleo | 08:55 | |
| *** paramite|afk is now known as paramite | 08:56 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: [NO MERGY] Test a fake periodic job https://review.openstack.org/229789 | 08:59 |
| *** hjensas_ has joined #tripleo | 08:59 | |
| *** hjensas_ has joined #tripleo | 08:59 | |
| *** zoli|brb is now known as zoliXXL | 09:01 | |
| *** tremble has quit IRC | 09:01 | |
| *** jaosorior has quit IRC | 09:02 | |
| *** jaosorior has joined #tripleo | 09:02 | |
| *** hjensas has quit IRC | 09:02 | |
| *** tremble has joined #tripleo | 09:03 | |
| *** tremble has quit IRC | 09:03 | |
| *** tremble has joined #tripleo | 09:03 | |
| openstackgerrit | Carlos Camacho proposed openstack/puppet-tripleo: Add mongodb profiles https://review.openstack.org/304780 | 09:03 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Composable roles within services - MongoDB https://review.openstack.org/315051 | 09:04 |
| *** hjensas_ has quit IRC | 09:05 | |
| *** yamahata has quit IRC | 09:08 | |
| *** tosky has joined #tripleo | 09:08 | |
| *** sambetts|afk is now known as sambetts | 09:09 | |
| *** fragatin_ has joined #tripleo | 09:11 | |
| *** fragatina has quit IRC | 09:15 | |
| *** fragatin_ has quit IRC | 09:16 | |
| *** paramite is now known as paramite|afk | 09:18 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Add Major upgrade function to the tripleo ci script https://review.openstack.org/323750 | 09:18 |
| *** paramite|afk is now known as paramite | 09:20 | |
| matbu | marios: jistr I pushed the review for the major upgrade but i have no idea on how to test it and how to add a job to the CI | 09:21 |
| *** hjensas_ has joined #tripleo | 09:22 | |
| *** jtomasek has joined #tripleo | 09:22 | |
| *** fragatina has joined #tripleo | 09:27 | |
| marios | matbu: ack thanks will have a look ... have you poked at tripleo-ci at all (I am not very familiar)...i see the jobs are defined in toci_gate_test.sh. i wonder if we can add a new job... but it is a different workflow. we should just invoke tripleo.sh --overcloud-upgrade ... perhaps look more at how the current 'upgrades' job is implemented as starter? | 09:27 |
| jaosorior | marios: Also gotta change deploy.sh | 09:27 |
| jaosorior | as that's where the actual deployment takes place | 09:28 |
| marios | jaosorior: yeah ... i noticed it has the actual call to tripleo.sh --deploy... (matbu actually adds a --upgrade at https://review.openstack.org/#/c/323750/) | 09:28 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Initial zaqar websockets https://review.openstack.org/319070 | 09:29 |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Register nodes new workflow https://review.openstack.org/323665 | 09:29 |
| matbu | marios: jaosorior yep, my concerns is where should i execute the script (or the deploy.sh) | 09:29 |
| matbu | I have some boxes ready for, but should i prep some vms there ? | 09:30 |
| marios | matbu: yeah would be good to get some testing cos bash | 09:31 |
| *** fragatina has quit IRC | 09:31 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Rework the pacemaker_common_functions for M..N upgrades https://review.openstack.org/321027 | 09:32 |
| *** ohamada has quit IRC | 09:32 | |
| *** saneax_AFK is now known as saneax | 09:35 | |
| *** dsariel has joined #tripleo | 09:36 | |
| *** mgould|afk is now known as mgould | 09:37 | |
| openstackgerrit | Carlos Camacho proposed openstack/puppet-tripleo: Add mongodb profiles https://review.openstack.org/304780 | 09:38 |
| *** akrivoka has joined #tripleo | 09:39 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Composable roles within services - MongoDB https://review.openstack.org/315051 | 09:39 |
| *** apetrich has quit IRC | 09:48 | |
| *** veteran has quit IRC | 09:50 | |
| *** milan has quit IRC | 09:54 | |
| *** apetrich has joined #tripleo | 09:54 | |
| *** ifarkas has quit IRC | 09:55 | |
| *** julim has quit IRC | 09:55 | |
| *** zoliXXL is now known as zoli|lunch | 09:57 | |
| *** ifarkas has joined #tripleo | 09:58 | |
| *** milan has joined #tripleo | 09:59 | |
| *** rhallisey has joined #tripleo | 10:02 | |
| *** julim has joined #tripleo | 10:03 | |
| *** dtantsur|bbl is now known as dtantsur | 10:06 | |
| *** apetrich has quit IRC | 10:06 | |
| *** dsariel has quit IRC | 10:07 | |
| *** apetrich has joined #tripleo | 10:08 | |
| *** snecklifter has quit IRC | 10:14 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Add Major upgrade function to the tripleo ci script https://review.openstack.org/323750 | 10:16 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: [NO MERGY] Test a fake periodic job https://review.openstack.org/229789 | 10:18 |
| *** larstobi has quit IRC | 10:20 | |
| *** yolanda has quit IRC | 10:22 | |
| *** samuelBartel has quit IRC | 10:23 | |
| *** snecklifter has joined #tripleo | 10:30 | |
| *** fzdarsky__ is now known as fzdarsky|lunch | 10:31 | |
| chem | gfidente: hi, there https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/manifests/overcloud_controller_pacemaker.pp#L97-L102 you've set the default timeout for resource operation on services. Did you comment mean that you'd rather have a command like this: "pcs resource op defaults timeout=200s" available ? | 10:33 |
| *** akuznetsov has quit IRC | 10:34 | |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Add Major upgrade function to the tripleo ci script https://review.openstack.org/323750 | 10:41 |
| *** ccamacho is now known as ccamacho|lunch | 10:44 | |
| jaosorior | sooo commits in CI are again taking a looong time; has anybody had a chance to check what's up? | 10:49 |
| *** ifarkas has quit IRC | 10:52 | |
| *** larstobi has joined #tripleo | 10:53 | |
| *** paramite is now known as paramite|afk | 10:58 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-heat-templates: Enable proxy header handling for nova https://review.openstack.org/323792 | 11:00 |
| *** snecklifter has quit IRC | 11:06 | |
| *** weshay has joined #tripleo | 11:08 | |
| *** apetrich has quit IRC | 11:11 | |
| *** apetrich has joined #tripleo | 11:12 | |
| *** slagle has joined #tripleo | 11:13 | |
| *** lucasagomes is now known as lucas-hungry | 11:16 | |
| *** snecklifter has joined #tripleo | 11:18 | |
| *** paramite|afk is now known as paramite | 11:20 | |
| gfidente | chem, actually so and so | 11:21 |
| gfidente | the good thing of doing it like that in puppet is that it only applies to those resources | 11:21 |
| gfidente | which are managed by the Service class | 11:21 |
| gfidente | not to those managed by the resource agents | 11:21 |
| chem | gfidente: oki, that what I though as well. Just wanted to be sure that I didn't misunderstand your comment. Thanks. | 11:22 |
| gfidente | chem, ty : | 11:22 |
| gfidente | :) | 11:22 |
| gfidente | so maybe setting a default is also nice | 11:22 |
| gfidente | but we won't use it in that context | 11:22 |
| chem | gfidente: ack | 11:23 |
| *** dciabrin has quit IRC | 11:25 | |
| jaosorior | ccamacho: the mongodb patch is now broken :/ | 11:26 |
| jaosorior | that's because the replset is being initializes with an undefined value | 11:26 |
| jaosorior | ccamacho|lunch ^^ | 11:27 |
| *** colonwq has joined #tripleo | 11:28 | |
| *** ccamacho|lunch is now known as ccamacho | 11:28 | |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-heat-templates: Composable roles within services - MongoDB https://review.openstack.org/315051 | 11:28 |
| ccamacho | jaosorior, fixed The thing is that when removing the duplicated code, missed to add the rplset name | 11:28 |
| jaosorior | ccamacho: The issue is in puppet-tripleo | 11:29 |
| slagle | derekh: hi, https://review.openstack.org/#/c/323516/ is failing b/c the dlrn repo is not setup as a repo for dep resolution during the actual build of nova | 11:29 |
| slagle | derekh: do you have any idea why that might be? | 11:29 |
| ccamacho | jaosorior, this is the error http://paste.openstack.org/show/506921/ just have deployed 5 mins ago in my local env | 11:30 |
| jaosorior | ccamacho: Did you check my feedback in the puppet-tripleo patch? | 11:30 |
| jaosorior | ccamacho: Even puppet-lint was failing because of the same thing https://review.openstack.org/#/c/304780/ | 11:31 |
| *** rodrigods has quit IRC | 11:32 | |
| ccamacho | jaosorior, checking it, ok ill do the same that as you did for conn_validator | 11:32 |
| *** rodrigods has joined #tripleo | 11:32 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: DO NOT MERGE testing 319345 https://review.openstack.org/323369 | 11:34 |
| *** links has quit IRC | 11:34 | |
| *** ifarkas has joined #tripleo | 11:36 | |
| openstackgerrit | Martin André proposed openstack/instack-undercloud: Install tripleo-common sudoers files https://review.openstack.org/323692 | 11:36 |
| openstackgerrit | Martin André proposed openstack/instack-undercloud: Create validations user during undercloud install https://review.openstack.org/322893 | 11:36 |
| *** dciabrin has joined #tripleo | 11:36 | |
| jaosorior | ccamacho: Do you have any idea why we are not using the conn_validator in the non-ha case? | 11:36 |
| jaosorior | that's one of the only differences really | 11:36 |
| jaosorior | marios: Do you happen to know why mongodb_conn_validator is only used in the ha case? | 11:37 |
| mandre | jaosorior: I just made the creation of the validations user optional, and disabled by default | 11:38 |
| mandre | thanks for the feedback | 11:38 |
| jaosorior | mandre: dude that was quick. Thanks | 11:38 |
| jaosorior | let me check it out again | 11:38 |
| ccamacho | jaosorior, from current THT on conn_validator # NOTE (spredzy) : The replset can only be run once all the nodes have joined the cluster. | 11:38 |
| *** pkovar has joined #tripleo | 11:39 | |
| jaosorior | ccamacho: Well... not sure if we can actually use the replset in the non-ha case | 11:39 |
| jaosorior | so then, why is it there? | 11:39 |
| *** zoli|lunch is now known as zoliXXL | 11:42 | |
| ccamacho | mm if no ha is used, I think we might remove it.. As it wont be needed | 11:42 |
| mandre | jaosorior: haha, no rush, take your time | 11:43 |
| mandre | i just happened to have the file opened in my editor | 11:44 |
| jaosorior | ccamacho: then that would make things way easier | 11:44 |
| ccamacho | gfidente, is really needed the replset config for non-ha ? | 11:45 |
| ccamacho | he added a comment before, he might have the answer | 11:45 |
| EmilienM | hello tripleo | 11:45 |
| matbu | o/ | 11:45 |
| jaosorior | ccamacho: And today is a good day to ask him. He'll be away for a couple of weeks O_O | 11:45 |
| ccamacho | yeahp ^ | 11:46 |
| gfidente | ccamacho, yep | 11:46 |
| *** links has joined #tripleo | 11:47 | |
| gfidente | non-ha doesn't support multiple nodes yet | 11:47 |
| gfidente | but the goal is to make it so | 11:47 |
| gfidente | so we setup some services like rabbitmq or mongodb for that | 11:47 |
| ccamacho | gfidente ack | 11:48 |
| gfidente | so it's not needed for the single controller scenario to pass CI but we won't take away the 'possibility' | 11:49 |
| gfidente | EmilienM, :) | 11:51 |
| gfidente | have 10 mins for me? | 11:52 |
| gfidente | maybe 15? | 11:52 |
| EmilienM | sure | 11:52 |
| gfidente | so the cinder submission | 11:52 |
| gfidente | I get this | 11:53 |
| gfidente | Error: Could not find dependency Package[cinder] for Service[cinder-api] at /etc/puppet/modules/cinder/manifests/ | 11:53 |
| gfidente | api.pp:237 | 11:53 |
| jaosorior | ccamacho: Lets try putting the conn_validator stuff that's in the ha, to the common mongodb manifest. that way it's easier to just use the common stuff there | 11:53 |
| gfidente | EmilienM, and 'cinder' should be mapped to 'openstack-cinder' for RedHat distros | 11:53 |
| EmilienM | gfidente: do you have ::cinder declared? | 11:53 |
| gfidente | (family) | 11:53 |
| gfidente | EmilienM, oh I probably haven't, let me check | 11:53 |
| EmilienM | you need it everywhere cinder runs | 11:54 |
| EmilienM | it's the same thing for all other modules | 11:54 |
| EmilienM | ::cinder, ::nova, etc. It's common bits needed everywhere the service is running | 11:54 |
| gfidente | I am including ::cinder from base yes | 11:55 |
| gfidente | https://review.openstack.org/#/c/310939/15/manifests/profile/base/cinder.pp | 11:55 |
| gfidente | which is included by api.pp | 11:56 |
| gfidente | https://review.openstack.org/#/c/310939/15/manifests/profile/base/cinder/api.pp | 11:56 |
| EmilienM | mhh, let me check | 11:56 |
| gfidente | thanks :) | 11:56 |
| gfidente | EmilienM++ | 11:56 |
| EmilienM | ah easy | 11:57 |
| EmilienM | you include ::cinder only at step 4 and beyond | 11:57 |
| EmilienM | but you need it also at step 3 | 11:57 |
| gfidente | where? | 11:58 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates: [EARLY WIP] Composable Upgrades https://review.openstack.org/321416 | 11:58 |
| EmilienM | see the 2 links you just sent | 11:58 |
| jaosorior | ccamacho: Makes sense? | 11:58 |
| gfidente | ah righ shit | 11:58 |
| EmilienM | :) | 11:58 |
| gfidente | god, thanks :) | 11:58 |
| ccamacho | yeahp only one common file Im playing with it just a sec | 11:59 |
| ccamacho | jaosorior ^ | 11:59 |
| jaosorior | ccamacho++ | 11:59 |
| *** egafford has quit IRC | 12:01 | |
| openstackgerrit | Giulio Fidente proposed openstack/puppet-tripleo: Add Cinder API/Scheduler/Volume roles https://review.openstack.org/310939 | 12:02 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates: [EARLY WIP] Composable Upgrades https://review.openstack.org/321416 | 12:03 |
| *** jpena is now known as jpena|lunch | 12:03 | |
| *** noslzzp has quit IRC | 12:03 | |
| *** noslzzp has joined #tripleo | 12:04 | |
| derekh | slagle: looking | 12:04 |
| *** MaxPC has joined #tripleo | 12:05 | |
| jistr | rhallisey, marios, matbu: ^ trying a more conventional approach. No external scripting needed. We lose the ability to just "drop a yaml here and it will be picked up at the right time", as we have to list the plays we expect to run, but we don't lose the ability to upgrade things in parallel. (e.g. run a few tasks from neutron-server upgrade, then run a few tasks from neutron-plugin-acme-inc upgrade, then | 12:07 |
| jistr | run tasks from neutron-server upgrade again -- i think this is important to support upgrading 3rd party plugins of various projects) | 12:07 |
| *** pblaho has quit IRC | 12:08 | |
| *** jcoufal has joined #tripleo | 12:08 | |
| jistr | that is, unless we want to bake all neutron plugins into a single neutron role, which doesn't sound ok to me -- it wouldn't follow the composable services structure on the Heat side | 12:08 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Better repo manipulation tools https://review.openstack.org/227563 | 12:09 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Cherrypick nova for https://bugs.launchpad.net/tripleo/+bug/1585641 https://review.openstack.org/323516 | 12:09 |
| openstack | Launchpad bug 1585641 in tripleo "CI: jobs failing with Message: No valid host was found. There are not enough hosts available., Code: 500"" [High,Triaged] - Assigned to James Slagle (james-slagle) | 12:09 |
| slagle | derekh: nm; i think that fixes it ^^ :) | 12:09 |
| slagle | need to pass --use-public when we call delorean | 12:09 |
| *** pblaho has joined #tripleo | 12:09 | |
| *** fzdarsky|lunch is now known as fzdarsky | 12:09 | |
| *** rbrady has joined #tripleo | 12:10 | |
| matbu | jistr: did you look at https://github.com/matbu/ansible-tripleo-upgrade | 12:10 |
| derekh | slagle: hmm, ya looks like something we'd need, I havn't come across --use-public before | 12:11 |
| *** jtomasek has quit IRC | 12:11 | |
| jistr | matbu: looking | 12:11 |
| *** lucas-hungry is now known as lucasagomes | 12:12 | |
| *** dsariel has joined #tripleo | 12:13 | |
| *** jayg|g0n3 is now known as jayg | 12:17 | |
| *** openstackgerrit has quit IRC | 12:18 | |
| *** openstackgerrit has joined #tripleo | 12:18 | |
| jistr | matbu: it's cool, but we may need more flexibility than what tripleo-upgrade provides. Not all services might follow stop-update-dbsync-start pattern always (we will be upgrading non-openstack services too) | 12:18 |
| matbu | jistr: ok, for non-openstack services, we could set up dedicated roles | 12:20 |
| jistr | matbu: also this would be good to split per-service, as we want separation between the services on code level, to make it possible for people to just edit their own service https://github.com/matbu/ansible-tripleo-upgrade/blob/master/roles/tripleo-upgrade/templates/db_sync.sh.j2 | 12:20 |
| social | I think we'll need dedicaterd roles anyway | 12:20 |
| social | *dedicated | 12:20 |
| *** jtomasek has joined #tripleo | 12:20 | |
| EmilienM | gfidente: can you review and +2 https://review.openstack.org/#/c/323297/ and https://review.openstack.org/#/c/323295/ so we can land the puppet patches | 12:20 |
| EmilienM | jaosorior: fyi ^ | 12:20 |
| jaosorior | EmilienM++ | 12:21 |
| gfidente | so upgrade is failing | 12:23 |
| gfidente | because we don't have the matching puppet part merged yet right? | 12:23 |
| jaosorior | gfidente: yep | 12:23 |
| *** dprince has joined #tripleo | 12:23 | |
| gfidente | ok careful | 12:23 |
| gfidente | I did same mistake with cinder | 12:23 |
| gfidente | :) | 12:23 |
| gfidente | going to +a | 12:23 |
| jaosorior | EmilienM, gfidente: Either way, the puppet patches need to merge first | 12:23 |
| gfidente | to make upgrade to work | 12:24 |
| EmilienM | yes | 12:24 |
| gfidente | ack going | 12:24 |
| EmilienM | upgrade wont work in this case, we need to land the puppet patches and then THT and then upgrade works | 12:24 |
| *** pradk has quit IRC | 12:24 | |
| jaosorior | EmilienM: So, time to land the puppet patches then? | 12:24 |
| EmilienM | ok go | 12:25 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates: [EARLY WIP] Composable Upgrades https://review.openstack.org/321416 | 12:26 |
| EmilienM | gfidente: remove your +A please, otherwise patch will never merge | 12:26 |
| EmilienM | gfidente: just put +2 | 12:26 |
| jistr | matbu: also the tripleo-upgrade pattern doesn't allow us to upgrade e.g. a neutron plugin at the same time as neutron. I posted an example to that review ^^ as i think this is something that we tend to miss in the alternative approaches. https://review.openstack.org/#/c/321416/7/ansible/upgrades/neutron/main.yml | 12:27 |
| *** trown|outtypewww is now known as trown | 12:27 | |
| gfidente | oh these will merge because it's only the lint/syntax jobs voting | 12:27 |
| jistr | rhallisey: ^ | 12:27 |
| ccamacho | jaosorior, now the very 80 first lines are kinda the same, now is easier to remove the dupe code | 12:27 |
| matbu | jistr: ok, when you say at the same time, it mean, in parallel ? | 12:28 |
| *** samuelBartel has joined #tripleo | 12:28 | |
| jaosorior | ccamacho: cool. So then is the next step to move the common stuff? | 12:28 |
| *** jdob has quit IRC | 12:28 | |
| *** jdob has joined #tripleo | 12:29 | |
| ccamacho | jaosorior, indeed, let me recheck the code and Ill create the common manifest | 12:29 |
| jaosorior | cool | 12:29 |
| ccamacho | jaosorior, based on what you did with the conn_validater | 12:30 |
| jistr | matbu: yea. stop neutron, upgrade neutron, upgrade 3rdparty-company-neutron-plugin, start neutron | 12:30 |
| rhallisey | jistr, what I was thinking we could do is have a templated include: - include {{ plugin }} | 12:30 |
| *** dsariel has quit IRC | 12:30 | |
| jaosorior | ccamacho: Well, you do as you see fit for that | 12:30 |
| rhallisey | and we have a selection of playbooks to choose from | 12:30 |
| jaosorior | doesn't need to be the same | 12:30 |
| jistr | rhallisey: that's essentially what i'm doing above ^^ :) | 12:30 |
| social | jistr: with what you plan to generate upgrades.yaml? | 12:31 |
| rhallisey | jistr, yes exactly, it would be the same method | 12:31 |
| rhallisey | it could be applied across each out solutions | 12:31 |
| *** numans has quit IRC | 12:31 | |
| jistr | rhallisey: it's doing the same `when: ` selection as the other roles, based on the info that comes from Heat, so no special rules required for 3rd party plugins | 12:32 |
| jistr | social: you mean what i called full-upgrade.yaml? i don't think we'll generate that, at least not in first iteration | 12:32 |
| jistr | social: all approaches we have so far have statically defined upgrade procedure, and roles are being selected based on what's enabled where | 12:33 |
| rhallisey | jistr, nice | 12:34 |
| rhallisey | So in the method of the templated include, we could have a j2 file with some logic the file to be added. Essentially it gets the same result | 12:34 |
| social | jistr: now I'm pondering how we'd reuse this also for minor version updates | 12:37 |
| jistr | rhallisey: ok, perhaps it's worth prototyping too, i'm just concerned that templated include means we need to come up with all such extension points upfront, but the `when: ` approach can be applied anywhere. E.g. the 3rd party neutron plugin might have a repo switching task too, and it can seamlessly plug into the repo switching phase too | 12:38 |
| jistr | https://review.openstack.org/#/c/321416/7/ansible/upgrades/repository_switch/execute/neutron-plugin-acme-inc.yml | 12:38 |
| jistr | rhallisey: i'm not sure we'd be able to put the templated include everywhere someone will need it, and maybe we'd put it somewhere where no one will need it :) | 12:38 |
| *** numans has joined #tripleo | 12:39 | |
| jistr | social: yea that's a good question, i was wondering about it before going to sleep actually, and i had to force myself to stop or i wouldn't get enough sleep :D | 12:40 |
| jistr | social: i think we'll have to have separate *playbook(s)* for minor update, but we should be able the same *roles* | 12:41 |
| jistr | *to use the same roles | 12:41 |
| *** rlandy has joined #tripleo | 12:41 | |
| *** itamarl has joined #tripleo | 12:42 | |
| jistr | social: we could have a flag whether to enable db sync perhaps, true for upgrades, false for updates | 12:43 |
| *** myoung has joined #tripleo | 12:43 | |
| matbu | jistr: yep or maybe tags could helpful here | 12:44 |
| rhallisey | jistr, idk I think we can. I think its a matter of building more advanced use cases as time goes on. | 12:44 |
| *** myoung|mtg has quit IRC | 12:45 | |
| marios | jistr: ack thanks. will have a look later/added to reviews for tomorrow ... on a call now and want to finish testing the upgrades scripts today | 12:45 |
| *** fultonj has joined #tripleo | 12:46 | |
| jistr | social: the high level workflow is very different though, quite likely "stop one/subset of services cloud-wide, upgrade cloud-wide, start cloud-wide", vs. "stop all services on a one/subset of nodes, upgrade there, start there" | 12:46 |
| jistr | we could do minor updates one service at a time too, but rolling the nodes + rolling the services means N*M time complexity, so minor update on larger environments with a bunch of services could take a *long* time :D | 12:47 |
| marios | jistr: (trying to follow some of the chat... i see you mentioned the neutron case (some services expected to be disabled for plugins)... and being able to at least 'decompose' to that level... anyway will take a proper look later | 12:48 |
| *** akshai has joined #tripleo | 12:49 | |
| jistr | matbu: yea tags could be used for skipping | 12:49 |
| marios | jistr: testing v8 of the m..n upgrade script rework (seems to be starting things OK so far) on controller upgrade. | 12:50 |
| *** Goneri has joined #tripleo | 12:52 | |
| jistr | marios: great :) | 12:53 |
| trown | sshnaidm: it looks like that tripleo-quickstart tempest patch is just hanging when running tempest... tried it locally and I see the same thing | 12:53 |
| marios | jistr: just finished so fwiw te script is syntactically OK. really it isn't execising systemd/at all/much at this point, that is the biggest issue... | 12:53 |
| sshnaidm | trown, hmm..I'll look what's wrong | 12:53 |
| marios | jistr: will update the review with note in sec | 12:54 |
| trown | sshnaidm: I wonder if it is the keystone v3 tests http://chunk.io/f/ef96cca1770343448c3ed8feec296247 | 12:54 |
| openstackgerrit | James Slagle proposed openstack/instack-undercloud: Regenerate undercloud.conf.sample https://review.openstack.org/323850 | 12:54 |
| sshnaidm | trown, I'll try to exclude them | 12:55 |
| sshnaidm | trown, is it minimal.yml ? | 12:56 |
| trown | sshnaidm: ya | 12:56 |
| *** tzumainn has joined #tripleo | 12:57 | |
| jaosorior | one liner patch passing CI https://review.openstack.org/#/c/323792/ :D | 12:58 |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart: Tempest support for quickstart https://review.openstack.org/318751 | 13:00 |
| trown | jaosorior: is that needed to work with latest nova? or does our CI not cover it? | 13:00 |
| *** bswartz has joined #tripleo | 13:01 | |
| jaosorior | trown: CI doesn't cover it | 13:01 |
| jaosorior | trown: if we would run tempest with SSL setup it would fail | 13:02 |
| trown | jaosorior: ah ok, I wonder if there would be some way to tweak the ping test to do the same | 13:02 |
| trown | seems like something we need to be CI'ing | 13:03 |
| jaosorior | trown: well... TLS is not even properly tested in devstack | 13:04 |
| jaosorior | FWIW | 13:04 |
| *** hjensas_ has quit IRC | 13:04 | |
| trown | jaosorior: I would like to set the bar for our CI higher than devstack :) | 13:04 |
| marios | jistr: i think you'd appreciate the paranoia associated with an upgrading having gone well... checking if the *actual* updated script was used https://paste.fedoraproject.org/373333/46478626/ | 13:04 |
| social | jistr: I think it should just order the nodes and go node by node, nothing else | 13:05 |
| trown | jaosorior: but ya, I don't have an immediate idea how to test it | 13:05 |
| marios | jistr: (noop upgrade... but just wanted to check it was actually using the new script and i wasn't just happy the existing thing had worked well :) ) | 13:05 |
| social | jistr: so same logic, stop everything on one node and do minor update | 13:05 |
| jistr | marios++ i always appreciate paranoia. I wasn't like that always, but upgrades taught me better :D | 13:05 |
| matbu | lol | 13:06 |
| *** jpena|lunch is now known as jpena | 13:07 | |
| marios | lol "I wasn't like that always" ... | 13:08 |
| openstackgerrit | Carlos Camacho proposed openstack/puppet-tripleo: Add mongodb profiles https://review.openstack.org/304780 | 13:11 |
| jistr | :) that's also why i'm trying to pack so much flexibility into the new approach, i think we'll need every drop, it's just not visible right now because we focus our PoCs on the simplest cases. If possible i'd like to skip situations like "we added ExtraConfig for vendor plugins, but it doesn't do too well if i need 2 vendor plugins at the same time, let's figure out something that will be more flexible". I'd | 13:11 |
| jistr | like us to start flexible :) | 13:11 |
| *** ayoung has joined #tripleo | 13:12 | |
| EmilienM | trown: can we promote tripleo CI to latest trunk? | 13:17 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/puppet-pacemaker: WIP: integrate PCS provider in the merge. https://review.openstack.org/310713 | 13:17 |
| trown | EmilienM: we need the job to pass :) | 13:17 |
| trown | EmilienM: but RDO got a promote this morning, so I think we should be able to get a pass today | 13:18 |
| EmilienM | cool | 13:18 |
| gfidente | guys | 13:18 |
| gfidente | you should all switch to radioparadise right now | 13:19 |
| gfidente | http://tunein.com/radio/Radio-Paradise-s13606/ | 13:19 |
| *** liverpooler has quit IRC | 13:19 | |
| gfidente | marios knows what I mean | 13:20 |
| gfidente | it's radioparadise or nothing | 13:20 |
| EmilienM | lol | 13:20 |
| trown | some solid cowbell | 13:20 |
| *** hjensas_ has joined #tripleo | 13:20 | |
| *** hjensas_ has quit IRC | 13:20 | |
| *** hjensas_ has joined #tripleo | 13:20 | |
| slagle | "Grazing in the Grass" | 13:20 |
| slagle | (no comment) | 13:20 |
| rbrady | jtomasek: re: https://review.openstack.org/#/c/300200 | 13:21 |
| rbrady | jtomasek: when a task in a workflow fails, the entire workflow is considered failed. | 13:21 |
| jaosorior | EmilienM: Got a +A for this? :D | 13:22 |
| *** andrey_veselov has joined #tripleo | 13:22 | |
| EmilienM | jaosorior: for what? | 13:22 |
| openstackgerrit | John Trowbridge proposed openstack-infra/tripleo-ci: [NO MERGY] Test a fake periodic job https://review.openstack.org/229789 | 13:23 |
| jaosorior | EmilienM https://review.openstack.org/#/c/323792/ | 13:23 |
| jaosorior | forgot to paste it before | 13:23 |
| jtomasek | rbrady: that is fine, but problem is that nodes registered before the one that failed are already registered in ironic | 13:23 |
| EmilienM | jaosorior: done | 13:23 |
| jtomasek | rbrady: and the workflow does not continue with moving them to correct state | 13:23 |
| jaosorior | EmilienM: Thanks | 13:23 |
| *** lblanchard has joined #tripleo | 13:24 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-docs: Add instructions on how to provision a custom Cinder backend https://review.openstack.org/323833 | 13:25 |
| jistr | social: sounds good re minor updates node by node. We've been doing it this way until now with the monolithic scripts and it worked fine. | 13:26 |
| social | jistr: any other way does not really make sense | 13:27 |
| jistr | yea | 13:27 |
| jtomasek | rbrady: the whole problem probably comes from the fact that each node is not registered in a separate subworkflow/action but it is done in batch which either succeeds or fails. And if it fails the nodes get to inconsistent state | 13:27 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Enable proxy header handling for nova https://review.openstack.org/323792 | 13:28 |
| jtomasek | (some are registered but in wrong state, some are lost) | 13:28 |
| rbrady | jtomasek: yes. I'm drawing out diagrams of the current workflows and proposed change | 13:28 |
| jtomasek | rbrady: sounds great! | 13:29 |
| openstackgerrit | Merged openstack/puppet-pacemaker: Pacemaker misc fixes https://review.openstack.org/323043 | 13:30 |
| *** olap has quit IRC | 13:32 | |
| *** olap has joined #tripleo | 13:33 | |
| *** paramite is now known as paramite|afk | 13:36 | |
| *** paramite|afk is now known as paramite | 13:40 | |
| *** pradk has joined #tripleo | 13:42 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/puppet-pacemaker: WIP: integrate PCS provider in the merge. https://review.openstack.org/310713 | 13:42 |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-quickstart: Enabled pacemaker by in minimal deployment https://review.openstack.org/323878 | 13:45 |
| openstackgerrit | Emilien Macchi proposed openstack/instack-undercloud: Update hiera parameter to use tripleo::haproxy https://review.openstack.org/323879 | 13:46 |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-quickstart: Enabled pacemaker by default in minimal deployment https://review.openstack.org/323878 | 13:46 |
| *** oshvartz has quit IRC | 13:47 | |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-docs: Add instructions on how to provision a custom Cinder backend https://review.openstack.org/323833 | 13:47 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-docs: Add instructions on how to provision a custom Cinder backend https://review.openstack.org/323833 | 13:48 |
| gfidente | guys, does anyone know what is going on CI with -ha failing on /sbin/pcs status | grep -q 'partition with quorum ? | 13:50 |
| *** links has quit IRC | 13:52 | |
| trown | panda: could you explain repo issue with tripleo-ci overcloud images? | 13:52 |
| panda | trown: there is no repo issue, the problem is that to successfully use the overcloud image, we have to use the same exact delorean hash with which it was created, and nothing mmore | 13:54 |
| trown | panda: "we have to use the exact delorean hash" on the undercloud? | 13:54 |
| trown | panda: isn' | 13:55 |
| trown | argh | 13:55 |
| panda | trown: yes | 13:55 |
| trown | panda: isn't that setup in the instack.qcow2 on the mirror server? | 13:55 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/tripleo-common: Updated from global requirements https://review.openstack.org/323090 | 13:55 |
| panda | trown: no, the instack.qcow2 has no delorean repo set up | 13:56 |
| trown | panda: so if we combine the instack.qcow2 and overcloud images from the same dir we would be good right? | 13:56 |
| trown | oh | 13:56 |
| panda | trown: yes, but only if we inject the right repo on the undercloud | 13:56 |
| panda | and there is only 1 right repo | 13:56 |
| trown | that does seem problematic... I guess tripleo-ci is dealing with that by injecting repos into overcloud images? | 13:56 |
| trown | derekh: ^ | 13:56 |
| panda | trown: no, the overcloud image is always rebuilt, hte image is uploaded but never used | 13:57 |
| trown | derekh: wdyt of putting repos on the instack.qcow2? | 13:57 |
| *** mcornea has quit IRC | 13:57 | |
| trown | panda: I am pretty sure it is used for some jobs (tripleo-heat-templates for one) | 13:58 |
| panda | trown: the undercloud repo changes on every test | 13:58 |
| trown | panda: it is just not used for any jobs that could have an element change that could effect building the image | 13:58 |
| panda | trown: because it has to be modified with the changes to be tested | 13:58 |
| panda | trown: and if you change the underlying delorean repo in the undercloud, you CANNOT use an old overcloud image | 13:59 |
| *** ramishra has quit IRC | 13:59 | |
| *** mcornea has joined #tripleo | 13:59 | |
| *** ramishra has joined #tripleo | 14:01 | |
| *** yamahata has joined #tripleo | 14:01 | |
| trown | panda: this big function is what takes care of updating the overcloud images when they are used https://github.com/openstack-infra/tripleo-ci/blob/master/scripts/common_functions.sh#L73-L162 | 14:02 |
| derekh | panda: ya, what trown said, we take the cached image, add the repos and do a yum update | 14:03 |
| panda | does "openstack overcloud image build" checks if they images exist and in case skips ? | 14:03 |
| derekh | panda: yup it skips the build step if they exist | 14:04 |
| panda | ... ok | 14:04 |
| panda | that explains some things | 14:04 |
| *** egafford has joined #tripleo | 14:06 | |
| * gfidente just got a new volume on cinder/roles deployment :) | 14:07 | |
| gfidente | took forever | 14:07 |
| gfidente | BUT | 14:07 |
| gfidente | at last | 14:07 |
| jistr | gfidente: re the CI fail, it's just one job so far, no? | 14:08 |
| panda | gfidente: not bad (the radio) thanks. | 14:08 |
| gfidente | panda :) | 14:08 |
| jistr | gfidente: cistatus.html looks fairly green | 14:08 |
| *** paramite is now known as paramite|afk | 14:08 | |
| trown | gfidente: ya I am also enjoying the radio | 14:08 |
| gfidente | guys actually | 14:09 |
| gfidente | today is not that great | 14:09 |
| jistr | what radio | 14:09 |
| trown | http://tunein.com/radio/Radio-Paradise-s13606/ | 14:09 |
| * jistr eager to get listen to some new music :) | 14:09 | |
| gfidente | aaaaaaaaaaaaahhhhhhhhhhhhhhhh | 14:09 |
| jistr | thx! will try | 14:09 |
| gfidente | like this song is pretty bad | 14:10 |
| gfidente | BUT | 14:10 |
| *** paramite|afk is now known as paramite | 14:10 | |
| gfidente | in general the selection is never terrible | 14:10 |
| gfidente | so you get highs and lows | 14:11 |
| gfidente | but you're never pushed to switch to something else | 14:11 |
| gfidente | or at least, that's my take | 14:11 |
| panda | gfidente: oh, it's on the playlist of rhythmbox by default | 14:12 |
| jistr | ah i like they offer higher quality streams. Often radios are just 128kbps mp3 | 14:12 |
| gfidente | yeah was going to say the same | 14:12 |
| gfidente | make sure to pick some good quality stream | 14:12 |
| gfidente | jistr, didn't know you were into hifi | 14:12 |
| gfidente | we can exchage gear | 14:13 |
| jistr | i'm building my own actually :P | 14:13 |
| jistr | a long time now though | 14:13 |
| gfidente | what , the ampli? | 14:13 |
| jistr | but just a player, i wouldn't build an amp or speakers well | 14:13 |
| jistr | don't have enough knowledge to do that | 14:13 |
| gfidente | so the player | 14:13 |
| gfidente | I turned the rspi with hdmi output | 14:14 |
| gfidente | into wifi receiver | 14:14 |
| gfidente | so it can stream digitally via hdmi into the amp | 14:14 |
| gfidente | and do the d/a conversion in the amp which is better quality | 14:14 |
| gfidente | and stream from any phone/pc at home | 14:14 |
| gfidente | to the rspi | 14:14 |
| gfidente | so I think hdmi is not bitperfect but it's lossless | 14:15 |
| jistr | yea exactly, same here i want to stream digi out so that DAC can be done by something that can do it well | 14:15 |
| gfidente | 1.4hdmi up to 7 channels should be lossless | 14:15 |
| gfidente | ack | 14:15 |
| jistr | i have this for the RPi https://www.hifiberry.com/digiplus/ | 14:16 |
| gfidente | yeah cool | 14:16 |
| jistr | coax is nice b/c i can plug it into my headphone DAC | 14:16 |
| gfidente | and yes you get bit-perfect over spdif | 14:16 |
| gfidente | and what headphones are they? | 14:17 |
| gfidente | SPAM SPAM SPAM SPAM | 14:18 |
| jistr | beats solo 2 wired | 14:18 |
| jistr | haha yea we should move the discussion elsewhere :D | 14:18 |
| gfidente | ah never tried | 14:18 |
| EmilienM | gfidente: I'm more a http://www.di.fm guy | 14:19 |
| gfidente | EmilienM, ah | 14:20 |
| gfidente | EmilienM, so I add it here | 14:20 |
| gfidente | and get back over some time | 14:20 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/puppet-pacemaker: WIP: integrate PCS provider in the merge. https://review.openstack.org/310713 | 14:20 |
| *** paramite has quit IRC | 14:21 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/puppet-tripleo: WIP: integration of the new puppet pacemaker. https://review.openstack.org/309069 | 14:21 |
| honza | When writing mistral workflows, we need to send a message to zaqar when an error occurs. Is there a standard error status? FAIL? For success we use SUCCESS. cc dprince rbrady | 14:22 |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Build and upload instack.raw image https://review.openstack.org/315072 | 14:22 |
| dprince | honza: FAILED would be fine I think | 14:24 |
| dprince | honza: FAILURE maybe | 14:24 |
| honza | dprince: SUCCESS and FAILURE then, cool | 14:25 |
| *** d0ugal has quit IRC | 14:25 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/puppet-tripleo: WIP: integration of the new puppet pacemaker. https://review.openstack.org/309069 | 14:25 |
| dprince | honza: actually we are using 'FAILED' in baremetal.yaml already | 14:26 |
| *** ramishra has quit IRC | 14:26 | |
| dprince | honza: so maybe go w/ that | 14:26 |
| *** itamarl has quit IRC | 14:26 | |
| dprince | honza: status: FAILED | 14:26 |
| honza | dprince: that seems strange (verb vs noun) but +1 for consistency | 14:27 |
| *** ramishra has joined #tripleo | 14:27 | |
| *** jprovazn has quit IRC | 14:27 | |
| jrist | heh | 14:29 |
| ccamacho | guys I'm checking several submissions with this error http://paste.openstack.org/show/506977/ but not only in CI also getting the same error in my local environment | 14:32 |
| *** itamarl has joined #tripleo | 14:32 | |
| openstackgerrit | mathieu bultel proposed openstack/tripleo-quickstart: Remove hardcoded playbook file and add option to set it https://review.openstack.org/323951 | 14:33 |
| gfidente | ccamacho, so it seems indication of something going bad on compute nodes in step4 | 14:35 |
| gfidente | check the compute node messages log | 14:35 |
| *** myoung has quit IRC | 14:35 | |
| rbrady | honza: you may not be seeing the FAILED usage in the baremetal patch because you overwrote that file recently | 14:35 |
| *** myoung has joined #tripleo | 14:36 | |
| *** d0ugal has joined #tripleo | 14:36 | |
| *** liverpooler has joined #tripleo | 14:36 | |
| honza | rbrady: ugh, how is that even possible? all i did was rebase my patch on top of yours and gerrit decided to update the dependent patch? | 14:38 |
| ccamacho | gfidente, this is from a CI patch http://paste.openstack.org/show/506980/ | 14:38 |
| honza | rbrady: what's the best way to fix this? can one delete a patchset from gerrit? | 14:38 |
| gfidente | ccamacho, what's the CI patch? | 14:39 |
| ccamacho | this one https://review.openstack.org/#/c/318840/ | 14:39 |
| rbrady | honza: I guess it depends on how you rebased. if you used git review -d it should have pulled down the latest changes first | 14:39 |
| *** apetrich has quit IRC | 14:40 | |
| honza | rbrady: i checked out your branch, pulled the latest changes, then switched back to my branch and did "git rebase <your branch>" i think | 14:41 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: loadbalancer: remove controller_host https://review.openstack.org/323955 | 14:41 |
| *** Guest9136 has quit IRC | 14:42 | |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: loadbalancer: remove controller_host https://review.openstack.org/323955 | 14:42 |
| thrash | ccamacho: the ha job failed on step 3 | 14:42 |
| gfidente | ccamacho, Could not start Service[nova-compute]: Execution of '/usr/bin/systemctl start openstack-nova-compute' returned 1 | 14:42 |
| *** apetrich has joined #tripleo | 14:42 | |
| gfidente | ccamacho, nova log says AccessRefused: (0, 0): (403) ACCESS_REFUSED - Login was refused using authentication mechanism AMQPLAIN. For details see the broker logfile. | 14:43 |
| *** itamarl has quit IRC | 14:43 | |
| thrash | both upgrade and ha fail at step 3 | 14:43 |
| rbrady | honza: to fix I think you can checkout ps#7 and run git commit --amend && git review | 14:43 |
| ccamacho | I think that patch was passing CI before, also this patch have the same error http://logs.openstack.org/80/304780/18/check-tripleo/gate-tripleo-ci-f22-nonha/8bac2c0/logs/overcloud-novacompute-0/var/log/messages | 14:44 |
| thrash | ccamacho: it was. | 14:44 |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-common: Add baremetal workflows https://review.openstack.org/300200 | 14:46 |
| ccamacho | im deploying master THT locally to see the behavior | 14:46 |
| *** d0ugal has quit IRC | 14:48 | |
| *** d0ugal has joined #tripleo | 14:50 | |
| *** oshvartz has joined #tripleo | 14:50 | |
| *** coolsvap is now known as coolsvap|afk | 14:51 | |
| honza | rbrady: does #9 look right to you? on 300200 | 14:51 |
| * rbrady looks | 14:52 | |
| *** diazjf has joined #tripleo | 14:53 | |
| *** jprovazn has joined #tripleo | 14:54 | |
| gfidente | ccamacho, let's bring this up in scrum | 14:55 |
| gfidente | it might be connectivity | 14:55 |
| ccamacho | gfidente ack | 14:55 |
| sshnaidm | dprince, hi, can you please respond in https://review.openstack.org/#/c/321019/ ? thanks! | 14:56 |
| rbrady | honza: "git review -m 300200,7-9" shows no differences between the two patchsets. I think you fixed it | 14:56 |
| sshnaidm | trown, jobs didn't reach the tempest step, reran them again | 14:57 |
| *** olap has quit IRC | 14:58 | |
| dprince | sshnaidm: I replied. I'd rather see to environment variables to control things | 14:58 |
| dprince | sshnaidm: Running tempest does not supplant our TripleO ping test which I would see growing in functionality over time | 14:58 |
| trown | sshnaidm: was just about to do that. centos jenkins needed a restart so I had to abort the last run | 14:58 |
| *** jaosorior has quit IRC | 14:59 | |
| trown | sshnaidm: but I tested PS6 locally and it worked for me | 14:59 |
| sshnaidm | trown, so you were right, it's v3 tests.. | 14:59 |
| *** mcornea_ has joined #tripleo | 14:59 | |
| sshnaidm | trown, let's see this run | 14:59 |
| *** mcornea has quit IRC | 15:00 | |
| sshnaidm | dprince, ok, I'll add RUN_PING_TEST as well | 15:01 |
| *** masco has quit IRC | 15:02 | |
| sshnaidm | dprince, although I wonder why reinvent the wheel with pingtest if we have tempest and project integration tests almost for each project? | 15:04 |
| *** apetrich has quit IRC | 15:04 | |
| *** pcaruana has quit IRC | 15:04 | |
| *** apetrich has joined #tripleo | 15:06 | |
| sshnaidm | dprince, do you want it to be like this? https://paste.fedoraproject.org/373393/64793664/ | 15:08 |
| *** rhallisey has quit IRC | 15:08 | |
| dprince | sshnaidm: ping test isn't reienventing the wheel I think. Even if Tempest covers all the same APi calls as our ping test it will never be as fast I'm afraid | 15:08 |
| *** ebarrera has quit IRC | 15:08 | |
| *** numans has quit IRC | 15:09 | |
| dprince | sshnaidm: different focus areas, ping test is focussed on speed first then coverage | 15:09 |
| dprince | sshnaidm: where as Tempest is more focussed on coverage I think | 15:09 |
| *** rcernin has quit IRC | 15:09 | |
| *** d0ugal has quit IRC | 15:10 | |
| dprince | sshnaidm: ++ on your paste | 15:10 |
| *** diazjf has quit IRC | 15:10 | |
| Ablu | Is there any simple example somewhere which shows how i can create a customized image using diskimage-builder? http://docs.openstack.org/developer/diskimage-builder/developer/developing_elements.html#phase-subdirectories lists a lot of stuff, but i do not really understand it... I think an example could help me here... | 15:11 |
| sshnaidm | derekh, is it ok for you too? https://paste.fedoraproject.org/373393/64793664/ | 15:11 |
| *** d0ugal has joined #tripleo | 15:12 | |
| *** diazjf has joined #tripleo | 15:13 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci: Add tempest tests to periodic and experimental job https://review.openstack.org/321019 | 15:15 |
| Ablu | also, if i run the example from disk-image-create --help (disk-image-create -a amd64 -o ubuntu-amd64 vm ubuntu). The command fails with: /usr/sbin/grub-install: warning: this msdos-style partition label has no post-MBR gap; embedding won't be possible. | 15:16 |
| Ablu | /usr/sbin/grub-install: error: embedding is not possible, but this is required for cross-disk install. | 15:16 |
| Ablu | https://paste.fedoraproject.org/373396/64794158/ | 15:16 |
| *** mcornea_ is now known as mcornea | 15:17 | |
| *** pblaho has quit IRC | 15:17 | |
| Ablu | if i try disk-image-create -a amd64 -o fedora vm fedora | 15:18 |
| Ablu | I get 404 errors when trying to download: | 15:18 |
| Ablu | > GET /pub/fedora/linux/releases/21/Cloud/Images/x86_64/Fedora-Cloud-Base-20141203-21.x86_64.qcow2 HTTP/1.1 | 15:18 |
| Ablu | > Host: download.fedoraproject.org | 15:18 |
| *** pblaho has joined #tripleo | 15:18 | |
| *** tesseract has quit IRC | 15:18 | |
| *** milan has quit IRC | 15:18 | |
| thrash | Ablu: you can download an image to base it on and set DIB_LOCAL_IMAGE env var. | 15:18 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Add websocket utils module https://review.openstack.org/322611 | 15:19 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Use Mistral for baremetal registration https://review.openstack.org/322612 | 15:19 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Add support for Zaqar websocket messaging https://review.openstack.org/313955 | 15:19 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Enable Manila integration https://review.openstack.org/188137 | 15:20 |
| *** rhallisey has joined #tripleo | 15:32 | |
| *** diazjf has quit IRC | 15:32 | |
| *** rhallisey has quit IRC | 15:32 | |
| *** rhallisey has joined #tripleo | 15:32 | |
| *** tremble has quit IRC | 15:32 | |
| openstackgerrit | Merged openstack/tripleo-ui: Optimize Nodes and Roles requests in DeploymentPlan component https://review.openstack.org/323368 | 15:36 |
| *** mbound has joined #tripleo | 15:36 | |
| EmilienM | slagle: let's take it here | 15:37 |
| EmilienM | they are in RDO meeting | 15:37 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Trove Integration https://review.openstack.org/233240 | 15:38 |
| EmilienM | slagle: do you have logs handy of what broke? | 15:38 |
| *** rbrady_ has joined #tripleo | 15:38 | |
| gfidente | egafford, ^^ | 15:38 |
| trown | EmilienM: I noticed it when trying to promote | 15:38 |
| slagle | EmilienM: any recently failed job | 15:38 |
| trown | https://review.openstack.org/#/c/229789/48 | 15:38 |
| EmilienM | damn | 15:38 |
| EmilienM | looking at logs now | 15:38 |
| *** rbrady_ has quit IRC | 15:38 | |
| trown | compute node doesnt start nova-compute service | 15:38 |
| gfidente | egafford, https://review.openstack.org/#/c/233240/25..26 | 15:38 |
| EmilienM | our CI is also down, we have an AVC | 15:39 |
| EmilienM | http://logs.openstack.org/46/323546/2/check/gate-puppet-openstack-integration-3-scenario002-tempest-centos-7/28a4f06/console.html#_2016-06-01_14_24_47_770 | 15:39 |
| *** rbrady has quit IRC | 15:39 | |
| gfidente | actually CI is broken right now but this looks ready to merge to me! | 15:39 |
| *** rbrady has joined #tripleo | 15:39 | |
| slagle | EmilienM: AMQPLAIN login refused: user 'guest' can only connect via localhost | 15:39 |
| slagle | EmilienM: so the compute nodes can't connect to rabbit since they use the guest user | 15:39 |
| gfidente | egafford, only, should it be enabled by default? | 15:39 |
| EmilienM | damn | 15:39 |
| social | slagle: ^^ I think we should change from guest to openstack user | 15:39 |
| EmilienM | 2 different issues | 15:39 |
| social | slagle: other solution is 'loopback_users' => '[]', | 15:40 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Composable manila tripleo-heat-templates side https://review.openstack.org/315658 | 15:40 |
| egafford | gfidente: Quite probably not at this stage. I can remove it from the default ServiceList; that's reasonable. | 15:41 |
| trown | slagle: EmilienM, I will push for revert of the RDO change after the meeting | 15:41 |
| trown | as well as better communication about this stuff... | 15:41 |
| EmilienM | trown: thx | 15:42 |
| *** diazjf has joined #tripleo | 15:44 | |
| *** rhallisey has quit IRC | 15:44 | |
| *** rhallisey has joined #tripleo | 15:45 | |
| slagle | social: yea, i'd be fine with either, or a revert | 15:45 |
| slagle | fastest path to green | 15:45 |
| *** rhallisey has quit IRC | 15:45 | |
| Ablu | trown: hm. that fails with different error messages. I guess I am giving up on this path... | 15:45 |
| social | slagle: bikeshed issue, can I name new user tripleo? | 15:45 |
| *** rhallisey has joined #tripleo | 15:45 | |
| *** coolsvap|afk is now known as coolsvap | 15:46 | |
| slagle | social: sure | 15:46 |
| trown | if we have a path to quick fix without RDO revert that is better | 15:46 |
| social | I also have issue with apache mod_remoteip but that's on puppet rerun >.> | 15:47 |
| *** aufi has quit IRC | 15:49 | |
| slagle | filed a bug: https://bugs.launchpad.net/tripleo/+bug/1587961 | 15:50 |
| openstack | Launchpad bug 1587961 in tripleo "tripleo-ci: all jobs failing with AMQPLAIN login refused: user 'guest' can only connect via localhost" [Undecided,New] | 15:50 |
| trown | social: are you putting up a patch for ^ or should I ask for revert in RDO? | 15:50 |
| *** slagle changes topic to "CI failing: https://bugs.launchpad.net/tripleo/+bug/1587961 | TripleO: http://tripleo.org/ | https://wiki.openstack.org/wiki/TripleO | Meetings On Tuesdays at 14:00 UTC in #openstack-meeting-alt" | 15:50 | |
| openstackgerrit | Lukas Bezdicka proposed openstack/tripleo-heat-templates: Don't use guest as rabbitmq user https://review.openstack.org/323995 | 15:51 |
| social | hmm now for undercloud | 15:51 |
| social | trown: ^^ should do it | 15:52 |
| trown | social: awesome | 15:52 |
| *** olap has joined #tripleo | 15:53 | |
| *** rajinir has joined #tripleo | 15:54 | |
| *** mikelk has quit IRC | 15:54 | |
| slagle | social: since you pushed the first patch, the bug is now assigned to you :) | 15:54 |
| *** shanky_ is now known as shanky | 15:55 | |
| *** shanky has left #tripleo | 15:55 | |
| *** mcornea has quit IRC | 15:56 | |
| *** mcornea has joined #tripleo | 15:57 | |
| *** penick has quit IRC | 15:58 | |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Enable Manila integration - as a composable controller service https://review.openstack.org/188137 | 15:59 |
| pabelanger | EmilienM: any TL;DR of the CI issues on tripleo? | 15:59 |
| *** mcornea has quit IRC | 15:59 | |
| EmilienM | pabelanger: https://review.openstack.org/323995 | 15:59 |
| *** mcornea has joined #tripleo | 15:59 | |
| *** zoliXXL is now known as zoli|gone | 15:59 | |
| pabelanger | derekh: dprince: I had to revert nodepool back to using 80GB for centos-7 slaves. The issue now is the size of the overcloud image after we convert it from qcow2 to raw | 16:00 |
| *** zoli|gone is now known as zoli_gone-proxy | 16:00 | |
| slagle | social: is there somethign already on the puppet side that is going to create that tripleo rabbitmq user? | 16:01 |
| *** penick has joined #tripleo | 16:01 | |
| pabelanger | dprince: derekh: At this point in time, I have idea how to proceed. But we need to migrate tripleo away from fedora-22 as 1) we are going to be removing snapshot builds from nodepool (which breaks fedora-22 slave). 2) is blocking a spec to use DIB in nodepool | 16:02 |
| *** andrey_veselov has quit IRC | 16:02 | |
| derekh | pabelanger: how big is the raw image when its converted ? | 16:03 |
| derekh | pabelanger: I'm wondering if anything else can be deleted | 16:04 |
| *** ooolpbot has joined #tripleo | 16:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 16:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1587961 | 16:10 |
| *** ooolpbot has quit IRC | 16:10 | |
| openstack | Launchpad bug 1587961 in tripleo "tripleo-ci: all jobs failing with AMQPLAIN login refused: user 'guest' can only connect via localhost" [Critical,Confirmed] - Assigned to Lukas Bezdicka (social-b) | 16:10 |
| gfidente | social, ^^ you actually know what is going on? | 16:10 |
| gfidente | or what caused it? | 16:10 |
| gfidente | oh slagle does | 16:11 |
| gfidente | slagle, I think we can allow the guest account to login from non-localhost too | 16:11 |
| gfidente | given we set a password for it | 16:11 |
| gfidente | I think allowing from all networks will just bring back the status to what it was before latest update | 16:12 |
| *** fragatina has joined #tripleo | 16:12 | |
| gfidente | maybe then we can trim down the allowed list to the internal networks | 16:12 |
| *** dtantsur is now known as dtantsur|afk | 16:13 | |
| *** ccamacho is now known as ccamacho|afk | 16:13 | |
| *** fragatina has quit IRC | 16:17 | |
| weshay | trown, can we merge this one? https://review.openstack.org/#/c/323951/ | 16:17 |
| pradk | Hey guys, can i get some help with this patch https://review.openstack.org/#/c/318413/ seeing this error .. "The Referenced Attribute (ControllerServiceChain config_settings) is incorrect. " | 16:19 |
| pradk | i'm passing in all the params for the roles as expected.. i'm doing something wrong wrt config_settings but the error is not obvious as to where | 16:20 |
| pabelanger | derekh: not sure, I have a patch up to use the centos-minimal and growroot elements, I believe that will help cut down on the image size. It looks like you create instack.qcow2 with a static 30GB file size. Not sure about overcloud | 16:26 |
| social | slagle: the user should be created by puppet, checking the code | 16:29 |
| openstackgerrit | Lukas Bezdicka proposed openstack/tripleo-heat-templates: Don't use guest as rabbitmq user https://review.openstack.org/323995 | 16:33 |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart: Add ability for images job to run with pending deps repo https://review.openstack.org/324008 | 16:34 |
| derekh | pabelanger: looks like the size of the qcow file it 1.2G, the size of the raw file is 2.9 | 16:34 |
| *** mcornea has quit IRC | 16:35 | |
| openstackgerrit | John Trowbridge proposed openstack/tripleo-quickstart: Add ability for images job to run with pending deps repo https://review.openstack.org/324008 | 16:36 |
| *** apetrich has quit IRC | 16:36 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Do not merge https://review.openstack.org/324010 | 16:38 |
| *** apetrich has joined #tripleo | 16:38 | |
| *** yamahata has quit IRC | 16:38 | |
| *** rasca has quit IRC | 16:41 | |
| pabelanger | derekh: so we need 2 things then, 1) bump HDD another 5GB (to 30GB) 2) stop using devstack to setup the slave (it consumes 10GB) and switch to zuul-cloner | 16:41 |
| pabelanger | we should do 2 regardless, but requires more work | 16:41 |
| *** jtomasek has quit IRC | 16:41 | |
| *** samuelBartel has quit IRC | 16:42 | |
| derekh | pabelanger: wouldn't a bump of 5GB bring us to 25GB? | 16:43 |
| derekh | pabelanger: are 1 and 2 both required together? | 16:43 |
| gfidente | EmilienM, https://github.com/puppetlabs/puppetlabs-rabbitmq/pull/479 | 16:43 |
| pabelanger | derekh: you already bumped the centos-7 node to 25GB, the new flavor you created 2 weeks ago | 16:44 |
| derekh | pabelanger: ahh ok, forgot that | 16:44 |
| pabelanger | derekh: No, I think they we can choose one of the other. So, maybe we can do #1 then do the work on #2 after we've migrated to centos-7. Once #2 is finished we can revert the change on #1, if needed | 16:45 |
| derekh | pabelanger: ok, I'm down with that | 16:45 |
| pabelanger | Once centos-7 is migrated, I can offer services to also migrate away from devstack | 16:45 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Ensure RabbitMQ list of loopback_users is set to empty array https://review.openstack.org/324016 | 16:45 |
| pabelanger | derekh: okay cool. If you want to delete and recreate the flavor you did a few weeks ago with more HDD space, that works for me | 16:46 |
| derekh | pabelanger: so I can remove and re-add h2.ci or create h3.ci | 16:46 |
| derekh | pabelanger: will do now | 16:46 |
| derekh | the dissapearing flavor wont screw up nodepool will it? | 16:46 |
| pabelanger | no, I've disabled h2.ci for the moment | 16:47 |
| pabelanger | so, should be safe to delete it | 16:47 |
| derekh | pabelanger: ok | 16:47 |
| *** panda has quit IRC | 16:48 | |
| derekh | pabelanger: done | 16:48 |
| *** panda has joined #tripleo | 16:49 | |
| *** jpena is now known as jpena|off | 16:50 | |
| pabelanger | derekh: great, let me try testing it | 16:51 |
| *** diazjf has quit IRC | 16:52 | |
| derekh | pabelanger: ack, gotta drop off in a minute but will check later if its passed | 16:52 |
| *** trown is now known as trown|lunch | 16:52 | |
| *** derekh has quit IRC | 16:53 | |
| sshnaidm | trown|lunch, it didn't run on liberty (why?), but passed all tests on mitaka: https://review.openstack.org/#/c/318751/ | 16:54 |
| sshnaidm | trown|lunch, (very surprising for me :D ) | 16:54 |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-common: Add RegisterNodesAction action https://review.openstack.org/319587 | 16:54 |
| *** fzdarsky is now known as fzdarsky|afk | 16:54 | |
| *** diazjf has joined #tripleo | 16:55 | |
| trown|lunch | sshnaidm: liberty failed because of no valid host... and shouldn't have posted SUCCESS... there is some CI bug there. https://ci.centos.org/artifacts/rdo/jenkins-tripleo-quickstart-gate-liberty-delorean-full-minimal-60/undercloud/home/stack/overcloud_deploy.log.gz | 16:55 |
| trown|lunch | really afk now though :P | 16:55 |
| *** diazjf has quit IRC | 16:58 | |
| gfidente | things are improving on radioparadise fwiw | 17:01 |
| *** links has joined #tripleo | 17:01 | |
| *** mbound has quit IRC | 17:03 | |
| *** yamahata has joined #tripleo | 17:03 | |
| *** mgould is now known as mgould|afk | 17:04 | |
| pabelanger | EmilienM: thanks for the review, just looked at it now | 17:05 |
| *** lucasagomes is now known as lucas-dinner | 17:05 | |
| *** ooolpbot has joined #tripleo | 17:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 17:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1587961 | 17:10 |
| *** ooolpbot has quit IRC | 17:10 | |
| openstack | Launchpad bug 1587961 in tripleo "tripleo-ci: all jobs failing with AMQPLAIN login refused: user 'guest' can only connect via localhost" [Critical,In progress] - Assigned to Lukas Bezdicka (social-b) | 17:10 |
| *** apetrich has quit IRC | 17:10 | |
| *** Goneri has quit IRC | 17:11 | |
| *** apetrich has joined #tripleo | 17:12 | |
| *** jcoufal_ has joined #tripleo | 17:15 | |
| *** xinwu has joined #tripleo | 17:17 | |
| *** jcoufal has quit IRC | 17:18 | |
| *** xinwu has quit IRC | 17:19 | |
| *** akrivoka has quit IRC | 17:20 | |
| *** akrivoka has joined #tripleo | 17:22 | |
| *** saneax is now known as saneax_AFK | 17:22 | |
| bandini | gfidente: I had proposed this one some days ago https://review.openstack.org/#/c/321650/ | 17:25 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/puppet-pacemaker: WIP: integrate PCS provider in the merge. https://review.openstack.org/310713 | 17:26 |
| gfidente | bandini, oh nice | 17:26 |
| social | bandini: so let's get that in though it would be nice not to use guest user | 17:27 |
| bandini | social, gfidente: I think to add a separate user we need to think about the upgrade a bit more, so maybe let's merge this one and track the separate user as a separate bug? | 17:27 |
| bandini | that way we unblock stuff right away and can plan the rest a bit more carefully | 17:28 |
| gfidente | bandini, but is that pasting loopback_users in the right place? | 17:28 |
| bandini | gfidente: why not? I tried it and it edited the rabbitmq.config correctly | 17:28 |
| bandini | gfidente: I tested it on rhosp9 were we saw the issue initially | 17:29 |
| gfidente | shouldn't it be {rabbit, [{loopback_users, [] | 17:29 |
| gfidente | ? | 17:29 |
| *** akrivoka has quit IRC | 17:30 | |
| *** dciabrin has quit IRC | 17:30 | |
| bandini | gfidente: let me check if I managed to push something wrong | 17:30 |
| gfidente | no I think it's working | 17:30 |
| social | gfidente: it's on correct place | 17:30 |
| gfidente | but it's not pasting it in the right place | 17:30 |
| bandini | damn the old env is gone | 17:31 |
| gfidente | oh but tcp_listen_options also is there | 17:32 |
| gfidente | social, we can check the CI job config | 17:33 |
| *** akrivoka has joined #tripleo | 17:34 | |
| social | gfidente: it works correctly ,) {loopback_users, []}, | 17:36 |
| *** links has quit IRC | 17:36 | |
| gfidente | social, yeah that's fine | 17:36 |
| gfidente | but is it in {rabbitmq, ? | 17:36 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart: prepend OPT_ to requirements and playbook variables https://review.openstack.org/324035 | 17:36 |
| social | gfidente: yes | 17:37 |
| gfidente | ok well I don't see it at all in the ci rabbitmq.config | 17:37 |
| social | gfidente: http://logs.openstack.org/50/321650/1/check-tripleo/gate-tripleo-ci-f22-ha/cb6613c/logs/overcloud-controller-0.tar.xz in tar there is etc | 17:37 |
| gfidente | ah cool | 17:37 |
| *** Goneri has joined #tripleo | 17:39 | |
| bandini | gfidente: yes it is correct I doublechecked the puppet module | 17:39 |
| bandini | you're right though it is not obvious at first glance | 17:39 |
| *** electrofelix has quit IRC | 17:39 | |
| gfidente | yeah how come it's appending in {rabbit, ? | 17:40 |
| openstackgerrit | Merged openstack/tripleo-quickstart: Remove hardcoded playbook file and add option to set it https://review.openstack.org/323951 | 17:40 |
| gfidente | it doesn't look like it is | 17:40 |
| bandini | gfidente: https://paste.fedoraproject.org/373447/80285214/ this is the rabbitmq.config from the ha job from my review | 17:41 |
| bandini | gfidente: the reason it is not completely obvious is that the indenting in templates/rabbitmq.config.erb in the puppet modules is a bit confusing | 17:41 |
| bandini | gfidente: but you should see that the loop over @config_variables is inside the {rabbit bracket | 17:42 |
| gfidente | right | 17:42 |
| gfidente | the indentation is wrong | 17:43 |
| gfidente | but the place is right | 17:43 |
| gfidente | tks ;) | 17:43 |
| bandini | yeah, it is totally non obvious, you're right | 17:43 |
| *** xinwu has joined #tripleo | 17:45 | |
| gfidente | and also tcp_listen_options should be in {rabbit, as well so we were doing it that way already :P | 17:45 |
| bandini | aye ;) | 17:46 |
| *** liverpooler has quit IRC | 17:46 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Set rabbitmq loopback_users explicitely https://review.openstack.org/321650 | 17:47 |
| *** ccamacho|afk has quit IRC | 17:50 | |
| *** numans has joined #tripleo | 17:51 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates: WIP: integration of the new puppet pacemaker. https://review.openstack.org/302409 | 17:52 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/tripleo-heat-templates: WIP: integration of the new puppet pacemaker. https://review.openstack.org/302409 | 17:53 |
| *** pkovar has quit IRC | 17:56 | |
| *** apetrich has quit IRC | 18:01 | |
| *** eggmaster has quit IRC | 18:01 | |
| *** apetrich has joined #tripleo | 18:04 | |
| *** sambetts is now known as sambetts|afk | 18:04 | |
| openstackgerrit | Merged openstack/tripleo-quickstart: prepend OPT_ to requirements and playbook variables https://review.openstack.org/324035 | 18:04 |
| *** trown|lunch is now known as trown | 18:05 | |
| EmilienM | gfidente: you have merge conflict on the pr | 18:08 |
| gfidente | EmilienM, which one? | 18:09 |
| gfidente | ah rabbitmq | 18:09 |
| EmilienM | https://github.com/puppetlabs/puppetlabs-rabbitmq/pull/479 | 18:09 |
| gfidente | I thought I pulled from master | 18:09 |
| gfidente | but I am not sure if we want to do that | 18:09 |
| gfidente | see https://review.openstack.org/#/c/321650/ | 18:10 |
| gfidente | wdyt? | 18:10 |
| EmilienM | gfidente: your pr makes sense | 18:11 |
| gfidente | ok sec | 18:11 |
| *** olap has quit IRC | 18:12 | |
| *** diazjf has joined #tripleo | 18:13 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Stop rebuilding locales on centos-7 https://review.openstack.org/318311 | 18:15 |
| *** rbrady has quit IRC | 18:15 | |
| *** rbrady_ has joined #tripleo | 18:15 | |
| *** dciabrin has joined #tripleo | 18:18 | |
| openstackgerrit | Honza Pokorny proposed openstack/tripleo-common: Add node assignment workflow https://review.openstack.org/320459 | 18:19 |
| gfidente | EmilienM, https://github.com/puppetlabs/puppetlabs-rabbitmq/pull/480 | 18:20 |
| gfidente | I was unable to update the existing request | 18:20 |
| gfidente | sorry :( | 18:21 |
| gfidente | the UI created a new one | 18:21 |
| *** rdo has quit IRC | 18:26 | |
| EmilienM | gfidente: I'll review it | 18:28 |
| *** dujelly has joined #tripleo | 18:30 | |
| *** apetrich has quit IRC | 18:32 | |
| *** mbound has joined #tripleo | 18:35 | |
| dprince | honza: hi, are you making changes here: https://review.openstack.org/#/c/300200/ | 18:35 |
| dprince | honza: I see you've pushed a few new patchets | 18:36 |
| *** apetrich has joined #tripleo | 18:36 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Stop rebuilding locales on centos-7 https://review.openstack.org/318311 | 18:37 |
| *** rdo has joined #tripleo | 18:39 | |
| *** mbound has quit IRC | 18:40 | |
| *** numans has quit IRC | 18:45 | |
| *** milan has joined #tripleo | 18:46 | |
| *** ifarkas has quit IRC | 18:50 | |
| EmilienM | social: are you working on https://review.openstack.org/#/c/323995/ ? | 18:50 |
| EmilienM | slagle, trown: do we keep tripleo CI is this state until https://review.openstack.org/#/c/323995/ lands? | 18:51 |
| EmilienM | I would suggest a packaging revert, it's blocking all the things | 18:51 |
| trown | EmilienM: I think https://review.openstack.org/321650 fixed it | 18:52 |
| EmilienM | ok | 18:53 |
| trown | just waiting on dlrn to build it, but could always depends-on that to get CI to build it | 18:53 |
| slagle | we're just in a holding pattern until dlrn builds it | 18:53 |
| trown | there is also a potential fix to rabbitmq package in http://cbs.centos.org/repos/cloud7-openstack-common-pending/x86_64/os/Packages/rabbitmq-server-3.6.2-3.el7.noarch.rpm | 18:54 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Cherrypick nova for bug 1585641 https://review.openstack.org/323516 | 18:54 |
| openstack | bug 1585641 in tripleo "CI: jobs failing with Message: No valid host was found. There are not enough hosts available., Code: 500"" [High,Triaged] https://launchpad.net/bugs/1585641 - Assigned to James Slagle (james-slagle) | 18:54 |
| slagle | i'm trying the depends-on ^ | 18:55 |
| slagle | maybe we should do that for the periodic job check as well | 18:55 |
| trown | can the periodic job use depends-on? | 18:55 |
| EmilienM | why did we land https://review.openstack.org/#/c/321650/ and CI was not passing? | 18:56 |
| trown | EmilienM: I think gfidente did not notice my recheck | 18:56 |
| EmilienM | are we sure it's the actual fix? | 18:56 |
| trown | it passed CI previously, I just rechecked to test it fixes rabbitmq issue | 18:56 |
| trown | ya nonha passed, and it was failing with rabbitmq issue | 18:56 |
| slagle | oh, apparently it doesnt | 18:56 |
| EmilienM | so we're still not sure | 18:56 |
| trown | oh... no it didnt | 18:57 |
| slagle | so it doesnt work | 18:57 |
| trown | oh it was the user change that passed nonha... https://review.openstack.org/323995 | 18:58 |
| EmilienM | yes | 18:58 |
| EmilienM | they are 2 different things | 18:58 |
| gfidente | trown, indeed I didn't | 18:59 |
| gfidente | but I see novacompute logging a different thing now, connrefused | 18:59 |
| *** paramite has joined #tripleo | 19:02 | |
| *** pkovar has joined #tripleo | 19:02 | |
| *** diazjf has quit IRC | 19:02 | |
| *** ebalduf_ has joined #tripleo | 19:03 | |
| *** diazjf has joined #tripleo | 19:03 | |
| *** chem has quit IRC | 19:06 | |
| gfidente | looks like rabbitmq didn't start at all | 19:07 |
| trown | hmm... I wonder if we shouldnt revert that change... | 19:09 |
| trown | is it Friday yet? | 19:09 |
| gfidente | which change? | 19:10 |
| *** dsariel has joined #tripleo | 19:10 | |
| gfidente | looks like it's the pcmk resource agent which fails to monitor the status | 19:10 |
| gfidente | and forces it off | 19:10 |
| trown | the one that broke rabbitmq worse | 19:10 |
| *** saneax_AFK is now known as saneax | 19:10 | |
| trown | https://review.openstack.org/#/c/321650/ | 19:10 |
| *** mbound has joined #tripleo | 19:12 | |
| gfidente | well | 19:12 |
| *** ccamacho has joined #tripleo | 19:12 | |
| gfidente | revert that to do what? | 19:14 |
| trown | hope the fix in RDO works? ... testing it via https://ci.centos.org/view/rdo/view/tripleo-quickstart/job/tripleo-quickstart-pending-mitaka-delorean-images-minimal/2/console | 19:15 |
| trown | it seems like going from rabbitmq up but unable to start nova-compute to rabbitmq not even starting is a move in the wrong direction | 19:16 |
| gfidente | so I think pcmk is logging this | 19:16 |
| gfidente | ERROR: Unexpected return code from '/usr/sbin/rabbitmqctl cluster status' exit code: 69 | 19:16 |
| gfidente | and stopping rabbitmq after that | 19:17 |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Stop rebuilding locales on centos-7 https://review.openstack.org/318311 | 19:20 |
| EmilienM | I suggest we revert the packaging change if it's blocking our CI | 19:21 |
| EmilienM | it also broke Puppet CI and we didn't land the patch yet in puppet to ignore the AVC | 19:21 |
| EmilienM | blocking Puppet & TripleO almost one day is really weird | 19:21 |
| trown | EmilienM: given we have about 40min to see if the packaging fix works, I would like to hold out for that | 19:22 |
| EmilienM | ok | 19:22 |
| EmilienM | just a reminder: we're suppose to produce a newton release this week | 19:22 |
| trown | since the revert is a bit dirty requiring we introduce epoch into rabbitmq package | 19:22 |
| trown | so if we can fix instead of revert it is much better | 19:22 |
| gfidente | yeah but I can't find the rabbitmq logs | 19:23 |
| trown | ya, the timing on this was pretty awful | 19:23 |
| gfidente | so I don't see why it didn't start | 19:23 |
| akshai | very new to heat templates. does anyone know what the netapp triple-o heat template is it designed to do/not do? | 19:23 |
| akshai | https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/extraconfig/pre_deploy/controller/cinder-netapp.yaml | 19:23 |
| social | gfidente: now I'm puzzled the loopback_users didn't work? | 19:28 |
| gfidente | so I see the option pasted in the config file as we expected | 19:29 |
| gfidente | but apparently that is causing rabbitmq not to start | 19:29 |
| EmilienM | gfidente: please don't leave on pto like this :P | 19:30 |
| gfidente | and I am looking into the logs | 19:30 |
| gfidente | but can't find why that is | 19:30 |
| gfidente | EmilienM, like breaking CI you mean? :) | 19:30 |
| gfidente | ahahaha | 19:30 |
| EmilienM | yeah | 19:30 |
| trown | lol | 19:30 |
| EmilienM | "ok I merged everything, CI broke, good luck folks" | 19:31 |
| *** rbrady_ has quit IRC | 19:31 | |
| trown | mic drop | 19:31 |
| *** rbrady has joined #tripleo | 19:31 | |
| gfidente | yeah if it wasn't that I didn't merge anything | 19:31 |
| gfidente | so only managed to break CI | 19:31 |
| gfidente | though, in all honesty | 19:31 |
| gfidente | I'm totally fine with reverting loopback_users and then "good luck folks" | 19:32 |
| social | gfidente: I dont think it's loopback_users breaking it | 19:32 |
| gfidente | social, right I am not sure | 19:32 |
| social | do you have something I can look at, at least logs/jenkins link? | 19:32 |
| *** apetrich has quit IRC | 19:32 | |
| gfidente | social, we need to get trown on board though | 19:33 |
| gfidente | I was looking at the logs from the CI run of the loopback_users patch | 19:33 |
| gfidente | https://review.openstack.org/#/c/321650/ | 19:33 |
| social | 2016-06-01 17:48:14.469 | warning: %post(rabbitmq-server-3.6.2-1.el7.noarch) scriptlet failed, exit status 1 | 19:35 |
| *** apetrich has joined #tripleo | 19:36 | |
| gfidente | social, ack, good catch | 19:38 |
| gfidente | I was looking into the config filr | 19:38 |
| gfidente | and didn't even notice the failure on install | 19:38 |
| social | gfidente: I'm reading through logs, give me some time :) | 19:41 |
| social | why there isn't /var/log/rabbitmq? I'm probably going to reproduce locally :( | 19:43 |
| trown | what board am I getting on? | 19:45 |
| *** noslzzp has quit IRC | 19:45 | |
| *** noslzzp has joined #tripleo | 19:46 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Stop rebuilding locales on centos-7 https://review.openstack.org/318311 | 19:48 |
| gfidente | social, though the %post seems to be registering the systemd service http://cbs.centos.org/kojifiles/packages/rabbitmq-server/3.6.2/1.el7/src/rabbitmq-server-3.6.2-1.el7.src.rpm | 19:48 |
| gfidente | if that failed maybe the service isn't installed in systemd at all so it never started | 19:49 |
| gfidente | and never logged | 19:49 |
| social | there still should be empty logdir | 19:49 |
| gfidente | logdir /var/log/rabbit is there | 19:49 |
| gfidente | though which version was installed before we landed loopback_users ? | 19:50 |
| gfidente | EmilienM, trown, you happen to have logs from a failed CI run from before loopback_users ? | 19:50 |
| EmilienM | not on my side | 19:50 |
| openstackgerrit | Martin Mágr proposed openstack/tripleo-puppet-elements: Install osops-tools-monitoring-oschecks package https://review.openstack.org/324075 | 19:51 |
| social | gfidente: the patch passed through ci once :) | 19:52 |
| trown | gfidente: http://logs.openstack.org/89/229789/48/check-tripleo/gate-tripleo-ci-f22-nonha/d240559/ | 19:52 |
| gfidente | trown, doh! | 19:52 |
| gfidente | same version of rabbitmq-server | 19:53 |
| gfidente | rabbitmq-server-3.6.2-1.el7.noarch | 19:53 |
| gfidente | but scriptlet not failing there | 19:53 |
| trown | ya that makes no sense... | 19:53 |
| gfidente | :( | 19:53 |
| social | recheck? <.> | 19:53 |
| gfidente | meh I have another change where scriptlet failed for all three jobs | 19:55 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Implement Nova base profile https://review.openstack.org/324080 | 19:56 |
| social | gfidente: on the loopback_users patch CI nonha job failed on compute deploy, not controller | 19:59 |
| gfidente | social, yeah because nonha doesn't have pcmk | 19:59 |
| gfidente | was rabbitmq up on controller at all? | 20:00 |
| gfidente | oh it started on controller yes | 20:01 |
| social | AMQPLAIN login refused: user 'guest' can only connect via localhost | 20:01 |
| *** paramite has quit IRC | 20:01 | |
| gfidente | yeah rabbitmq.config doesn't have the setting there | 20:02 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Implement Nova conductor profile https://review.openstack.org/324082 | 20:02 |
| gfidente | social, but this is yet another issue | 20:04 |
| gfidente | none of the static hieradata rabbitmq is applied | 20:04 |
| social | huh, why? | 20:05 |
| gfidente | because when nodes are =< 1 | 20:05 |
| social | what it uses than? | 20:05 |
| gfidente | the manifest is not consuming the rabbitmq_config_environment | 20:05 |
| *** david-lyle has joined #tripleo | 20:05 | |
| gfidente | social, https://github.com/openstack/puppet-tripleo/blob/master/manifests/profile/base/rabbitmq.pp#L85 | 20:05 |
| gfidente | it does a clean ::include in that case | 20:06 |
| gfidente | so we can fix that | 20:06 |
| gfidente | EmilienM, ^^ | 20:06 |
| gfidente | I think the purpose was to protect 'cluster_nodes' | 20:07 |
| *** akuznetsov has joined #tripleo | 20:07 | |
| gfidente | so we'll need to fix this too | 20:07 |
| gfidente | or nonha won't become green | 20:08 |
| EmilienM | gfidente: I copy/pasted the code from THT when I move to composable role | 20:08 |
| gfidente | EmilienM, yeah I know | 20:08 |
| EmilienM | but yeah, I see what you mean | 20:08 |
| trown | I am pretty sure if we revert https://review.openstack.org/#/c/321650/ and fix in RDO we are back to green | 20:09 |
| *** diazjf has quit IRC | 20:10 | |
| pabelanger | EmilienM: So, just looked at devstack-gate. Pretty sure we can update the JJB template for tripleo and set export OVERRIDE_ENABLED_SERVICES="". This will stop devstack from installing and configuring openstack services on the jenkins slave. Saving some HDD and cutting down on build times. I'm in the process of testing the change on 318311 first | 20:10 |
| EmilienM | wow nice | 20:10 |
| *** diazjf has joined #tripleo | 20:10 | |
| EmilienM | pabelanger: do you test on experimental pipeline? | 20:11 |
| openstackgerrit | Giulio Fidente proposed openstack/puppet-tripleo: Apply RabbitMQ rabbitmq_* static hiera in nonha with single controller https://review.openstack.org/324089 | 20:11 |
| pabelanger | EmilienM: I test centos-7 stuff on experimental. Right now 318311 is being tested on tripleo-check, I'm using the depends-on header to hack in the export command | 20:12 |
| EmilienM | ok | 20:12 |
| pabelanger | EmilienM: I am hoping tripleo-centos-7 stuff will be finish in the next day or so. | 20:13 |
| pabelanger | Once the CI is green, I'm rechecking my patch, then pushing the migration | 20:13 |
| EmilienM | ok, let me know when review is needed | 20:13 |
| social | gfidente: I do think the failure is because of %post script but I have no idea why it's not working | 20:18 |
| gfidente | yeah I just installed it | 20:18 |
| gfidente | and it didn't fail | 20:18 |
| *** lucas-dinner has quit IRC | 20:19 | |
| gfidente | https://paste.fedoraproject.org/373525/ | 20:20 |
| social | Failed to get D-Bus connection: Operation not permitted | 20:21 |
| social | so hmm | 20:21 |
| gfidente | though in ci that is running as | 20:21 |
| gfidente | sudo chroot /tmp/tmp.exZV1XoAx1 /bin/yum update -y | 20:21 |
| social | and systemd tries to contact dbus | 20:21 |
| gfidente | yeah | 20:22 |
| social | but on noha we had same post install error and it started rabbit | 20:23 |
| social | it just couldn't connect, so I guess issue is elsewhere :) | 20:23 |
| *** milan has quit IRC | 20:24 | |
| EmilienM | at this stage, a revert is likely the best option | 20:24 |
| *** jayg is now known as jayg|g0n3 | 20:25 | |
| *** lucasagomes has joined #tripleo | 20:25 | |
| *** milan has joined #tripleo | 20:27 | |
| *** milan has quit IRC | 20:27 | |
| *** akuznetsov has quit IRC | 20:28 | |
| openstackgerrit | Zane Bitter proposed openstack/instack-undercloud: Work around deletion of _member_ role assignments on upgrade https://review.openstack.org/307352 | 20:28 |
| *** lucasagomes has quit IRC | 20:29 | |
| *** dujelly has quit IRC | 20:33 | |
| *** jprovazn has quit IRC | 20:34 | |
| EmilienM | trown: I think a revert would be safe | 20:35 |
| *** fragatin_ has joined #tripleo | 20:35 | |
| *** fragatin_ has quit IRC | 20:35 | |
| *** noslzzp has quit IRC | 20:35 | |
| trown | EmilienM: packaging fix in RDO resolved for me... and same people who can push that through are the ones with revert power... and both EMEA | 20:35 |
| trown | (though on at all hours) | 20:36 |
| *** fragatina has joined #tripleo | 20:36 | |
| *** noslzzp has joined #tripleo | 20:36 | |
| *** lucasagomes has joined #tripleo | 20:36 | |
| gfidente | fwiw, I can start with loopback_users on 3.3.5 | 20:40 |
| gfidente | so it should be safe | 20:40 |
| gfidente | and https://review.openstack.org/#/c/324089/ is the other issue we had | 20:40 |
| trown | gfidente: hmm, maybe CI fail was just red herring? | 20:40 |
| gfidente | which should be landed as well or nonha won't pass | 20:41 |
| trown | ah ok, so that review will tell us if we are still broken for nonha too | 20:41 |
| trown | err non-nonha | 20:41 |
| trown | :) | 20:41 |
| gfidente | we *are* broken | 20:41 |
| gfidente | but nonha also wasn't consuming loopback_users at all | 20:42 |
| social | ;.; localy the change works | 20:42 |
| trown | so loopback_users is or is not the fix? | 20:42 |
| social | what am I doing wrong? | 20:42 |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Stop rebuilding locales on centos-7 https://review.openstack.org/318311 | 20:43 |
| pabelanger | EmilienM: ignore what I said about devstack, finally figured out what the jobs are doing. Going to completely by pass devstack-gate now and use zuul-cloner | 20:45 |
| EmilienM | pabelanger: ok | 20:46 |
| pabelanger | but first testing tripleo-centos-7 again | 20:46 |
| trown | hmm... I dont think depends-on works with merged change? http://logs.openstack.org/16/323516/8/check-tripleo/gate-tripleo-ci-f22-nonha/1f7642a/logs/rpm-qa.txt.gz did not build the tht patch | 20:46 |
| trown | slagle: ^ | 20:46 |
| *** panda has quit IRC | 20:48 | |
| *** panda has joined #tripleo | 20:49 | |
| trown | hmm, ya it only built nova... http://logs.openstack.org/16/323516/8/check-tripleo/gate-tripleo-ci-f22-nonha/1f7642a/console.html#_2016-06-01_19_14_50_443 | 20:49 |
| *** david-lyle has quit IRC | 20:51 | |
| *** ayoung has quit IRC | 20:55 | |
| bnemec | trown: Yeah, I'm pretty sure depends-on assumes anything merged is already available since it would be in the master git repos. | 20:55 |
| bnemec | Our use of packages probably breaks that assumption somewhat. | 20:56 |
| gfidente | social, yeah | 20:57 |
| *** akrivoka has quit IRC | 20:58 | |
| *** julim has quit IRC | 21:00 | |
| trown | ok packaging fix for rabbitmq is being pushed, so we should be good later this evening | 21:01 |
| trown | will check back a bit later | 21:01 |
| *** trown is now known as trown|outtypewww | 21:01 | |
| *** david-lyle has joined #tripleo | 21:01 | |
| *** apetrich has quit IRC | 21:05 | |
| *** diazjf has quit IRC | 21:05 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Enable nova-conductor as a composable service https://review.openstack.org/324102 | 21:05 |
| *** diazjf has joined #tripleo | 21:06 | |
| *** apetrich has joined #tripleo | 21:07 | |
| *** ccamacho has quit IRC | 21:12 | |
| *** Goneri has quit IRC | 21:13 | |
| *** lblanchard has quit IRC | 21:14 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: composable neutron server https://review.openstack.org/315129 | 21:16 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: composable neutron ml2 and ovs agent https://review.openstack.org/308514 | 21:16 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Implement Nova conductor profile https://review.openstack.org/324082 | 21:16 |
| openstackgerrit | Emilien Macchi proposed openstack/puppet-tripleo: Implement Nova base profile https://review.openstack.org/324080 | 21:16 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates: Enable nova-conductor as a composable service https://review.openstack.org/324102 | 21:16 |
| *** mbound has quit IRC | 21:22 | |
| *** fultonj has quit IRC | 21:26 | |
| *** fultonj has joined #tripleo | 21:26 | |
| slagle | i assume we'll see the new rabbitmq-server show up here: http://buildlogs.centos.org/centos/7/cloud/x86_64/openstack-mitaka/common/ once it's available | 21:27 |
| * slagle is watching | 21:27 | |
| *** MaxPC has quit IRC | 21:29 | |
| *** akshai has quit IRC | 21:30 | |
| *** apetrich has quit IRC | 21:33 | |
| *** apetrich has joined #tripleo | 21:35 | |
| EmilienM | gfidente: not now but it would be fun to have https://review.openstack.org/#/c/324089/ with full hiera driven | 21:43 |
| EmilienM | that would allow us to just include ::rabbitmq and boom | 21:43 |
| *** egafford has quit IRC | 21:43 | |
| gfidente | EmilienM, yeah | 21:43 |
| gfidente | that's a change which goes in tht though | 21:44 |
| EmilienM | yeah | 21:44 |
| gfidente | so we'd need to do that in another submission | 21:44 |
| EmilienM | gfidente: I commented | 21:44 |
| EmilienM | please add it in TODO, I'll do it tonight or tomorrow or one day | 21:44 |
| gfidente | I mean, I'd have helped if it wasn't that I want to close things for pto ;P | 21:45 |
| EmilienM | gfidente: it's ok | 21:46 |
| EmilienM | gfidente: this patch is not a blocker I mean | 21:46 |
| EmilienM | we can improve later | 21:46 |
| EmilienM | gfidente: do we need to land it to fix CI though? | 21:46 |
| gfidente | not if rabbitmq-server is reverted | 21:47 |
| gfidente | but yes if it's updated | 21:47 |
| EmilienM | asking on #rdo, I'm a bit confused about the situation | 21:47 |
| gfidente | EmilienM, so if it's not reverted | 21:48 |
| *** jcoufal_ has quit IRC | 21:48 | |
| gfidente | nonha won't pass unless we land tha | 21:48 |
| gfidente | and it will continue to fail with AUTHFAILURE | 21:48 |
| *** sshnaidm has quit IRC | 21:50 | |
| *** sshnaidm has joined #tripleo | 21:51 | |
| *** gumbyblake has joined #tripleo | 21:51 | |
| social | gfidente: I think I reproduced the issue from CI | 21:52 |
| *** dprince has quit IRC | 21:52 | |
| EmilienM | gfidente: not reverted | 21:53 |
| gfidente | social, how? | 21:53 |
| social | gfidente: rebuilt images from latest delorean and applied the patch, it failed on rabbitmq and I'm looking on what's wrong | 21:54 |
| social | but I got ERROR: Unexpected return code from '/usr/sbin/rabbitmqctl cluster status' exit code: 69 | 21:55 |
| social | so pretty sure it's the issue | 21:55 |
| *** david-lyle has quit IRC | 22:01 | |
| gfidente | right | 22:03 |
| gfidente | did it start at all? | 22:03 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Switch Cinder Api/Scheduler/Volume to composable roles https://review.openstack.org/310940 | 22:03 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Configure BlockStorage services via resource chains https://review.openstack.org/322858 | 22:03 |
| *** tosky has quit IRC | 22:06 | |
| *** diazjf has quit IRC | 22:09 | |
| *** ebalduf_ has quit IRC | 22:10 | |
| social | gfidente: no but manually I can start it from systemd | 22:12 |
| social | gfidente: but pacemaker does not | 22:12 |
| gfidente | yeah pcmk is using a resource agent for rabbit | 22:12 |
| gfidente | not systemd | 22:12 |
| *** dsariel has quit IRC | 22:14 | |
| gfidente | social, https://github.com/ClusterLabs/resource-agents/blob/master/heartbeat/rabbitmq-cluster | 22:15 |
| *** david-lyle has joined #tripleo | 22:16 | |
| *** mbound has joined #tripleo | 22:23 | |
| social | gfidente: I think errorcode changed with new version | 22:23 |
| social | gfidente: and they check for 0 1 anything else is super fail | 22:24 |
| gfidente | which isn't the case anymore? | 22:25 |
| gfidente | like 69 is not fail? | 22:25 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Ceilometer composable roles https://review.openstack.org/324117 | 22:25 |
| social | gfidente: https://github.com/rabbitmq/rabbitmq-server/commits/7984540175d0b8852025165b6b6a0ac05d692c98 :/ dunno | 22:28 |
| social | err | 22:28 |
| *** mbound has quit IRC | 22:28 | |
| social | gfidente: https://github.com/rabbitmq/rabbitmq-server/commit/7984540175d0b8852025165b6b6a0ac05d692c98 says it all | 22:30 |
| gfidente | yeah well the client returns 69 when it can't connect now | 22:30 |
| social | gfidente: so all in all we do need to temporary revert rabbitmq-server in rdo | 22:30 |
| social | gfidente: rabbitmq-server-3.6.2-3.el7.noarch.rpm will fix it and I'm off, enjoy your PTO | 22:41 |
| gfidente | oh man | 22:41 |
| gfidente | I was actually working oin the ceph roles | 22:41 |
| gfidente | so it'll take a bit | 22:41 |
| gfidente | social, though we need to land https://review.openstack.org/#/c/324089/ to see nonha passing if rabbitmq isn't going to be reverted | 22:43 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Cherrypick nova for bug 1585641 https://review.openstack.org/323516 | 22:45 |
| slagle | the new rabbitmq is live | 22:45 |
| openstack | bug 1585641 in tripleo "CI: jobs failing with Message: No valid host was found. There are not enough hosts available., Code: 500"" [High,Triaged] https://launchpad.net/bugs/1585641 - Assigned to James Slagle (james-slagle) | 22:45 |
| *** yamahata has quit IRC | 22:45 | |
| *** apetrich has quit IRC | 22:57 | |
| *** apetrich has joined #tripleo | 22:57 | |
| *** Goneri has joined #tripleo | 23:00 | |
| openstackgerrit | Pradeep Kilambi proposed openstack/puppet-tripleo: Add gnocchi profiles https://review.openstack.org/315527 | 23:04 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Gnocchi composable roles https://review.openstack.org/318413 | 23:04 |
| *** yamahata has joined #tripleo | 23:05 | |
| *** Goneri has quit IRC | 23:05 | |
| pabelanger | good, 30GB seems to be enough for centos-7 currently | 23:11 |
| pabelanger | once CI is passing, I assume the morning. I'll recheck and we can migrate | 23:11 |
| *** pradk has quit IRC | 23:20 | |
| gfidente | slagle, I gave a recheck https://review.openstack.org/#/c/324089/1 | 23:25 |
| *** noslzzp has quit IRC | 23:34 | |
| *** ibravo has joined #tripleo | 23:35 | |
| *** noslzzp has joined #tripleo | 23:35 | |
| *** afazekas has quit IRC | 23:52 | |
| *** pkovar has quit IRC | 23:56 | |
| *** afazekas has joined #tripleo | 23:56 | |
| *** apetrich has quit IRC | 23:56 | |
| *** apetrich has joined #tripleo | 23:58 | |
| *** david-lyle has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
