| *** jdennis has quit IRC | 00:00 | |
| *** jdennis has joined #tripleo | 00:00 | |
| *** tiswanso has joined #tripleo | 00:04 | |
| *** tiswanso has quit IRC | 00:09 | |
| openstackgerrit | Steve Baker proposed openstack/os-collect-config: Treat ec2 collector data as immutable https://review.openstack.org/384743 | 00:10 |
|---|---|---|
| openstackgerrit | Steve Baker proposed openstack/os-collect-config: Treat ec2 collector data as immutable https://review.openstack.org/384712 | 00:11 |
| *** bana_k has quit IRC | 00:16 | |
| *** dsneddon_ has joined #tripleo | 00:26 | |
| *** dsneddon_ has quit IRC | 00:31 | |
| *** pmannidi is now known as pmannidi|brb | 00:33 | |
| *** apetrich has quit IRC | 00:48 | |
| *** apetrich has joined #tripleo | 00:49 | |
| *** bana_k has joined #tripleo | 00:49 | |
| *** limao has joined #tripleo | 00:53 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-common: WIP: test the new hiera-heat-hook https://review.openstack.org/384751 | 00:56 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-common: WIP: test the new hiera-heat-hook https://review.openstack.org/384751 | 00:59 |
| openstackgerrit | gecong proposed openstack/instack-undercloud: Replace 'MagicMock' with 'Mock' https://review.openstack.org/384752 | 01:04 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Split out hosts config deployment https://review.openstack.org/384756 | 01:17 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Hiera optimization: use a new hiera hook https://review.openstack.org/384757 | 01:17 |
| *** dmacpher-afk has quit IRC | 01:22 | |
| *** bana_k has quit IRC | 01:26 | |
| *** dmacpher has joined #tripleo | 02:05 | |
| *** mbozhenko has joined #tripleo | 02:35 | |
| *** mbozhenko has quit IRC | 02:39 | |
| *** rlandy|afk is now known as rlandy | 02:49 | |
| *** rlandy has quit IRC | 02:49 | |
| -openstackstatus- NOTICE: Jobs running on osic nodes are failing due to network issues with the mirror. We are temporarily disabling the cloud. | 02:49 | |
| *** xuao has joined #tripleo | 02:53 | |
| *** yamahata has quit IRC | 02:55 | |
| *** pradk has quit IRC | 02:59 | |
| *** apetrich has quit IRC | 03:06 | |
| *** apetrich has joined #tripleo | 03:10 | |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates: GATE TEST, please ignore https://review.openstack.org/365449 | 03:30 |
| *** davidlenwell has quit IRC | 03:34 | |
| *** davidlenwell has joined #tripleo | 03:43 | |
| *** ChanServ sets mode: +v davidlenwell | 03:43 | |
| *** links has joined #tripleo | 04:01 | |
| *** tiswanso has joined #tripleo | 04:06 | |
| *** tiswanso has quit IRC | 04:11 | |
| *** bfournie has quit IRC | 04:15 | |
| openstackgerrit | Steve Baker proposed openstack/os-collect-config: WIP ec2 collector attempt config-drive before metadata server https://review.openstack.org/369731 | 04:32 |
| *** radeks has quit IRC | 04:34 | |
| *** mbozhenko has joined #tripleo | 04:35 | |
| *** davidlenwell has quit IRC | 04:35 | |
| *** mbozhenko has quit IRC | 04:40 | |
| *** bfournie has joined #tripleo | 05:02 | |
| *** ayoung has quit IRC | 05:18 | |
| *** ayoung has joined #tripleo | 05:20 | |
| *** ayoung has quit IRC | 05:22 | |
| *** kjw3 has joined #tripleo | 05:28 | |
| *** cwolferh has quit IRC | 05:36 | |
| *** adam_g` has quit IRC | 05:38 | |
| *** d0ugal has quit IRC | 05:40 | |
| openstackgerrit | afazekas proposed openstack/python-tripleoclient: Remove stack_owner_role from the deployer input https://review.openstack.org/384820 | 05:40 |
| *** d0ugal has joined #tripleo | 05:42 | |
| *** hjensas_ has quit IRC | 05:46 | |
| *** mbozhenko has joined #tripleo | 06:02 | |
| *** rcernin has joined #tripleo | 06:07 | |
| *** radeks has joined #tripleo | 06:12 | |
| *** adam_g has joined #tripleo | 06:18 | |
| *** adam_g has quit IRC | 06:18 | |
| *** adam_g has joined #tripleo | 06:18 | |
| *** jprovazn has joined #tripleo | 06:19 | |
| *** rasca has joined #tripleo | 06:30 | |
| *** mbozhenko has quit IRC | 06:35 | |
| *** pcaruana has joined #tripleo | 06:36 | |
| *** cwolferh has joined #tripleo | 06:40 | |
| *** dsariel has quit IRC | 06:49 | |
| *** davidlenwell has joined #tripleo | 06:51 | |
| *** ChanServ sets mode: +v davidlenwell | 06:51 | |
| *** sshnaidm|away has quit IRC | 06:54 | |
| *** mcornea has joined #tripleo | 06:54 | |
| *** davidlenwell has quit IRC | 06:56 | |
| *** hjensas has joined #tripleo | 06:58 | |
| *** ebalduf has quit IRC | 07:02 | |
| *** jaosorior has joined #tripleo | 07:03 | |
| *** dciabrin has quit IRC | 07:05 | |
| openstackgerrit | afazekas proposed openstack/python-tripleoclient: Remove heat_stack_user from the deployer input https://review.openstack.org/384820 | 07:05 |
| openstackgerrit | Christian Schwede proposed openstack/puppet-tripleo: Add versioned_writes to Swift proxy config https://review.openstack.org/384704 | 07:06 |
| *** cylopez has joined #tripleo | 07:06 | |
| openstackgerrit | Christian Schwede proposed openstack/tripleo-heat-templates: Enable object versioning in Swift proxy https://review.openstack.org/384616 | 07:07 |
| *** dciabrin has joined #tripleo | 07:08 | |
| *** pgadiya has joined #tripleo | 07:09 | |
| *** pgadiya has quit IRC | 07:09 | |
| *** ccamacho has joined #tripleo | 07:10 | |
| *** tremble has joined #tripleo | 07:12 | |
| *** tremble has joined #tripleo | 07:12 | |
| *** aufi has joined #tripleo | 07:15 | |
| *** davidlenwell has joined #tripleo | 07:16 | |
| *** ChanServ sets mode: +v davidlenwell | 07:16 | |
| *** zoli_gone-proxy is now known as zoliXXL | 07:17 | |
| *** dbecker has joined #tripleo | 07:18 | |
| *** hewbrocca-afk is now known as hewbrocca | 07:23 | |
| *** flaper87 has quit IRC | 07:24 | |
| ccamacho | bandini morning!! related to "[Bug 1622360] [NEW] Deployment of overcloud fails at step4" | 07:24 |
| openstack | bug 1622360 in tripleo "Deployment of overcloud fails at step4" [Undecided,New] https://launchpad.net/bugs/1622360 | 07:24 |
| ccamacho | Just thinking, what about doing the same upstream and creating instead of a "puppet-pacemaker.yaml" environment file create a file like "puppet-nonpacemaker", by default the deployments will use but if you dont want to use it.. just -e environments/puppet-nonpacemaker.yaml ?? | 07:24 |
| ccamacho | I mean to be consistent upstream and downstream | 07:24 |
| openstackgerrit | afazekas proposed openstack/python-tripleoclient: Remove heat_stack_user from the deployer input https://review.openstack.org/384820 | 07:26 |
| *** rawanh has joined #tripleo | 07:26 | |
| *** abregman has joined #tripleo | 07:26 | |
| bandini | ccamacho: absolutely agreed. Actually I would love to kill the non HA stuff completely. With Newton the use for nonHA is minimal (https://review.openstack.org/359060) | 07:27 |
| bandini | morning * | 07:27 |
| rawanh | Hi, is there a way to update the overcloud kernel parameters before deploying the overcloud from the undercloud node ? | 07:28 |
| *** abregman has quit IRC | 07:28 | |
| ccamacho | I asked like 1 month before and some folks on IRC told me that there are still users that needs non ha deployments | 07:29 |
| *** ebarrera has joined #tripleo | 07:29 | |
| ccamacho | so thats why both downstream and upstream behavior is not the same | 07:29 |
| bandini | is it me or CI is broken due to missing eth1 http://logs.openstack.org/67/384667/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/f70000c/logs/undercloud/var/log/undercloud_install.txt.gz#_2016-10-11_06_52_15_000 | 07:29 |
| bandini | ccamacho: I think with newton the need has become close to 0. What would be the use case? | 07:29 |
| *** zoliXXL is now known as zoli_gone-proxy | 07:29 | |
| ccamacho | mmmm not remember correctly but I think was related to deployments with network isolation | 07:30 |
| *** dmacpher has quit IRC | 07:31 | |
| bandini | ccamacho: netiso works correctly with HA, so not sure that is a valid point ;) | 07:31 |
| bandini | maybe there are, I can't think of any atm | 07:31 |
| jaosorior | bandini: why don't you try switching the default to be the pacemaker case (and the CI for that matter) | 07:33 |
| ccamacho | bandini agreed on that lets try to default to HA and if you dont need it, then use the non-ha env. file.. for the sake of simplicity | 07:33 |
| *** jlinkes has joined #tripleo | 07:35 | |
| bandini | jaosorior: I have a review for the default (need to update it). Am not sure where to poke to switch the CI to it | 07:36 |
| bandini | I'd need a few hints in the right direction to do that | 07:37 |
| bandini | jaosorior: btw. ever seen this one http://logs.openstack.org/67/384667/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/f70000c/logs/undercloud/var/log/undercloud_install.txt.gz#_2016-10-11_06_52_15_000 ? | 07:37 |
| *** zoli_gone-proxy is now known as zoliXXL | 07:38 | |
| jaosorior | bandini: haven't seen it, but currently my overcloud deployment is broken due to os-net-config too | 07:39 |
| jaosorior | so it might be that some recent commit there broke us | 07:39 |
| *** amoralej|off is now known as amoralej | 07:39 | |
| *** jpena|off is now known as jpena | 07:39 | |
| bandini | I see, will fetch some moarrrr coffee and then poke at it | 07:39 |
| *** davidlenwell has quit IRC | 07:44 | |
| *** b00tcat has joined #tripleo | 07:45 | |
| *** davidlenwell has joined #tripleo | 07:46 | |
| *** ChanServ sets mode: +v davidlenwell | 07:46 | |
| *** hogepodge has quit IRC | 07:51 | |
| *** egafford has joined #tripleo | 07:52 | |
| *** panda|Zz is now known as panda | 07:57 | |
| *** hogepodge has joined #tripleo | 07:58 | |
| *** dsneddon has quit IRC | 07:58 | |
| jaosorior | panda: hey dude, how's it going? | 07:59 |
| jaosorior | panda: noticed that the nonha job passed from your patch. at least the master version. | 07:59 |
| *** cwolferh has quit IRC | 07:59 | |
| jaosorior | panda: it seems that liberty and mitaka failed because of package issues :/ | 08:00 |
| ccamacho | bandini jaosorior, quick question, is there any guideline for creating python tests for our projects? I mean, there are cases in tripleo-common in which you have one test per method, and you have cases also there with a lot of tests without following any naming convention... | 08:00 |
| panda | jaosorior: yes, I have a patch for that too | 08:00 |
| jaosorior | panda: the next step, I think, is to figure out how to separate this SSL stuff from the nonha multinode. Since we can't run netiso on that. | 08:00 |
| jaosorior | panda: nice! did you get that working already? | 08:01 |
| jaosorior | ccamacho: no guidelines unfortunately | 08:01 |
| panda | jaosorior: I still don't understand how my certificate where wrong, has is something to do with country/CN informations | 08:01 |
| panda | jaosorior: liberty, yes, mitaka has another error | 08:01 |
| *** tzumainn has quit IRC | 08:01 | |
| ccamacho | I see :P | 08:02 |
| jaosorior | panda: so, I believe you created the certificate the right way. So that was alright. But for some reason there were two self-signed certificates. Did you make several attempts at creating the certs? It might be that you just copied the wrong one in one of the fields. | 08:02 |
| panda | jaosorior: first attempt, yes. THe second attempt I started from an empty dir | 08:03 |
| jaosorior | ccamacho: if we could get some guidelines (such as trying to follow behaviour driven tests or something like that) it would be cool, but it would require a bunch of refactoring for the tests. | 08:03 |
| jaosorior | panda: maybe you missed updating one of the files? not sure dude. If you have a local deployment you could attempt setting up SSL there. | 08:05 |
| panda | jaosorior: to separate multinode I need to refactor the entire case block, *OR* change the name of the multinode job | 08:06 |
| ccamacho | jaosorior, yeah that's what im seeing, Im starting to write some tests and I wanted to follow a pattern, but not able to find it.. would be cool to start following a pattern and then step by step do some refactoring there.. | 08:06 |
| jaosorior | panda: so which do you think is the best idea? refactoring or changing the name? | 08:08 |
| openstackgerrit | Martin André proposed openstack/tripleo-common: Create new docker command hook. https://review.openstack.org/312723 | 08:09 |
| openstackgerrit | Martin André proposed openstack/tripleo-common: Install configuration files for all downloaded packages https://review.openstack.org/347412 | 08:09 |
| openstackgerrit | Martin André proposed openstack/tripleo-common: Clean up configure_containers.sh script https://review.openstack.org/384865 | 08:09 |
| openstackgerrit | Martin André proposed openstack/tripleo-common: Allow building heat-agents image from master https://review.openstack.org/384866 | 08:09 |
| panda | jaosorior: changing the name is certainly the quickest, we'll just have to add nonha configuration to the multinode case and it's done | 08:09 |
| *** ohamada has joined #tripleo | 08:09 | |
| panda | jaosorior: that is until we have a multinode-HA job | 08:10 |
| jaosorior | panda: any idea when we would be able to do something like that? | 08:10 |
| jaosorior | I guess right now the limitation is zuul, right? | 08:10 |
| panda | jaosorior: yes | 08:10 |
| *** cwolferh has joined #tripleo | 08:11 | |
| *** dsneddon has joined #tripleo | 08:11 | |
| panda | jaosorior: the other thing is the objection from bnemec, he doesn't want to lose coverage on nonha without netiso | 08:13 |
| jaosorior | panda: hey, so you mentioned you had a patch to fix the package issues in the liberty and mitaka gates. How's that one going? | 08:13 |
| panda | jaosorior: https://review.openstack.org/384628 | 08:13 |
| jaosorior | panda: and we wouldn't if you change the name of the multinode-nonha. That is still without net-iso. And we could still do SSL there. | 08:14 |
| *** davidlenwell has quit IRC | 08:14 | |
| *** athomas has joined #tripleo | 08:15 | |
| *** davidlenwell has joined #tripleo | 08:18 | |
| *** ChanServ sets mode: +v davidlenwell | 08:18 | |
| openstackgerrit | afazekas proposed openstack/python-tripleoclient: Disable encrypted volume in the tempest-deployer-input.conf https://review.openstack.org/384868 | 08:24 |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/puppet-tripleo: Ensure presence of pacemaker restart directory. https://review.openstack.org/384870 | 08:27 |
| *** dsneddon has quit IRC | 08:28 | |
| *** dsneddon has joined #tripleo | 08:28 | |
| openstackgerrit | Athlan-Guyot sofer proposed openstack/puppet-tripleo: Ensure presence of pacemaker restart directory. https://review.openstack.org/384870 | 08:29 |
| *** abregman has joined #tripleo | 08:33 | |
| *** limao_ has joined #tripleo | 08:33 | |
| *** derekh has joined #tripleo | 08:34 | |
| *** Slower has joined #tripleo | 08:36 | |
| *** soc_off has joined #tripleo | 08:36 | |
| *** another_larsks has joined #tripleo | 08:39 | |
| *** amoralej_ has joined #tripleo | 08:40 | |
| *** jraju has joined #tripleo | 08:40 | |
| *** dsneddon has quit IRC | 08:41 | |
| *** davidlenwell has quit IRC | 08:41 | |
| *** jlinkes has quit IRC | 08:41 | |
| *** dbecker has quit IRC | 08:41 | |
| *** adam_g has quit IRC | 08:41 | |
| *** links has quit IRC | 08:41 | |
| *** limao has quit IRC | 08:41 | |
| *** rbrady-afk has quit IRC | 08:41 | |
| *** amoralej has quit IRC | 08:41 | |
| *** soc_off_ has quit IRC | 08:41 | |
| *** Slower_ has quit IRC | 08:41 | |
| *** rhefner has quit IRC | 08:41 | |
| *** mwhahaha has quit IRC | 08:41 | |
| *** hrybacki has quit IRC | 08:41 | |
| *** gregwork has quit IRC | 08:41 | |
| *** fungi has quit IRC | 08:41 | |
| *** mgagne has quit IRC | 08:41 | |
| *** honza_ has quit IRC | 08:41 | |
| *** larsks has quit IRC | 08:41 | |
| *** onovy has quit IRC | 08:41 | |
| *** amoralej_ is now known as amoralej | 08:41 | |
| *** dbecker has joined #tripleo | 08:44 | |
| *** dsneddon has joined #tripleo | 08:49 | |
| *** davidlenwell has joined #tripleo | 08:49 | |
| *** jlinkes has joined #tripleo | 08:49 | |
| *** adam_g has joined #tripleo | 08:49 | |
| *** rbrady-afk has joined #tripleo | 08:49 | |
| *** mwhahaha has joined #tripleo | 08:49 | |
| *** hrybacki has joined #tripleo | 08:49 | |
| *** gregwork has joined #tripleo | 08:49 | |
| *** fungi has joined #tripleo | 08:49 | |
| *** mgagne has joined #tripleo | 08:49 | |
| *** honza_ has joined #tripleo | 08:49 | |
| *** onovy has joined #tripleo | 08:49 | |
| *** tepper.freenode.net sets mode: +v davidlenwell | 08:49 | |
| *** gregwork has quit IRC | 08:50 | |
| *** gregwork has joined #tripleo | 08:52 | |
| *** gfidente has joined #tripleo | 08:52 | |
| *** mwhahaha has quit IRC | 08:52 | |
| *** mwhahaha has joined #tripleo | 08:53 | |
| *** r-mibu has quit IRC | 09:00 | |
| panda | CI is on fire. derekh: do you remember if our test always needed two nics to be present on the undercloud ? | 09:05 |
| *** akrivoka has joined #tripleo | 09:05 | |
| *** adam_g has quit IRC | 09:06 | |
| *** adam_g has joined #tripleo | 09:08 | |
| *** adam_g has quit IRC | 09:08 | |
| *** adam_g has joined #tripleo | 09:08 | |
| *** r-mibu has joined #tripleo | 09:15 | |
| openstackgerrit | yolanda.robla proposed openstack/tripleo-quickstart: Create directories with root https://review.openstack.org/384892 | 09:16 |
| *** hjensas has quit IRC | 09:20 | |
| derekh | panda: yes it does, here is the initial problem http://logs.openstack.org/65/384865/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/1e45fa9/console.html#_2016-10-11_08_15_44_648785 | 09:21 |
| derekh | panda: ovb testenv creation is failing, (that also adds the second nic) | 09:21 |
| derekh | panda: looking into it now | 09:24 |
| *** limao_ has quit IRC | 09:24 | |
| *** davidlenwell has quit IRC | 09:24 | |
| panda | derekh: just to understand, that is the error reported from gearman, because something on our ovb is failing to add the second nic ? | 09:27 |
| panda | derekh: maybe we should stop the build if testenv get an error in env creation ? | 09:27 |
| *** ubijtsa has joined #tripleo | 09:27 | |
| *** ubijtsa has quit IRC | 09:28 | |
| *** ubijtsa has joined #tripleo | 09:28 | |
| *** ubijtsa is now known as assassin | 09:28 | |
| derekh | panda: we did stop on fail at one stage, something must have changed | 09:28 |
| *** paramite has joined #tripleo | 09:28 | |
| *** zoliXXL is now known as zoli|wfh | 09:28 | |
| *** openstack has joined #tripleo | 09:29 | |
| *** onovy has quit IRC | 09:30 | |
| panda | derekh: we are stopping testenv when the error message is "Couldn't retrieve env". Adding this error too to the error condition | 09:33 |
| *** onovy has joined #tripleo | 09:38 | |
| derekh | panda: I've restarted rabbit, it was using a lot of RAM, lets see if that helps | 09:38 |
| *** jaosorior has quit IRC | 09:39 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci: Add "Failed creating OVB stack" to failure conditions in testenv-client https://review.openstack.org/384906 | 09:39 |
| *** jaosorior has joined #tripleo | 09:39 | |
| panda | derekh: noo, 1 minute and I could test if this last change was working ... :) | 09:41 |
| panda | derekh: ok, let's see what happens | 09:41 |
| *** jpich has joined #tripleo | 09:41 | |
| derekh | panda: brb | 09:44 |
| panda | derekh: 2016-10-11 09:46:29.271225 | 2016-10-11 09:46:29,270 - testenv-client - INFO - Received job : Failed creating OVB stack | 09:46 |
| *** shardy has joined #tripleo | 09:47 | |
| panda | and my change didn't stop it .. | 09:47 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci: Add "Failed creating OVB stack" to failure conditions in testenv-client https://review.openstack.org/384906 | 09:50 |
| *** tosky has joined #tripleo | 09:50 | |
| *** zoli|wfh is now known as zoli|lunch | 09:55 | |
| panda | derekh: OVB continues to fail, but my change is stopping the jobs now. | 09:57 |
| panda | 2016-10-11 09:56:52.225675 | 2016-10-11 09:56:52,225 - testenv-client - INFO - Received job : Failed creating OVB stack | 09:57 |
| panda | 2016-10-11 09:56:52.225761 | 2016-10-11 09:56:52,225 - testenv-client - ERROR - Failed creating OVB stack | 09:57 |
| *** chem has quit IRC | 10:00 | |
| panda | so we get a quick feedback when we get that error again | 10:00 |
| panda | I noticed that testenv returns the error very quickly. | 10:01 |
| shardy | d0ugal: Hey, question - are there plans for an openstack overcloud delete command? | 10:03 |
| shardy | I thought I saw a patch at one point but may have imagined it | 10:03 |
| shardy | it'd be nice to have a clean way to remove the stack, mistral env and swift container | 10:03 |
| *** pblaho has joined #tripleo | 10:04 | |
| d0ugal | shardy: there is a plan delete, it wont allow you to do it if there is a stack - but maybe we could add a flag? | 10:04 |
| d0ugal | shardy: There are no plans that I am aware of, but I am not doing much CLI "design" work now :) | 10:04 |
| shardy | d0ugal: Yeah I mean openstack overcloud delete <overcloud name> | 10:04 |
| d0ugal | jpich: Has this came up with your lot? ^ | 10:04 |
| shardy | which would be equivalent to openstack stack delete overcloud && openstack overcloud plan delete overcloud | 10:05 |
| shardy | it's not a huge deal, but would be marginally more convenient in some cases | 10:05 |
| shardy | bash alias provides a fine workaround tho :) | 10:05 |
| d0ugal | shardy: right, I follow. I just wonder if it will be confusing having a overcloud delete and a plan delete. | 10:05 |
| jpich | d0ugal: Do you mean have folks requested this as well? Not that I'm aware of | 10:06 |
| d0ugal | jpich: requested and/or discussed :) | 10:06 |
| shardy | jpich: I've been involved with a hackfest this week, and it's proven a minor source of confusion | 10:06 |
| d0ugal | right | 10:06 |
| shardy | not a big deal, just something to think about I guess :) | 10:06 |
| *** milan has joined #tripleo | 10:06 | |
| d0ugal | Yeah, an O-1 bug might make sense for it | 10:06 |
| shardy | ack, I'll raise one and we can discuss further in the bug if needed | 10:07 |
| *** abregman is now known as abregman|afk | 10:07 | |
| d0ugal | How do I make sense of this failure? http://logs.openstack.org/94/383694/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha/9ecf26d/console.html#_2016-10-11_07_53_34_020255 | 10:07 |
| jpich | shardy: Interesting! Eventually/soon we want to decouple 1 plan == 1 stack though will it still be as relevant then? | 10:07 |
| d0ugal | jpich: good point! | 10:07 |
| d0ugal | oh, it looks like CI is all red now? | 10:08 |
| d0ugal | all ovb jobs anyway | 10:08 |
| shardy | jpich: I think there will still be times where you want a clean way to iterate on deleting all-the-things, but if you're saying we're going to deprecate overcloud deploy then you could be right | 10:08 |
| derekh | panda: here is the error I'm seeing while creating the ovb testenv | 10:08 |
| derekh | 2016-10-11 09:59:13.029 33652 INFO heat.engine.stack [-] Stack CREATE FAILED (baremetal_10372): Resource CREATE failed: Forbidden: resources.public_network: (rule:create_network and rule:create_network:shared) on {'router:external': False, 'availability_zone_hints': [], 'description': '', 'provider:physical_network': <object object at 0x7f487d4f0140>, u'admin_state_up': True, 'tenant_id': u'b79291658f384b7ebbc9019b6349e5c9', 'segments': <object object at | 10:08 |
| derekh | 0x7f487d4f0140>, 'dns_domain': '', 'provider:network_type': <object object at 0x7f487d4f0140>, 'is_default': False, 'qos_policy_id': None, 'port_security_enabled': True, u'shared': True, 'provider:segmentation_id': <object object at 0x7f487d4f0140>, u'name': u'public-10372'} by {'domain': None, 'project_name': u'openstack-nodepool', 'tenant_name': u'openstack-nodepool', 'project_domain': None, 'timestamp': '2016-10-11 09:59:12.856425', 'auth_token': 'c631 | 10:08 |
| derekh | 6e0559684f2a8f0f7ea48fb889d9', 'resource_uuid': None, 'is_admin': False, 'user': u'ba119eef29ce49f5b8697f4d63948e3c', 'tenant': u'b79291658f384b7ebbc9019b6349e5c9', 'read_only': False, 'project_id': u'b79291658f384b7ebbc9019b6349e5c9', 'user_id': u'ba119eef29ce49f5b8697f4d63948e3c', 'show_deleted': False, 'roles': [u'_member_'], 'user_identity': 'ba119eef29ce49f5b8697f4d63948e3c b79291658f384b7ebbc9019b6349e5c9 - - -', 'tenant_id': u'b79291658f384b7ebbc90 | 10:08 |
| shardy | AFAIK we've not quite agreed on that point yet tho, definitely worth some wider discussion tho | 10:08 |
| derekh | 19b6349e5c9', 'request_id': 'req-3fda0674-6a57-4aed-8068-5c03bcc6e32c', 'user_domain': None, 'user_name': u'openstack-nodepool'} disallowed by policy | 10:08 |
| derekh | panda: Resource CREATE failed: Forbidden: resources.public_network: | 10:08 |
| jpich | shardy: I wasn't, but good to know that's being discussed :-) | 10:09 |
| * derekh is wondering if anybody changed anything overnight | 10:09 | |
| jpich | shardy, d0ugal: What I've noticed with my last few failed deployments if that there's no way to delete a stack from the UI (that I know about? jtomasek?) so that'll probably come up before deleting a plan + stack | 10:09 |
| jpich | derekh: Noticed https://bugs.launchpad.net/tripleo/+bug/1632245 pass by this morning, could it be related or something else going on? | 10:10 |
| openstack | Launchpad bug 1632245 in tripleo "All CI jobs failing on missing eth1" [Undecided,New] | 10:10 |
| jtomasek | jpich: yeah, we need to implement stack undeploy, but I think we don't want to delete the plan | 10:11 |
| panda | jpich: that was my first analysis, but it's the OVB stack that is failing | 10:11 |
| panda | jpich: and derekh just pasted the error | 10:11 |
| jtomasek | jpich this is where CLI and GUI diverges a bit afaik. as CLI still uses plan as a side effect (afaik) | 10:11 |
| panda | jpich: on ovb side | 10:11 |
| jpich | jtomasek: Yeah, just wondering if we might want the option to also delete plan + stack(s) from the plan management side at some point, but it's not as high priority as "undeploying" indeed | 10:12 |
| jpich | panda: I'll stop randomly pasting bug numbers then, thank you for the additional information :) | 10:12 |
| jtomasek | jpich: ah, right | 10:13 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Actually start the systemd services in step3 of the major-upgrade step https://review.openstack.org/384912 | 10:13 |
| jtomasek | jpich: the GUI workflow is if deployment fails, undeploy, tweak environments selection/update parameters of the plan or update the plan by uploading modified/updated files, run validations and try deploy again | 10:13 |
| derekh | jpich: panda I've pasted the error into the bug, still not sure why its happening | 10:13 |
| d0ugal | jpich: I think deleting both at the same time makes less sense in the GUI | 10:13 |
| shardy | jpich, d0ugal: https://bugs.launchpad.net/tripleo/+bug/1632271 | 10:14 |
| openstack | Launchpad bug 1632271 in tripleo "No overcloud delete command" [Low,Triaged] | 10:14 |
| shardy | One thing I'm not that clear on yet is does deprecating the overcloud deploy stuff fix the issues we've had with resolving files not in the plan? | 10:14 |
| shardy | or do we still have that issue when doing plan create? | 10:14 |
| * shardy needs to do more testing with plan create && plan deploy | 10:14 | |
| *** jraju has quit IRC | 10:15 | |
| d0ugal | shardy: I don't think it will yet, as there are some missing features I think | 10:15 |
| jpich | jtomasek: Yeah, that makes sense to me. So only the undeploy is currently missing, right? (side-note: I got a successful deploy this morning, but it's still showing in red with a big cross as step 4 in the UI :-o) | 10:15 |
| marios | bandini: did the autopick at https://review.openstack.org/#/c/384912/1 | 10:15 |
| shardy | d0ugal: Ok, I was just wondering if we can introduce e.g the --extra-files-dir option we discussed to plan create and deprecate the deploy command | 10:15 |
| jaosorior | shardy: hey dude, I'm getting some really strange issue... where overcloud deploy starts getting stalled in the network deployment if I modify the overcloud-full.qcow2 image. Basically only thing I do to it is an an additional package. | 10:15 |
| jtomasek | jpich: hmm, nice!, I am planning to test it today. It seems that the condition for stack state is incorrect then | 10:16 |
| shardy | jaosorior: what happens when you run the network configuration manually? | 10:16 |
| jaosorior | shardy: I can't even ssh to the node :/ | 10:16 |
| jtomasek | jpich: what is you'r stack state in openstack stack list? | 10:16 |
| jpich | jtomasek: Also if you have some cycles/ideas for https://bugs.launchpad.net/tripleo/+bug/1632007 , since RC3 isn't officially cut yet it'd be great if we could get this into it, AFAICT all the deployments fail until that's manually set :( (Maybe prio should be critical? I find these hard to set) | 10:17 |
| openstack | Launchpad bug 1632007 in tripleo "Failed deployment due to Hiera resolution of keystone::wsgi::apache::workers" [High,Triaged] | 10:17 |
| d0ugal | shardy: Yeah, that is the direction I was headed when I raised the idea before | 10:17 |
| d0ugal | shardy: but I am not totally against keeping the current deploy command if we can remove the mess from it :) | 10:17 |
| jpich | jtomasek: Status is "CREATE_COMPLETE" | 10:17 |
| bandini | marios: thanks ;) (forgot a cherry-pick -x?) | 10:17 |
| jpich | jtomasek: I'll try to track it down tomorrow or on Thu otherwise and we can backport it asap after that | 10:18 |
| jtomasek | jpich: thanks, I'll fix that problem with incorrect status | 10:18 |
| jpich | jtomasek: Though if you don't meet it in your own deploys maybe it's ok then :) Both dtrainor and myself saw it | 10:18 |
| jpich | jtomasek: Cheers! Want me to file a bug? | 10:19 |
| jtomasek | jpich: I was hitting the EC2 param one. Did not get further yet | 10:19 |
| jtomasek | jpich: would be great, thanks | 10:19 |
| jtomasek | jpich: The bug seems to me like a templates problem. I am not sure what could cause it from GUI pow | 10:19 |
| jpich | jtomasek: Hm, EC2 param doesn't ring a bell. I got the KeystoneCredential one, Snmpd one, and that keystone one. I got my CREATE_COMPLETE after that | 10:20 |
| *** dtantsur|afk is now known as dtantsur | 10:20 | |
| shardy | jaosorior: did you try using the primary console and a backdoor user/password like I suggested yesterday? | 10:20 |
| jpich | jtomasek: That's quite likely, it's still breaking us though :) | 10:20 |
| jpich | jtomasek: I'll have a look after these meetings tomorrow/Thu | 10:21 |
| jpich | It doesn't look like folks are encountering on the CLI so there may be something not sync'd again | 10:21 |
| marios | bandini: no the autopick didn't add it sec (not sure why sometimes it includes it and sometimes not... i think it may be cos they share the change id | 10:21 |
| jtomasek | jpich: yes, probably | 10:21 |
| jaosorior | shardy: I did try to set up a password for the root user. but haven't been able to access any sort of consolde :/ | 10:21 |
| marios | bandini: but i will add it anyway sec | 10:21 |
| jtomasek | jpich: btw. what environments did you use for your deployment? | 10:22 |
| jaosorior | shardy: not from virsh, not from virt-manager, not from nova | 10:22 |
| jtomasek | jpich: just the preselected one? | 10:22 |
| shardy | jaosorior: what do you mean - even if the networking is totally broken you can access the primary console? | 10:22 |
| shardy | unless the node isn't even booting? | 10:22 |
| jpich | jtomasek: Yeah, the only settings I set where the ones breaking the deployment really | 10:22 |
| jaosorior | shardy: that might be a thing | 10:22 |
| jtomasek | jpich: ok | 10:22 |
| jpich | jtomasek: I think I also did not profile matching, just clicked deploy directly (but I've tested so many things yesterday I could be confusing myself) | 10:23 |
| shardy | jaosorior: FWIW I just use virt-manager and access the console for the VM when this happens | 10:23 |
| shardy | that wil also show you if it's hung trying to boot | 10:23 |
| jaosorior | shardy: right, so that's what I'm trying to do and the console just looks black | 10:23 |
| shardy | jaosorior: So the image got broken then I guess | 10:23 |
| shardy | what package did you add, and how? | 10:24 |
| jaosorior | virt-customize -a overcloud-full.qcow2 --install ipa-client | 10:24 |
| jaosorior | shardy: and then I just did openstack overcloud image upload --update-existing | 10:25 |
| jtomasek | jpich: this is the parameter from the bug https://github.com/openstack/tripleo-heat-templates/blob/master/puppet/services/keystone.yaml#L83 | 10:26 |
| openstackgerrit | Marios Andreou proposed openstack/tripleo-heat-templates: Actually start the systemd services in step3 of the major-upgrade step https://review.openstack.org/384912 | 10:26 |
| jtomasek | jpich: it is type: string | 10:26 |
| jtomasek | jpich: so probably removing the "" should do | 10:26 |
| jpich | jtomasek: Filed https://bugs.launchpad.net/tripleo/+bug/1632278 fwiw | 10:28 |
| openstack | Launchpad bug 1632278 in tripleo "UI shows successful deployment like an error message" [Medium,Triaged] | 10:28 |
| jpich | jtomasek: Thanks for looking that up! Want to add it to the bug? | 10:28 |
| jtomasek | jpich: I'll investigate a bit more and update the bug | 10:28 |
| jpich | jtomasek: It's weird that no one appears to have seen the issue on the CLI | 10:29 |
| jtomasek | jpich: if I could see the mistral environment of the deployment which fails on this bug, it would be helpful | 10:30 |
| jtomasek | I'll try to reproduce it once I work around the EC2 parameter problem | 10:30 |
| jpich | jtomasek: I used the workaround to fix it, I can also try to undo this and reproduce the breakage later in the afternoon (demoing after lunch!) | 10:32 |
| jtomasek | jpich: ok, enjoy!:) | 10:32 |
| jpich | jtomasek: Thanks :-) Maybe add your ec2 error to the workarounds at https://etherpad.openstack.org/p/tripleo-ui-newton-rc once you've solved it, in case people hit it too this afternoon :-0 | 10:33 |
| jtomasek | jpich: ack | 10:34 |
| jpich | jtomasek: Cheers | 10:34 |
| derekh | bnemec: by any chance did you re enable the refresh cron job on the te-broker last night? | 10:36 |
| *** xuao has quit IRC | 10:42 | |
| derekh | panda: see comment 3, will know soon if it works https://bugs.launchpad.net/tripleo/+bug/1632245 | 10:46 |
| openstack | Launchpad bug 1632245 in tripleo "All CI jobs failing on OVB stack creation" [Critical,Triaged] | 10:46 |
| *** fultonj has joined #tripleo | 10:47 | |
| panda | derekh: whoa, it's like a murder cold case. Past and present intertwine. I think we could make a TV show out of CI failures. | 10:51 |
| jaosorior | ccamacho: hey dude, have you had some time to try out the stuff from the blog? | 10:53 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Enable showing multiple locations when using Ceph https://review.openstack.org/384926 | 10:53 |
| ccamacho | jaosorior, Yeahp I started yesterday evening, but messed the environment nad now finishing to re installing it, Im working from today till Thursday on your patches :) | 10:54 |
| ccamacho | and in the mean time some python tests | 10:55 |
| derekh | panda: things now appear to be working again | 10:55 |
| openstackgerrit | Giulio Fidente proposed openstack/tripleo-heat-templates: Enable showing multiple locations when using Ceph https://review.openstack.org/384926 | 10:55 |
| jaosorior | ccamacho: fuck yeah :D | 10:56 |
| ccamacho | I have only those 2 thing for this week to see if we can finish to land it :) | 10:56 |
| *** hjensas has joined #tripleo | 10:57 | |
| panda | derekh: \o/ | 10:57 |
| derekh | panda: cold case closed | 10:58 |
| rawanh | Hi, is there a UI for overcloud RHEL OSP9 after installing undercloud ? how can we access it ? | 11:00 |
| openstackgerrit | Martin André proposed openstack/tripleo-heat-templates: WIP: Containerized Services for Composable Roles https://review.openstack.org/330659 | 11:01 |
| jaosorior | ccamacho: that would be brilliant | 11:01 |
| *** thrash|g0ne is now known as thrash | 11:01 | |
| *** zoli|lunch is now known as zoli|wfh | 11:02 | |
| *** rhallisey has joined #tripleo | 11:06 | |
| *** links has joined #tripleo | 11:09 | |
| *** ooolpbot has joined #tripleo | 11:10 | |
| ooolpbot | URGENT TRIPLEO TASKS NEED ATTENTION | 11:10 |
| ooolpbot | https://bugs.launchpad.net/tripleo/+bug/1632245 | 11:10 |
| *** ooolpbot has quit IRC | 11:10 | |
| openstack | Launchpad bug 1632245 in tripleo "All CI jobs failing on OVB stack creation" [Critical,Triaged] | 11:10 |
| panda | sorry, bug just closed now. | 11:12 |
| *** egafford1 has joined #tripleo | 11:21 | |
| *** egafford1 has left #tripleo | 11:21 | |
| *** egafford has quit IRC | 11:22 | |
| *** yolanda has quit IRC | 11:25 | |
| *** egafford has joined #tripleo | 11:25 | |
| *** radeks has quit IRC | 11:30 | |
| *** abregman|afk is now known as abregman | 11:31 | |
| *** abregman is now known as abregman|afk | 11:33 | |
| *** pkovar has joined #tripleo | 11:33 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Actually start the systemd services in step3 of the major-upgrade step https://review.openstack.org/384667 | 11:38 |
| panda | shardy: do you remember if during this https://review.openstack.org/362904 you discussed about overcloud nodes logs, and if it was acceptable to lose the logs because thery were gathered after deletion ? | 11:40 |
| openstackgerrit | afazekas proposed openstack/python-tripleoclient: Disable encrypted volume in the tempest-deployer-input.conf https://review.openstack.org/384868 | 11:40 |
| *** yolanda has joined #tripleo | 11:41 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-puppet-elements: Add ipa-client package to overcloud-controller image https://review.openstack.org/384947 | 11:42 |
| openstackgerrit | Brad P. Crochet proposed openstack/python-tripleoclient: Downloads templates from swift before processing update https://review.openstack.org/381899 | 11:42 |
| *** jpena is now known as jpena|lunch | 11:43 | |
| *** adam_g has quit IRC | 11:43 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/tripleo-puppet-elements: Add ipa-client package to overcloud-controller image https://review.openstack.org/384947 | 11:44 |
| *** ohamada_ has joined #tripleo | 11:45 | |
| *** ohamada has quit IRC | 11:45 | |
| jaosorior | ccamacho: this would remove the virt-customize step to install ipa-client https://review.openstack.org/384947 | 11:46 |
| ccamacho | jaosorior, thanks :) | 11:47 |
| jaosorior | ccamacho: yeah, I'm trying to reduce the number of steps, that blog post is pretty damn big haha | 11:47 |
| *** dprince has joined #tripleo | 11:47 | |
| *** lucas-afk is now known as lucasagomes | 11:48 | |
| *** openstackgerrit has quit IRC | 11:48 | |
| *** dsneddon has quit IRC | 11:48 | |
| *** openstackgerrit has joined #tripleo | 11:48 | |
| *** dsneddon has joined #tripleo | 11:49 | |
| jaosorior | dprince, dsneddon: hey dude. So apparently the stuff I was seeing wasn't really an issue with os-net-config... it just got stuck there. But it seems like the image I was using was actually corrupted. And that part is pretty strange, since only modification I did to it was to install a package using virt-customize | 11:50 |
| jaosorior | only thing I did was virt-customize -a overcloud-full.qcow2 --install ipa-client | 11:51 |
| dprince | jaosorior: interesting | 11:52 |
| dprince | jaosorior: is it reproducable | 11:52 |
| jaosorior | dprince: pretty much every time I try to install a package like that I get that behavior | 11:52 |
| jaosorior | so I do the virt-customize, then I do openstack overcloud image upload --update-existing | 11:52 |
| *** egafford1 has joined #tripleo | 11:53 | |
| *** egafford has quit IRC | 11:54 | |
| openstackgerrit | afazekas proposed openstack/tripleo-heat-templates: Allow versioned containers https://review.openstack.org/384955 | 11:54 |
| *** egafford1 is now known as egafford | 11:55 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-common: WIP: test the new hiera-heat-hook https://review.openstack.org/384751 | 11:55 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Split out hosts config deployment https://review.openstack.org/384756 | 11:56 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Hiera optimization: use a new hiera hook https://review.openstack.org/384757 | 11:56 |
| hewbrocca | jaosorior: ping Pino Toscano (pinotree) | 11:56 |
| hewbrocca | see if he has any idea what virt-customize might be doing | 11:56 |
| jaosorior | hewbrocca: thanks dude | 11:57 |
| *** flaper87 has joined #tripleo | 11:57 | |
| dprince | ccamacho: hi, so I actually figured out a pattern that would accomidate the nova workers issue quite nicely I think: https://review.openstack.org/#/c/384757/2/puppet/services/nova-base.yaml | 11:58 |
| *** flaper87 is now known as Guest9738 | 11:58 | |
| dprince | ccamacho: I was working on the hiera hook... and needed this. But I could use it to work around the nova workers breakage too | 11:58 |
| dprince | ccamacho: although, I may still push on the nova fix for this too | 11:58 |
| dprince | ccamacho: it is the line at the bottom: starting at 121 | 11:59 |
| *** Guest9738 has quit IRC | 12:01 | |
| jaosorior | dprince: tripleo-common is present in the overcloud images, right? | 12:03 |
| ccamacho | dprince, awesome! I just added me to the reviewers list, Im testing some TLS submissions, then Ill give it a look locally in my dev env :) | 12:04 |
| EmilienM | hi | 12:06 |
| *** honza_ is now known as honza | 12:07 | |
| *** honza is now known as Guest60959 | 12:08 | |
| panda | EmilienM: do you remember if during this https://review.openstack.org/362904 you discussed about overcloud nodes logs, and if it was acceptable to lose the logs because thery were gathered after deletion ? I'm trying to understand if we want overcloud nodes logs anyway. | 12:09 |
| jaosorior | jistr: ping | 12:12 |
| EmilienM | panda: yes, we want overcloud logs anyway | 12:13 |
| EmilienM | panda: and no, I don't recall this discussion | 12:13 |
| panda | EmilienM: ok, thanks | 12:14 |
| *** trown|outtypewww is now known as trown | 12:15 | |
| *** pradk has joined #tripleo | 12:15 | |
| jaosorior | mwhahaha: ping | 12:18 |
| *** openstackgerrit has quit IRC | 12:18 | |
| ccamacho | dprince the only thing that worries me is that pushing the fix in nova might take a while | 12:18 |
| *** ccamacho is now known as ccamacho|lunch | 12:18 | |
| *** openstackgerrit has joined #tripleo | 12:18 | |
| *** maticue has joined #tripleo | 12:20 | |
| jistr | jaosorior: pong | 12:20 |
| jaosorior | jistr: hey dude, regarding the TLS patch. I'm trying to address that TODO for the post-save command. All I need is to put the script in some repo that both the overcloud and the undercloud can take it from. I was initially thinking of using tripleo-common, but apparently that's not in the overcloud images :/, got any ideas? | 12:23 |
| *** amoralej is now known as amoralej|lunch | 12:23 | |
| *** jayg|g0n3 is now known as jayg | 12:23 | |
| *** kjw3 has quit IRC | 12:25 | |
| *** jpich has quit IRC | 12:28 | |
| mwhahaha | jaosorior: whats up? | 12:32 |
| openstackgerrit | Merged openstack/tripleo-quickstart: Update libvirt role to identify flavor correctly https://review.openstack.org/382601 | 12:34 |
| EmilienM | all ovb jobs seem red, anyone know what happens? | 12:34 |
| *** dsariel has joined #tripleo | 12:34 | |
| hewbrocca | EmilienM: 2 nics | 12:35 |
| hewbrocca | 1 OVB | 12:35 |
| hewbrocca | or so I'm told | 12:35 |
| hewbrocca | from this morning at 9:29 | 12:36 |
| hewbrocca | <bandini> is it me or CI is broken due to missing eth1 | 12:36 |
| hewbrocca | 12:36 | |
| EmilienM | is anyone working on it? | 12:37 |
| *** jpena|lunch is now known as jpena | 12:37 | |
| EmilienM | derekh: ^ have you seen http://logs.openstack.org/06/384906/2/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/20e63c0/console.html#_2016-10-11_09_56_37_666186 ? | 12:38 |
| panda | EmilienM: the bug should be solved now | 12:39 |
| panda | EmilienM: https://bugs.launchpad.net/tripleo/+bug/1632245 | 12:39 |
| openstack | Launchpad bug 1632245 in tripleo "All CI jobs failing on OVB stack creation" [Critical,Fix released] | 12:39 |
| EmilienM | panda, derekh: thanks! | 12:40 |
| *** dmsimard|pto is now known as dmsimard | 12:40 | |
| EmilienM | I'm still catching up from pto :) | 12:40 |
| panda | EmilienM: one thing left to do is https://review.openstack.org/384906 | 12:41 |
| EmilienM | ccamacho|lunch: do you plan to write tests for https://review.openstack.org/#/c/384321/ ? or should we approve it now? | 12:42 |
| *** rbrady-afk is now known as rbrady | 12:42 | |
| panda | sigh .. s/catched/caught/ | 12:42 |
| EmilienM | panda: multinode now fails | 12:43 |
| EmilienM | http://logs.openstack.org/06/384906/2/check/gate-tripleo-ci-centos-7-nonha-multinode/b8858d9/console.html#_2016-10-11_10_17_37_977058 | 12:43 |
| EmilienM | EPEL? | 12:43 |
| *** jcoufal has joined #tripleo | 12:44 | |
| *** rbowen has joined #tripleo | 12:45 | |
| ccamacho|lunch | EmilienM yeahp Im trying to write some tests for all the validations from https://review.openstack.org/#/c/384321/ let me finish them and Ill let you know when they are working fine (when its the new deadline for rc3?) | 12:46 |
| EmilienM | ccamacho|lunch: ack | 12:46 |
| panda | mmm, epel should have been removed even there ... | 12:48 |
| EmilienM | thrash: do we need to backport https://review.openstack.org/#/c/383708/ ? | 12:49 |
| mrunge | is there a way to skip tripleo-ui installation in undercloud? | 12:49 |
| *** rlandy has joined #tripleo | 12:49 | |
| mrunge | tripleo-quickstart install fails currently due to failed dependencies for openstack-tripleo-ui | 12:50 |
| *** lblanchard has joined #tripleo | 12:50 | |
| EmilienM | panda: it seems transient | 12:50 |
| slagle | mrunge: did you check in undercloud.conf? there is an option: enable_ui | 12:50 |
| *** flaper87 has joined #tripleo | 12:51 | |
| *** flaper87 has joined #tripleo | 12:51 | |
| mrunge | slagle, I looked at it, but did not see the option | 12:51 |
| mrunge | slagle, thanks, will check again | 12:51 |
| slagle | maybe quickstart strips the default options? | 12:51 |
| slagle | anyway, check the sample in instack-undercloud | 12:51 |
| mrunge | will do | 12:52 |
| *** limao has joined #tripleo | 12:52 | |
| *** pkovar has quit IRC | 12:53 | |
| panda | maybe it's just the outdated undercloud.conf template in quickstart | 12:53 |
| jistr | jaosorior: hmm as for immediate solution, i don't know to be honest... Though in the future i think we'd like to deploy the undercloud with t-h-t as well. So on one hand, it feels like such script could go to t-h-t, but on the other hand it probably depends how complex/big it is. Can you provide an idea of what you want to do in the script? | 12:53 |
| *** pkovar has joined #tripleo | 12:55 | |
| trown | ya... that template is not maintainable... need to open a bug to remove that template in favor of just copying from /usr/share and using ini_file or the like to modify the conf if needed | 12:57 |
| jistr | jaosorior: for my own server i hacked together a tool which has a "<cert-name>.d" directory for each certificate, and one can drop in files that get executed when the cert changes (yea i'm afraid i basically rewrote a single-purpose ACME-protocol certmonger with hooks, using cron + bash...) | 12:57 |
| *** jpich has joined #tripleo | 12:58 | |
| *** limao_ has joined #tripleo | 12:59 | |
| *** dsavineau has joined #tripleo | 13:01 | |
| *** limao has quit IRC | 13:01 | |
| jistr | jaosorior: or maybe i'd start with a simpler question, given i'm not very familiar with the topic -- why can't the postsave_cmd be just "systemctl reload haproxy"? Don't we use that key/cert for haproxy exclusively? | 13:02 |
| *** chem has joined #tripleo | 13:03 | |
| *** flepied has quit IRC | 13:03 | |
| jaosorior | jistr: so the issue is that haproxy doesn't just take the cert and key as separate files | 13:03 |
| jaosorior | jistr: those files nee to be appended in a single PEM file | 13:04 |
| jistr | ahhh you need to pre-chew it into a single one | 13:04 |
| jistr | got it | 13:04 |
| jaosorior | why? cause haproxy is weird | 13:04 |
| *** fultonj_ has joined #tripleo | 13:04 | |
| *** Guest60959 is now known as honza | 13:04 | |
| *** chandankumar has joined #tripleo | 13:04 | |
| openstackgerrit | Merged openstack/tripleo-quickstart: Add centosci configs for cloudsig-stable pipeline https://review.openstack.org/384512 | 13:05 |
| *** limao_ has quit IRC | 13:06 | |
| *** tzumainn has joined #tripleo | 13:07 | |
| *** Goneri has joined #tripleo | 13:07 | |
| jaosorior | jistr: so you think it should go in t-h-t instead? | 13:09 |
| *** tiswanso has joined #tripleo | 13:10 | |
| jaosorior | mwhahaha: I was just gonna ask you about a bug ID in a commit. But I figured which bug it was | 13:11 |
| *** cdearborn has joined #tripleo | 13:11 | |
| jistr | jaosorior: so on my server i need to do this for ZNC bouncer, because it does the single-pem thing too | 13:13 |
| jistr | here's the code, it's a two liner https://paste.fedoraproject.org/448598/61915731/raw/ | 13:13 |
| *** jeckersb is now known as jeckersb_gone | 13:14 | |
| jistr | so if we're talking something similar, and we need to notify just haproxy and not a bunch of other services too, and all the file locations are well known, then i think we could probably inline it? wdyt? | 13:15 |
| jaosorior | jistr: so, we already have that script in instack-undercloud https://github.com/openstack/instack-undercloud/blob/master/scripts/instack-haproxy-cert-update | 13:16 |
| jaosorior | jistr: so the script is not an issue. The issue is that we need that available for the both the undercloud and overcloud, and I don't know what repo to put that on :/ | 13:16 |
| *** flepied has joined #tripleo | 13:17 | |
| matbu | where i can find the tripleo-ci periodic jobs ? | 13:17 |
| jaosorior | jistr: for other services such as apache stuff is more straight forward, since they use certs and keys just as they come from certmonger. So only thing I do for those is do a reload | 13:17 |
| panda | matbu: what are you looking for ? | 13:18 |
| panda | matbu: definitions ? configurations ? results ? | 13:18 |
| jistr | jaosorior: ok i see, you added a bunch of safety checks etc. Hmm personally i'd probably still go with putting the script into t-h-t at this point. Currently we do many things twice, because uc/oc aren't using the same installation method. If we use t-h-t for undercloud in the future too, things will become DRY (not only w/r/t certs, also service configuration etc.). | 13:19 |
| jaosorior | jistr: haha yeah, the script is a bit overkill | 13:20 |
| matbu | panda: i want to see the status of the mitaka periodic jobs | 13:20 |
| matbu | panda: if they are consistent | 13:21 |
| matbu | panda: cause i can't have a consistent mitaka deployment, which is really weird | 13:21 |
| jaosorior | jistr: anyway, for the reference, this is what I'm giving for httpd-based services https://review.openstack.org/#/c/327029/51/manifests/certmonger/httpd.pp | 13:21 |
| *** adam_g has joined #tripleo | 13:22 | |
| *** adam_g has quit IRC | 13:22 | |
| *** adam_g has joined #tripleo | 13:22 | |
| panda | matbu: http://status-tripleoci.rhcloud.com/#periodic-tripleo-ci-centos-7-ovb-ha-mitaka | 13:22 |
| *** morazi has joined #tripleo | 13:23 | |
| matbu | panda: k thanks, and who is monitoring that ? | 13:23 |
| matbu | for example the undercloud install fail (which is what i'm hitting) | 13:24 |
| matbu | this job is broken since september 30 | 13:24 |
| marios | matbu: can you update when you get a chance pls? https://bugs.launchpad.net/tripleo/+bug/1632330 | 13:24 |
| openstack | Launchpad bug 1632330 in tripleo "Undercloud upgrade fails on the yum update we run before undercloud is re installed" [High,Triaged] - Assigned to Marios Andreou (marios-b) | 13:24 |
| matbu | marios: yep sure | 13:24 |
| jistr | jaosorior: yea i think we could either use the script with checks and do get_file in t-h-t, or we could inline the `cat ... > ...` and `systemctl reload haproxy` directly into the template, similarly as you inlined the httpd reload into puppet | 13:25 |
| *** b3nt_pin is now known as beagles | 13:25 | |
| *** ccamacho|lunch is now known as ccamacho | 13:26 | |
| jaosorior | jistr: alright, sounds like a plan. I'll try the first option and see how that goes. Thanks dude | 13:26 |
| *** beagles is now known as Guest48927 | 13:26 | |
| openstackgerrit | Jiri Tomasek proposed openstack/tripleo-ui: Fix deployment status identification https://review.openstack.org/384995 | 13:27 |
| jtomasek | jpich: ^, would be great if you could test this on succeeding deployment | 13:27 |
| panda | matbu: me and sagi, but I was focusing on master and newton until last week. There is on change out to fix mitaka and liberty jobs now. | 13:28 |
| jpich | jtomasek: I was about to delete it! Will do now, thank you for looking into it | 13:28 |
| lucasagomes | hi all, can I get some eyes on https://review.openstack.org/#/c/370773/ ? It's a 1L change and is already +2'd. Thanks | 13:28 |
| jtomasek | jpich: :) cool, thanks | 13:28 |
| jpich | jtomasek: Sounds like something we'll want to backport too | 13:28 |
| jtomasek | jpich: btw. how old is your undercloud setup? | 13:29 |
| jtomasek | jpich: yep | 13:29 |
| matbu | panda: k, and i see in the log, it use the "consistent" tag to dpeloy mitaka | 13:29 |
| matbu | panda: do you how this tag is "promoted" ? | 13:29 |
| matbu | panda: maybe i can help, but i don't know how tbh right now | 13:30 |
| *** morazi has quit IRC | 13:30 | |
| dciabrin | bandini, jaosorior ping | 13:30 |
| jaosorior | dciabrin: hey | 13:30 |
| d0ugal | slagle: Hey | 13:30 |
| bandini | dciabrin: wasn't me | 13:31 |
| panda | matbu: you mean http://trunk.rdoproject.org/centos7/consistent ? | 13:31 |
| matbu | panda: yes | 13:31 |
| slagle | d0ugal: hi | 13:31 |
| d0ugal | slagle: I am not a fan of https://review.openstack.org/#/c/383694/ either :) | 13:31 |
| panda | matbu: I don't know the specifics, better ask in #rdo | 13:31 |
| d0ugal | slagle: but it was the best we got after some discussion in here | 13:31 |
| matbu | panda: this is the rdo pipeline which create this tag ? | 13:32 |
| jpich | jtomasek: That one is a bit of a mixed environment, looks like the oldest bits were installed on Sept 29th but I also updated some mistral / common / UI / etc packages right up to yesterday. Think this may be because my heat is too old? | 13:32 |
| d0ugal | slagle: but maybe we could just have instack create a mistral environment (different from heat's env) and add it there. I just don't think that is in any way secure. | 13:32 |
| d0ugal | slagle: however, since we already store other passwords in mistral, that point is probably mute | 13:33 |
| *** amoralej|lunch is now known as amoralej | 13:33 | |
| dciabrin | jaosorior, hey, looking at https://bugs.launchpad.net/tripleo/+bug/1628521. do i get that right that t-h-t in newton now passes the node_names to the galera ras as "before", i.e. overcloud-controller-0 rather than overcloud-controller-0.internalapi.localdomain? | 13:33 |
| openstack | Launchpad bug 1628521 in tripleo "nodename list disregards network" [High,Fix released] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 13:33 |
| jtomasek | jpich: not this one. I am wondering about the keystone workers one. I didn't hit that, I checked git blame and last time it was changed like this https://github.com/openstack/tripleo-heat-templates/commit/da419573fe3519c0d3119ab6f0a27217f0e39a61 | 13:34 |
| slagle | d0ugal: if it's not any less secure than what we already have, i think i'd prefer to see a mistral environment | 13:34 |
| jtomasek | jpich: which seems like what we're seeing now | 13:34 |
| jaosorior | dciabrin: yes, that's what we're doing currently. Though it would be nice to remove that workaround. | 13:34 |
| jtomasek | jpich: but the patch is relatively old | 13:34 |
| d0ugal | slagle: I don't think it will be, I'll do that. | 13:34 |
| d0ugal | slagle: it is also much easier, I think | 13:34 |
| jtomasek | jpich: I'll install latest setup tomorrow morning and try to reproduce it again | 13:34 |
| dciabrin | jaosorior, yeah that's what i understood. i'm asking because someone on RDO might have a similar issue, so i was wondering whether rdo needed to align to upstream or something | 13:35 |
| *** jeckersb_gone is now known as jeckersb | 13:35 | |
| jaosorior | dciabrin: they probably do. | 13:35 |
| rawanh | is there a way to trigger the hiera configurations to be loaded as a first boot script ? | 13:36 |
| pradk | can i get some reviews on https://review.openstack.org/#/c/384481/ .. pretty simple fix | 13:36 |
| matbu | panda: grazie btw | 13:36 |
| jpich | jtomasek: Oh, the keystone / trusted_cas issue was on a brand new setup last week, I think it was the same for dtrainor as well... I wonder. The demo is behind me so I'll try to remove my manual workaround and see if I can reproduce the error. You wanted to see a mistral env to help debug it, right? | 13:36 |
| jtomasek | jpich: can you check your undercloud templates how that parameter is defined? | 13:36 |
| jaosorior | dciabrin: well, actually, the workaround is done in puppet-tripleo | 13:37 |
| jaosorior | dciabrin: so t-h-t at the moment passes both the shortnames and the fqdns | 13:37 |
| jtomasek | jpich: ok, yep, checking mistral environment to see if that parameter is defined there could help, but I suspect it is not, in which case GUI has nothing to do with that bug | 13:38 |
| openstackgerrit | mathieu bultel proposed openstack-infra/tripleo-ci: Implement overcloud upgrade job - Mitaka -> Newton https://review.openstack.org/323750 | 13:38 |
| jpich | jtomasek: It's set to '"%{::processorcount}"' in my default templates | 13:40 |
| dtrainor | i have an osp10 environment that keeps hitting this error jpich jtomasek if either ofyou want to jump in to it | 13:40 |
| dtrainor | er, a newton environment that generates this error... | 13:41 |
| jtomasek | jpich: ook, that ruins my theory :D | 13:41 |
| mcornea | jaosorior: regarding the issue dciabrin mentioned I've got an environment where it shows up. Do I need any patch in puppet-tripleo to workaround it? | 13:41 |
| jpich | dtrainor: Oooh is it failed on this now? I think jtomasek has some ideas on data we should look at to debug further | 13:41 |
| *** eggmaster has joined #tripleo | 13:42 | |
| dtrainor | it is yes, i have not changed the state of that environment since i ran in to the error yesterday | 13:42 |
| *** morazi has joined #tripleo | 13:43 | |
| jtomasek | dtrainor: can you paste the 'mistral environment-get overcloud', please? | 13:43 |
| jistr | jaosorior: just thinking now -- when using get_file, you'd be inlining the content into postsave_cmd too. Can postsave_cmd be a bash script? (I don't mean if it can call a bash script, but rather if the thing itself gets evaluated as bash.) | 13:43 |
| jaosorior | mcornea: https://review.openstack.org/#/q/I4ef7786474c229d5212a0deb2ca02ee992b030d8 | 13:43 |
| jaosorior | jistr: yep | 13:43 |
| jaosorior | jistr: AFAIK | 13:43 |
| dtrainor | jtomasek, http://paste.openstack.org/show/585324/ | 13:43 |
| jtomasek | jistr: we're hitting https://bugs.launchpad.net/tripleo/+bug/1632007 on https://github.com/openstack/tripleo-heat-templates/commit/da419573fe3519c0d3119ab6f0a27217f0e39a61 | 13:44 |
| openstack | Launchpad bug 1632007 in tripleo "Failed deployment due to Hiera resolution of keystone::wsgi::apache::workers" [High,Triaged] | 13:44 |
| jtomasek | jistr: any ideas welcome:) | 13:44 |
| jistr | jaosorior: ok, cool. Also to make any sort of param substitution in it, we'd probably have to use str_replace or yaql or something of that nature | 13:44 |
| jtomasek | dtrainor: ok, I see it, it is there.... that is probably the problem... | 13:44 |
| jtomasek | dtrainor: did you explicitly set this parameter on this setup? | 13:45 |
| jtomasek | dtrainor: "KeystoneWorkers" | 13:45 |
| mcornea | jaosorior: thanks! trying that on my env | 13:45 |
| dtrainor | not in this setup no, but previously i had explicitly set KeystoneWorkers to 11 and the deployment worked | 13:45 |
| jaosorior | mcornea: I assume you have this available too https://review.openstack.org/#/q/I754c4a41d8a294a4c7c18bd282ae014efd4b9b16 | 13:46 |
| jtomasek | jistr: sorry, I think I see where the problem is coming from now... | 13:46 |
| jistr | jtomasek: hmm looks like we're losing quotes along the way | 13:46 |
| *** dprince has quit IRC | 13:47 | |
| mcornea | jaosorior: yes, I think the problem is that I have that but missing the puppet-tripleo change | 13:47 |
| jistr | jtomasek: here it's double quoted, so that the inner quotes get preserved when we translate from Heat into Hiera https://github.com/openstack/tripleo-heat-templates/blob/22decf4276c56b58b41ab6523c9a8da41ddeeab6/puppet/services/keystone.yaml#L86 | 13:47 |
| *** dprince has joined #tripleo | 13:47 | |
| jtomasek | dtrainor, jistr: yeah, GUI paraemters processing causes it | 13:47 |
| dtrainor | oh no kidding | 13:47 |
| dtrainor | ok then, that's good to know | 13:47 |
| jaosorior | mcornea: if you plan on deploying ceph you'll need this too https://review.openstack.org/#/c/382408/ | 13:47 |
| panda | EmilienM: I'll be able to join the meeting only on the second half. | 13:48 |
| *** panda is now known as panda|afk | 13:48 | |
| jtomasek | dtrainor: when GUI shows the parameter value, it displays it inside input and when this happens the string gets mangled, I'll take a look into it unless jpich wants to:) | 13:48 |
| EmilienM | panda|afk: ack | 13:49 |
| *** limao has joined #tripleo | 13:49 | |
| dtrainor | well that's excellent news, to know where it's getting butchered | 13:49 |
| dtrainor | thank you | 13:49 |
| EmilienM | weekly meeting in 10 min! | 13:50 |
| jpich | jtomasek: If the fix is in the GUI code I think I will stick to testing the fix once ready if that is ok? (If you're busy with other things and can give me a pointer as to where that's happening I can give it a shot though!!) Great work on narrowing the problem down, thank you :) | 13:51 |
| * social_ throws in https://review.openstack.org/#/c/381899/ for weekly meeting | 13:51 | |
| jtomasek | jpich: I'll try to fix it. This might be a tough one because the value is basically a string within a string.... | 13:52 |
| jistr | jtomasek: btw even the fact that we need to use nested quotes on Heat side to produce normal quotes on Hiera side is effectively a bug, as we're not preserving data types correctly, but it's a tough one to fix because for some uses we actually depend on it being broken... | 13:53 |
| jistr | it's been in tripleo since the dawn of time | 13:53 |
| jpich | jtomasek: Thank you! | 13:53 |
| *** rawanh has quit IRC | 13:53 | |
| jpich | jistr: Ouch | 13:53 |
| openstackgerrit | Marios Andreou proposed openstack/python-tripleoclient: Stop services before yum update for undercloud upgrade https://review.openstack.org/385012 | 13:54 |
| *** adam_g has quit IRC | 13:54 | |
| *** jcoufal_ has joined #tripleo | 13:54 | |
| jaosorior | rhallisey: hey, is it OK to ping you about SELinux stuff? | 13:56 |
| rhallisey | ya | 13:56 |
| rhallisey | jaosorior, what issue are you having? | 13:56 |
| *** jcoufal has quit IRC | 13:57 | |
| *** kjw3 has joined #tripleo | 13:59 | |
| *** Guest48927 is now known as beagles | 14:00 | |
| jaosorior | rhallisey: So, I'm having issues starting apache. This is what I got from the audit logs: http://paste.openstack.org/show/585328/ | 14:00 |
| *** yamahata has joined #tripleo | 14:00 | |
| jaosorior | rhallisey: seems to be having some issue with the gnocchi port | 14:00 |
| *** absubram has joined #tripleo | 14:01 | |
| rhallisey | i'll have a loo | 14:03 |
| rhallisey | look | 14:03 |
| *** absubram_ has joined #tripleo | 14:03 | |
| jaosorior | rhallisey: thanks dude | 14:04 |
| *** jprovazn has quit IRC | 14:04 | |
| *** mrunge is now known as mrunge__ | 14:05 | |
| *** absubram has quit IRC | 14:05 | |
| *** absubram_ is now known as absubram | 14:05 | |
| rhallisey | np | 14:06 |
| *** rodrigods has quit IRC | 14:06 | |
| *** rodrigods has joined #tripleo | 14:06 | |
| *** rcernin has quit IRC | 14:07 | |
| *** dsariel has quit IRC | 14:09 | |
| rhallisey | jaosorior, what os are you running? | 14:09 |
| *** limao has quit IRC | 14:10 | |
| jaosorior | rhallisey: CentOS Linux release 7.2.1511 (Core) | 14:10 |
| rhallisey | jaosorior, update to 7.3. This policy is fixed there in base rhel | 14:10 |
| jaosorior | rhallisey: good to know. unfortunately that's the base that quickstart is downloading by default. | 14:11 |
| jaosorior | trown: ^^ | 14:11 |
| rhallisey | oh :( | 14:11 |
| *** limao has joined #tripleo | 14:11 | |
| trown | is there a 7.3 for centos? | 14:12 |
| trown | http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2 is what we point to, so it should always be latest | 14:13 |
| jaosorior | rhallisey: ^^ | 14:13 |
| trown | but, if 7.3 is really recent, we have not had a promotion as the whole RDO CI infra was down for hardware move until this morning | 14:13 |
| rhallisey | it's in rhel 7.3 | 14:14 |
| rhallisey | so I'm assuming a centos 7.3 should be around soon if it isn't | 14:14 |
| trown | not sure how centos release cycle works... | 14:14 |
| *** egafford has quit IRC | 14:17 | |
| *** panda|afk is now known as panda | 14:17 | |
| *** egafford has joined #tripleo | 14:18 | |
| jaosorior | shardy: hey dude, have you encountered a heat error like this one? http://paste.openstack.org/show/585332/ | 14:19 |
| shardy | jaosorior: No but it looks like a heat bug and/or the DB is very broken | 14:21 |
| rook | dtantsur: heya | 14:23 |
| dtantsur | rook, hi, I have 7 minutes before the next meeting, so shoot :) | 14:23 |
| jaosorior | shardy: so I stumble upon that when trying to re-deploy an overcloud | 14:23 |
| rook | dtantsur lol, ill catch with you later then, no worries. | 14:23 |
| bnemec | derekh: Crap, I did. Sorry about that. | 14:24 |
| jaosorior | shardy: however, I'm using templates from my home directory, and environment files in another folder of my home dir | 14:24 |
| *** dtrainor_ has joined #tripleo | 14:24 | |
| *** dtrainor has quit IRC | 14:26 | |
| *** dtrainor_ is now known as dtrainor | 14:26 | |
| *** tosky has quit IRC | 14:28 | |
| *** mrunge has joined #tripleo | 14:28 | |
| *** limao has quit IRC | 14:28 | |
| derekh | bnemec: no prob, you switched it to the way it should have been, it shouldn't have been left like that for so long | 14:29 |
| *** limao has joined #tripleo | 14:29 | |
| shardy | jaosorior: ack, could be related but I still wouldn't expect heat to fail like that | 14:29 |
| *** tosky has joined #tripleo | 14:29 | |
| shardy | if you can reproduce please do raise a heat bug with details | 14:29 |
| bnemec | derekh: Yeah, I figured we just forgot to turn it back on the last time we were debugging. | 14:30 |
| *** anshul has joined #tripleo | 14:30 | |
| *** dtrainor_ has joined #tripleo | 14:30 | |
| derekh | bnemec: yup, unfortunatly it had a adverse side effect https://bugs.launchpad.net/tripleo/+bug/1632245 | 14:31 |
| openstack | Launchpad bug 1632245 in tripleo "All CI jobs failing on OVB stack creation" [Critical,Fix released] | 14:31 |
| derekh | bnemec: but we're all good again now, I havn't figured out how to change our deploy scripts to represent the change though | 14:31 |
| *** mhenkel has joined #tripleo | 14:32 | |
| *** dtrainor has quit IRC | 14:33 | |
| *** dtrainor_ is now known as dtrainor | 14:33 | |
| *** jeckersb is now known as jeckersb_gone | 14:35 | |
| *** jeckersb_gone is now known as jeckersb | 14:35 | |
| *** mah has joined #tripleo | 14:36 | |
| mah | Hi all, I have a problem when I use jclouds with tripleo to create an instance, anybody could help in that ? | 14:36 |
| *** dhill_ has joined #tripleo | 14:39 | |
| *** rajinir has joined #tripleo | 14:46 | |
| *** limao has quit IRC | 14:50 | |
| openstackgerrit | Dougal Matthews proposed openstack/instack-undercloud: Make the snmpd_readonly_user_password available to Mistral https://review.openstack.org/383694 | 14:50 |
| *** limao has joined #tripleo | 14:50 | |
| *** egafford1 has joined #tripleo | 14:51 | |
| *** egafford has quit IRC | 14:51 | |
| pradk | hmm looks like redis is having issues .. i cant connect to the vip or individual redis ips -> https://bugs.launchpad.net/tripleo/+bug/1632356 | 14:51 |
| openstack | Launchpad bug 1632356 in tripleo "unable to connect to redis" [Critical,New] | 14:51 |
| pradk | jistr, dint you already fix this? ^^ looks like its back again? | 14:53 |
| jistr | hmm yea i've fixed one issue related to redis/vip, essentially that we weren't creating the redis VIP at all | 14:54 |
| * jistr looking at the bug link | 14:54 | |
| pradk | i just noticed this in today's fresh deployment | 14:55 |
| jistr | pradk: is it pcmk or non-pcmk deployment? | 14:55 |
| pradk | ha deploy.. so yea pcmk | 14:55 |
| *** egafford1 is now known as egafford | 14:56 | |
| pradk | ah | 14:56 |
| pradk | see my latest comment in the bug | 14:56 |
| jistr | pradk: so that's likely a different issue than before. The previous issue was with keepalived non-pcmk | 14:56 |
| pradk | redis is having issues coming up | 14:56 |
| pradk | 7910:M 10 Oct 13:43:02.535 # Opening Unix socket: bind: Permission denied | 14:56 |
| pradk | see the same in ci jobs | 14:56 |
| pradk | http://logs.openstack.org/81/384481/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/94cf8c0/logs/overcloud-controller-0/var/log/redis/redis.txt.gz | 14:56 |
| weshay | matbu, marios getting questions about newton-> newton updates.. can you point me at jobs that are covering that | 14:57 |
| openstackgerrit | Merged openstack/python-tripleoclient: Add ReNo support https://review.openstack.org/382046 | 14:57 |
| pradk | jistr, that error is new.. | 14:57 |
| *** akuznetsov has joined #tripleo | 14:57 | |
| pradk | 7910:M 10 Oct 13:43:02.534 # Server can't set maximum open files to 10032 because of OS error: Operation not permitted. | 14:57 |
| jistr | pradk: hmm wondering why does it want such a huge amount of file descriptors open | 14:58 |
| jistr | pradk: i guess the main problem is the subsequent "Opening Unix socket: bind: Permission denied", which i guess could be prevented by the FD problem too | 14:58 |
| matbu | weshay: since the rdo tab is still with master | 14:58 |
| matbu | weshay: it should be this one : https://ci.centos.org/view/rdo/view/tripleo-periodic/job/tripleo-quickstart-upgrade-minor-master-to-master/ | 14:58 |
| pradk | jistr, yea seems like it | 14:59 |
| jistr | pradk: err s/prevented/caused/ | 14:59 |
| matbu | weshay: but since 1 week it does a master to master and not a newton to newton | 14:59 |
| rhallisey | EmilienM, https://wiki.openstack.org/wiki/Design_Summit/Ocata/Etherpads | 15:00 |
| matbu | weshay: i create a newton to master btw | 15:00 |
| pradk | jistr, this seems pretty recent as this was working in my previous deploy a week ago | 15:00 |
| EmilienM | rhallisey: thx | 15:00 |
| rhallisey | EmilienM, I'll update the schedule in there. I added the skeleton yesterday | 15:00 |
| jistr | pradk: btw i'm not sure actually about the bind being related to the FD limit | 15:01 |
| weshay | matbu, we need newton upgrade jobs created.. | 15:01 |
| EmilienM | rhallisey: excellent thanks | 15:01 |
| EmilienM | rhallisey: I'll update it when we'll decide what session at what slot | 15:01 |
| rhallisey | oh | 15:01 |
| EmilienM | rhallisey: please give me your constraints in PM | 15:01 |
| rhallisey | ya sure | 15:01 |
| rhallisey | EmilienM, gotcha i thought the times were in the etherpad | 15:02 |
| pradk | jistr, selinux? | 15:02 |
| jistr | pradk: it may not be that the limit has been used up, just trying to set it high, and maybe it fails in the sense of "i'm setting it to lower than you wanted but everything's fine in general, carry on" and then the bind error could be unrelated -- i'm just guessing but don't want us to focus on a red herring | 15:02 |
| pradk | jistr, wonder if selinux is not permitting bind | 15:02 |
| EmilienM | dprince, slagle, trown, flaper87 ^ you too please | 15:02 |
| pradk | lemme check audit log | 15:02 |
| trown | EmilienM: I have no constraints, I plan to be at all tripleo sessions | 15:02 |
| jistr | pradk: yea it definitely could be one cause, another one that comes up to mind would be a port collision | 15:02 |
| pradk | EmilienM, btw this redis issue is a newton blocker i think ttps://bugs.launchpad.net/tripleo/+bug/1632356 | 15:02 |
| pradk | EmilienM, i just ran into this in my env and see it happen in ci jobs as well .. http://logs.openstack.org/81/384481/1/check-tripleo/gate-tripleo-ci-centos-7-ovb-ha/94cf8c0/logs/overcloud-controller-0/var/log/redis/redis.txt.gz | 15:03 |
| EmilienM | pradk: please add the tag then | 15:03 |
| EmilienM | trown: ack | 15:03 |
| pradk | hmm nothing obvious in audit log | 15:04 |
| *** jtomasek is now known as jtomasek|bbl | 15:05 | |
| *** limao has quit IRC | 15:06 | |
| openstackgerrit | Dougal Matthews proposed openstack/instack-undercloud: Make the snmpd_readonly_user_password available to Mistral https://review.openstack.org/383694 | 15:06 |
| *** jlinkes has quit IRC | 15:12 | |
| *** tremble has quit IRC | 15:13 | |
| social_ | matbu: marios: so https://review.openstack.org/#/c/381899/ +1 from me | 15:13 |
| matbu | social_: ack | 15:13 |
| social_ | just finished successfull update | 15:13 |
| *** [1]cdearborn has joined #tripleo | 15:16 | |
| openstackgerrit | Dougal Matthews proposed openstack/instack-undercloud: Make the snmpd_readonly_user_password available to Mistral https://review.openstack.org/383694 | 15:17 |
| *** panda is now known as panda|afk | 15:20 | |
| *** ebarrera has quit IRC | 15:20 | |
| *** aufi has quit IRC | 15:20 | |
| openstackgerrit | Dougal Matthews proposed openstack/tripleo-common: Fetch the snmpd_readonly_user_password from the Mistral environment https://review.openstack.org/383696 | 15:21 |
| pradk | jistr, EmilienM, seems like selinux issue.. if i set to permissive, i can start redis | 15:21 |
| openstackgerrit | Dougal Matthews proposed openstack/python-tripleoclient: Use Password Generation provided by the Mistral workflows https://review.openstack.org/375544 | 15:21 |
| jaosorior | hey guys, can someone check this commits https://review.openstack.org/#/c/376255/ https://review.openstack.org/#/c/376267/ ?? | 15:22 |
| openstackgerrit | Merged openstack/python-tripleoclient: Allow referencing rendered yaml files in resource_registry https://review.openstack.org/383829 | 15:22 |
| jaosorior | pradk: I stumbled upon a similar issue with Gnocchi, apparently this is fixed in RHEL 7.3, so we need to wait till that gets to CentOS. Maybe you gotta check if this is also the case for redis | 15:22 |
| pradk | jaosorior, interesting.. do you know where/what the fix was? | 15:23 |
| jaosorior | pradk: not really | 15:24 |
| jaosorior | rhallisey: seems pradk is also stumbling with selinux issues. | 15:24 |
| rhallisey | pradk, can you send me your the AVCs you're seeing | 15:25 |
| *** jistr is now known as jistr|biab | 15:25 | |
| *** fultonj_ has quit IRC | 15:25 | |
| rhallisey | send me the AVCs you're seeing* | 15:25 |
| pradk | rhallisey, so that would be in audit.log? | 15:25 |
| rhallisey | pradk, yes | 15:25 |
| *** dmacpher has joined #tripleo | 15:26 | |
| rhallisey | pradk, grep for AVC if you don't mind. It would make it easier on me | 15:26 |
| rhallisey | :) | 15:26 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Remove the nova-metadata.yaml service https://review.openstack.org/385058 | 15:26 |
| pradk | rhallisey, k added to the bug https://bugs.launchpad.net/tripleo/+bug/1632356 | 15:27 |
| openstack | Launchpad bug 1632356 in tripleo "unable to connect to redis" [Critical,New] | 15:27 |
| dprince | jaosorior: ^^^ removing the nova-metadata.yaml okay with you? | 15:27 |
| jaosorior | dprince: I responded in the commit. | 15:28 |
| jaosorior | dprince: so, this was the original bug that was fixed by the addition of that role https://bugs.launchpad.net/tripleo/+bug/1625543 | 15:29 |
| openstack | Launchpad bug 1625543 in tripleo "Haproxy config for nova metadata uses ctlplane ip addresses instead of internal_api when using network isolation" [Undecided,Fix released] - Assigned to Juan Antonio Osorio Robles (juan-osorio-robles) | 15:29 |
| dprince | jaosorior: there are *no* dependencies on this file. How would it matter? | 15:29 |
| jaosorior | dprince: cause that messes up the yaql magic that generates the hieradata for nova_metadata, like I mentioned in the comment. | 15:30 |
| pradk | rhallisey, does that info help? | 15:30 |
| *** cdearborn has quit IRC | 15:31 | |
| *** pcaruana has quit IRC | 15:31 | |
| rhallisey | pradk, perfect thank you | 15:31 |
| jaosorior | dprince: if you remove that service profile, we will not have nova_metadata_network nor nova_metadata_node_names, which is generated in yaql based on the list of enabled services. | 15:32 |
| dprince | jaosorior: at the very least I think nova-api should require nova-metadata if we do actually require this? | 15:32 |
| *** lucasagomes is now known as lucas-hungry | 15:32 | |
| *** flepied1 has joined #tripleo | 15:32 | |
| dprince | jaosorior: it is the duplicate settings, and associated logic that I'm going to have to add I'm trying to avoid here | 15:32 |
| shardy | The problem is we've got no way to express dependencies between services (yet) | 15:32 |
| jaosorior | dprince: I understand what you're trying to fix. but stuff breaks if we don't have it. | 15:32 |
| shardy | I raised a heat spec which would allow it, but it's not been implemented | 15:33 |
| dprince | shardy: we sort of do though, I can include the nova-metadata resource in the nova-api. And then remove the hiera settings there. | 15:33 |
| dprince | shardy: both files can be included, and we only maintain the settings in a single place | 15:33 |
| jaosorior | dprince: trust me, I'm not too happy about having a service profile that's quite unnecessary. But we need those keys from heira | 15:33 |
| shardy | dprince: Yeah, but then as jaosorior says that breaks the hieradata generation, which uses the service_name | 15:33 |
| shardy | dprince: another option would be to allow service_name to be a list | 15:33 |
| jaosorior | dprince: so if you wanna get rid of it, I'm but, but we need a solution that doesn't break thigns. | 15:34 |
| shardy | or have some other means to remap the hieradata that's needed I guess | 15:34 |
| jaosorior | *I'm in, but | 15:34 |
| shardy | maybe we can just juggle the hieradata in puppet-tripleo? | 15:34 |
| dprince | jaosorior: at the very least I think I should remove the duplicate keys from nova-api.yaml right? | 15:35 |
| jaosorior | shardy: one things could be to stop using nova_metadata_* and switch everything to use nova_api_* | 15:35 |
| *** flepied has quit IRC | 15:35 | |
| dprince | jaosorior: right now you are probably winning because nova-metadata comes after nova-api in the service list | 15:35 |
| dprince | jaosorior: and map_merge takes the latter | 15:35 |
| dprince | I'm fine to go the other way now that you've pointed out the valid use case we have | 15:36 |
| jaosorior | dprince, shardy: So, if we're OK with that, we can switch the references to use nova_api_* | 15:36 |
| dprince | jaosorior: The simplest solution is to simply remove the keys from nova-api.yaml. I'd be entirely happy with that | 15:37 |
| jaosorior | dprince: which keys again? | 15:38 |
| panda|afk | any ideas what this could be ? http://logs.openstack.org/28/384628/4/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha-mitaka/688da1d/console.html#_2016-10-11_12_29_42_888733 | 15:39 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Remove duplicate metadata keys from nova-api.yaml https://review.openstack.org/385058 | 15:39 |
| dprince | jaosorior: those ^^ | 15:39 |
| jaosorior | dprince: I'm fine with that | 15:40 |
| dprince | shardy: FWIW, I think the other needed fix here is service_names does need to be a list | 15:40 |
| dprince | shardy: we'll certainly need that. We already "extend" one service via another. The use case for this is coming, and we could already make use of it here I think too | 15:41 |
| dprince | shardy: if you agree with that I'd be willing to push it | 15:42 |
| marios | thanks social_ | 15:42 |
| openstackgerrit | Pradeep Kilambi proposed openstack/tripleo-heat-templates: Include ceilometer in swift proxy pipeline https://review.openstack.org/371950 | 15:42 |
| dprince | shardy: it would require a unit test update in t-h-t to enforce though | 15:42 |
| *** egafford1 has joined #tripleo | 15:42 | |
| *** egafford has quit IRC | 15:43 | |
| jaosorior | dprince: so you want to aggregate the list of services to take into account the sub-services too? Say, you would also generate apache for services that run over it? | 15:44 |
| rhallisey | pradk, can you do `ls -lZ /var/run/redis` | 15:44 |
| rhallisey | pradk, I think you have some mislabled files | 15:45 |
| pradk | rhallisey, sure one sec | 15:45 |
| pradk | rhallisey, http://paste.openstack.org/show/585348/ | 15:47 |
| rhallisey | pradk, `sudo restorecon -Rv /var/run/redis` should fix the issue | 15:47 |
| pradk | rhallisey, so this happened out of the box install (even in ci) .. so is there something we need to change during deploy? | 15:48 |
| rhallisey | pradk, ya this restorecon will need to be run after /var/run/redis is created | 15:50 |
| rhallisey | pradk, so maybe in puppet | 15:50 |
| pradk | understood, i'll look into it.. it is weird though that we only ran into this now | 15:52 |
| shardy | dprince: ack, yeah provided it's optional I think that should be fine | 15:52 |
| *** egafford1 has quit IRC | 15:52 | |
| shardy | it'd be kinda cleaner if we could stick to one service per template, but it doesn't look like that's going to be possible | 15:52 |
| *** links has quit IRC | 15:54 | |
| thrash | rbrady: marios d0ugal this has the deprecation language now: https://review.openstack.org/#/c/381899/ | 15:54 |
| thrash | and is passing CI | 15:54 |
| jaosorior | any +A's for this https://review.openstack.org/#/c/360366/ ? | 15:55 |
| *** egafford has joined #tripleo | 15:55 | |
| EmilienM | pradk, chem, rhallisey: you should now be part of tripleo launchpad group | 15:56 |
| rhallisey | EmilienM, thx | 15:56 |
| EmilienM | if anyone is not part of the group please let me know | 15:56 |
| pradk | thx | 15:56 |
| thrash | jaosorior: +A | 15:59 |
| jaosorior | thrash: thanks dude | 16:00 |
| *** ebarrera has joined #tripleo | 16:02 | |
| *** b00tcat has quit IRC | 16:02 | |
| *** yamahata has quit IRC | 16:03 | |
| *** akuznetsov has quit IRC | 16:03 | |
| *** milan is now known as milan|afk | 16:07 | |
| *** milan|afk has quit IRC | 16:07 | |
| *** rasca has quit IRC | 16:07 | |
| *** jistr|biab is now known as jistr | 16:07 | |
| *** ebarrera has quit IRC | 16:09 | |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Only set NovaWorkers in the non-default case https://review.openstack.org/385078 | 16:12 |
| dprince | ccamacho: ^^ | 16:12 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates: Switch Nova workers to '' by default https://review.openstack.org/383478 | 16:12 |
| jaosorior | mwhahaha: you might want to sync with dprince ^^ | 16:13 |
| dsneddon | dprince, shardy: Do you guys have a suggestion for who might be able to implement this NTP on Undercloud spec in Newton? I think it should be pretty easy. https://review.openstack.org/#/c/350729 | 16:13 |
| *** kjw3 has quit IRC | 16:15 | |
| gfidente | dsneddon heh looks like you just raised another case where we'd benefit from having composable roles in the undercloud | 16:16 |
| gfidente | it's not going to happen in newton though | 16:16 |
| gfidente | just saying we have two use cases already I think | 16:16 |
| dsneddon | gfidente, Yeah, I can see the benefit. | 16:16 |
| dsneddon | gfidente, What's the other use case? | 16:16 |
| gfidente | dsneddon basically to have pluggability on the undercloud | 16:17 |
| gfidente | to have optional services | 16:17 |
| gfidente | monitoring for example | 16:17 |
| dsneddon | gfidente, Right. Or the UI. | 16:17 |
| gfidente | yeah | 16:17 |
| dsneddon | gfidente, Speaking of that, what do you think about this blueprint? I helped Dan Trainor put it together. https://blueprints.launchpad.net/tripleo/+spec/enable-communication-ui-undercloud | 16:18 |
| gfidente | we'd need to reprise the conversation about the undercloud template dprince worked on | 16:18 |
| dprince | mwhahaha: I hadn't realized you were working on this (bug was assigned to me FWIW) | 16:18 |
| dsneddon | gfidente, The idea is to have an API proxy on the Undercloud to allow the UI to connect to the various APIs on the same IP as the UI is hosted. | 16:18 |
| gfidente | dsneddon ok | 16:19 |
| dsneddon | gfidente, I couldn't think of a better or more secure way to do it, since the UI contacts the APIs directly, rather than there being some sort of AJAX gateway. | 16:19 |
| gfidente | what is the problem with going directly to the APIs? | 16:19 |
| dsneddon | gfidente, They live on the Control Plane, which is not neccesarily available remotely. | 16:20 |
| gfidente | ah | 16:20 |
| gfidente | while the UI could be elsewhere? | 16:20 |
| gfidente | I mean not hosted on the undercloud? | 16:20 |
| dsneddon | gfidente, The UI runs on the external uplink on the Undercloud | 16:20 |
| gfidente | so this sounds like yet another case where we could re-use the haproxy role | 16:20 |
| jaosorior | dsneddon: well, right now haproxy runs on the undercloud, and can bind to another interface (other than ctlplane) if specified. Wouldn't that be enough? | 16:20 |
| gfidente | on the undercloud? | 16:20 |
| dsneddon | gfidente, Yeah, the UI can be anywhere, there is a list of API endpoints provided to the UI client | 16:21 |
| dsneddon | jaosorior, Yeah, but we want to require SSL for remote connections. | 16:21 |
| jaosorior | dsneddon: sure, that wouldn't really be an issue | 16:21 |
| *** egafford has quit IRC | 16:21 | |
| dsneddon | jaosorior, I mean, that's basically exactly what I'm saying, we would add endpoints in HAProxy on the second IP. | 16:22 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-common: Default the J2 excludes files to safe values. https://review.openstack.org/384321 | 16:22 |
| *** zoli|wfh is now known as zoli|gone | 16:22 | |
| ccamacho | dprince ack will check it soon | 16:23 |
| * gfidente back in a while | 16:23 | |
| dsneddon | jaosorior, gfidente: I'll add you guys to the spec when I write it, if you have any feedback. | 16:23 |
| jaosorior | dsneddon: so, for the SSL case, we already have SSL for haproxy, and one can already bind it to any given interface (needs to be configured in undercloud.conf), so... basically seems that that issue is already fixed, by requiring SSL for the undercloud, and just binding haproxy elsewhere | 16:23 |
| jaosorior | * elsewhere than the ctlplane | 16:23 |
| jaosorior | dsneddon: which one can already do with the wrongly named "undercloud_public_vip", which isn't necessarily just a vip (I actually use fqdns there) | 16:24 |
| dsneddon | jaosorior, Yes, you just described what I planned to put into the spec. The only work around it is adding the necessary params to undercloud.conf and modifying the HAProxy config appropriately. | 16:24 |
| *** zoli|gone is now known as zoli_gone-proxy | 16:24 | |
| jaosorior | dsneddon: I think I'm misunderstanding something, cause that can already be done | 16:24 |
| jaosorior | like, now | 16:24 |
| EmilienM | jistr, marios: why did you guys abandon https://review.openstack.org/#/c/319264/ ? | 16:25 |
| dsneddon | jaosorior, Yes, they can do that, but then the Keystone catalog gets updated with the external IP, which forces an extra hop if the Undercloud isn't the default gateway for the Control Plane. | 16:25 |
| *** athomas has quit IRC | 16:25 | |
| jaosorior | dsneddon: I see what you mean now | 16:25 |
| jaosorior | dsneddon: though... I don't really see an issue with having an extra hop | 16:25 |
| *** egafford has joined #tripleo | 16:26 | |
| dsneddon | jaosorior, Although, considering that the Undercloud catalog is really only used by the Undercloud itself... | 16:26 |
| dsneddon | jaosorior, Maybe you are right. | 16:26 |
| jistr | EmilienM: that approach i took there was -2'd (not on the spec but on the code part) | 16:26 |
| jistr | EmilienM: and i got pulled into downstream work at the same time | 16:26 |
| EmilienM | jistr: is here any open spec about upgrades right now? | 16:26 |
| dsneddon | jaosorior, I'll do some testing, and I wouldn't mind at all if you just made that spec redundant. Thanks for the ideas. | 16:27 |
| jaosorior | dsneddon: didn't mean to bring any ideas down. Just thought it would be easier if we used what's there already. | 16:27 |
| *** jpich has quit IRC | 16:27 | |
| dsneddon | jaosorior, Ha, I love it when I find out that work is already done or not needed. It's not like I'm having trouble keeping busy. | 16:28 |
| jistr | marios, EmilienM: hmm not from my side. We had a couple of other PoCs for new upgrades but i don't think any of them matched requirements like tight integration with the Heat templates for composable services. (AFAIK) | 16:28 |
| jaosorior | dsneddon: haha well, hopefully that works for this case. Had a talk about just this with dtrainor a bit ago. | 16:28 |
| *** chandankumar has quit IRC | 16:28 | |
| marios | EmilienM: i think mainly as there was pushback to the initial approach taken with ansible... but as noted on the https://etherpad.openstack.org/p/ocata-tripleo-upgrades we can restore and edit it dependign on how the discussion goes | 16:29 |
| dtrainor | (testing this right now) | 16:29 |
| marios | jistr: ^ | 16:29 |
| *** dmacpher is now known as dmacpher-afk | 16:29 | |
| *** paramite has quit IRC | 16:29 | |
| dtrainor | darn meetings are getting in the way though, it'll be a few | 16:29 |
| EmilienM | marios: I think we should first split bash scripts into chunks and run them separately depending on the services | 16:30 |
| *** hewbrocca is now known as hewbrocca-afk | 16:30 | |
| EmilienM | marios, jistr: and next iteration for the fiture is to investigate ansible | 16:30 |
| EmilienM | future* | 16:30 |
| EmilienM | but I don't think we'll switch upgrades to ansible during ocata cycle | 16:30 |
| EmilienM | not enough time and no concensus has been reached so far | 16:30 |
| *** dtantsur is now known as dtantsur|afk | 16:31 | |
| EmilienM | marios, jistr: it would be great to have a spec for that. | 16:31 |
| marios | EmilienM: /me going in a minute, but i think the idea was to investigate integrating the ansible into the heat templates per service (upgrade snippets)... for ocata the main thing is 'composable service upgrades'... whtether we use ansible or any other approach is still open at the moment | 16:31 |
| marios | jistr: ^ | 16:31 |
| marios | EmilienM: but delivering 'composable service upgrades' implies a complete change in the current upgrades workflow which is done per role/node | 16:32 |
| marios | EmilienM: and given the short cycle this becomes even greater challenge | 16:32 |
| *** jaosorior has quit IRC | 16:33 | |
| EmilienM | that's why adding ansible is a bad idea for such a short cycle | 16:33 |
| EmilienM | I would rather just split the bash scripts | 16:33 |
| EmilienM | so have a script by service | 16:33 |
| EmilienM | marios: ^ | 16:33 |
| *** akuznetsov has joined #tripleo | 16:34 | |
| marios | EmilienM: sure i am pretty sure we considered that when we started doing that ansible stuff... we can definitely investigate it ... if we are going that way we can even consider puppet manifests so we can get dependencies for example | 16:34 |
| *** lucas-hungry is now known as lucasagomes | 16:34 | |
| EmilienM | marios: we need to iterate here, and shell scripts split is the easiest solution for now | 16:35 |
| marios | EmilienM: this makes sense ... and precisely the kind of thing we need to finalise at summit | 16:37 |
| marios | EmilienM: needs some more thought but adding note for now thanks | 16:37 |
| jistr | marios, EmilienM: re integrating Ansible -- it also depends how. E.g. we previously discussed 2 different ways, 1) as a normal SoftwareConfig/SoftwareDeployment, where Ansible only sees a single machine, or 2) somehow more classic execution, where ansible sees the whole deployment | 16:37 |
| jistr | i think all we prototyped previously was 2), but that brings some challenges on the integration itself, and that was the reason for the pushback | 16:38 |
| jistr | 1) would be probably easier to integrate, but it also doesn't bring that many benefits over plain bash scripts... | 16:38 |
| EmilienM | right | 16:38 |
| EmilienM | I would rather defer ansible to after Ocata and focus now on composability by splitting the bash scripts. | 16:39 |
| chem | bandini: I'm hitting the nova upgrade problem on compute node upgrade upstream, did you have a patch ready ? | 16:42 |
| marios | EmilienM: so no pushback to the notion of iterating with bash for the first pass especially given the short cycle... thanks for bringing it up ... /me ed | 16:43 |
| jistr | +1 | 16:44 |
| mwhahaha | dprince: yea sorry i had typoed the bug, so i had a review for tripleo specifically and not the nova thing. I think it's a larger issue than just how nova is handling that value and we might want to rethink those worker values. I'm ok abandoning my review but I thought I'd propose a possible tripleo specific solution | 16:45 |
| bandini | chem: yes, sec | 16:45 |
| bandini | chem: https://review.openstack.org/#/q/I811ff19d7b44a935b2ec5c5e66e5b5191b259eb3,n,z | 16:46 |
| dprince | mwhahaha: sure, I filed the bug under both Nova and Tripleo | 16:46 |
| dprince | mwhahaha: I've got the tripleo fix here: https://review.openstack.org/#/c/385078/ | 16:46 |
| *** rwsu has quit IRC | 16:46 | |
| *** derekh has quit IRC | 16:46 | |
| chem | bandini: ah, oki that's the one. Thanks. | 16:46 |
| dprince | mwhahaha: my take is I'd rather leave this parameter an integer in Tripleo. Because is should be... | 16:46 |
| mwhahaha | dprince: sure i'll abandon mine it was last week | 16:46 |
| mwhahaha | dprince: peronsally i don't agree with the use of 0 making it to the config file for any of the workers unless it specifically triggers a special case in the service, 0 or -1 should trigger the 'default' handling | 16:47 |
| mwhahaha | dprince: puppet has already started using os_worker instead of processor count to improve the default number of workers | 16:48 |
| dprince | mwhahaha: ack, sorry for the confusion. We agree there. | 16:48 |
| dprince | mwhahaha: I think this patch to nova will land FWIW, but my t-h-t fix keeps it as an integer and allows us to move on now. No puppet fixes required ATM I think | 16:48 |
| EmilienM | marios, jistr: so I would like our design sessions focused on that | 16:49 |
| mwhahaha | dprince: yea unfortunately my need for consistency across all things really doesn't like the novaworker only case but we should at least create a bug to fix the others | 16:49 |
| marios | EmilienM: ack i will revisit the spec and that etherpad ... i had a first pass but will revise before summit (not sure when ;) ) | 16:49 |
| dprince | mwhahaha: all of them should be integers I think | 16:49 |
| EmilienM | marios, jistr: the group to discuss about our problems to solve to make it happen | 16:49 |
| EmilienM | marios: thanks for taking your time on it, it's really important to have a productive session. | 16:50 |
| mwhahaha | dprince: yea they should all be integers, but we should handle the default cause of use # processors or whatever be consistently applied all workers | 16:50 |
| mwhahaha | dprince: so we should come up with either 0 or -1 where it's the default and it falls back to puppet/service defaults for these numbers. | 16:51 |
| mwhahaha | dprince: there by allowing people to tweak the workers if need be but having the defaults ship | 16:51 |
| mwhahaha | dprince: it was my understanding that was what the 0 was supposed to do | 16:51 |
| mwhahaha | dprince: basically your tht logic needs to be applied to *all* workers | 16:52 |
| mwhahaha | not just nova | 16:52 |
| mwhahaha | so that should get captured in a bug for later | 16:52 |
| dprince | mwhahaha: lets see if people by the Nova logic patch. If they do then I'd like to make them all integers perhaps using a similar mechanism | 16:52 |
| dprince | mwhahaha: FWIW we've got other flags that would benefit from stronger typeing in Heat. The 'Debug' flags are mostly strings I think and should actually be booleans. The conditionals would help there too | 16:53 |
| *** rbrady is now known as rbrady-run | 16:54 | |
| dprince | I said flags. I mean 'parameters' | 16:54 |
| mwhahaha | yea i'm not against typing, it would be nice to have a null or somthing for types tho | 16:54 |
| mwhahaha | where it's an integer or null | 16:54 |
| *** rbowen has quit IRC | 16:54 | |
| *** akuznetsov has quit IRC | 16:58 | |
| *** bana_k has joined #tripleo | 17:02 | |
| *** kjw3 has joined #tripleo | 17:02 | |
| *** ayoung has joined #tripleo | 17:10 | |
| *** ayoung has quit IRC | 17:12 | |
| *** ayoung has joined #tripleo | 17:13 | |
| *** ayoung has quit IRC | 17:14 | |
| openstackgerrit | Merged openstack/instack-undercloud: Make HAProxy terminate OpenStack internal/admin endpoints https://review.openstack.org/360366 | 17:15 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Populate vnc_api_lib.ini on compute nodes with OpenContrail https://review.openstack.org/383445 | 17:16 |
| openstackgerrit | Brad P. Crochet proposed openstack/tripleo-common: Make UpdateManager use a passed-in stack setup https://review.openstack.org/385096 | 17:17 |
| *** ayoung has joined #tripleo | 17:17 | |
| pradk | rhallisey, is there a way to fix the context issue directly in openstack-selinux module? | 17:23 |
| pradk | rhallisey, the issue is we rely on a 3rd party puppet module for redis.. we can try submitting a patch to run exec , but chances are they wont accept | 17:23 |
| *** fultonj has quit IRC | 17:24 | |
| pradk | rhallisey, i'm curious why we are running into this now.. this has been the case with redis for a while now.. never had this issue | 17:24 |
| rhallisey | pradk, the problem has to do with when/how /var/run/redis is created | 17:25 |
| *** fultonj has joined #tripleo | 17:25 | |
| rhallisey | could we provide them instructions to fix it? | 17:25 |
| rhallisey | they just need to run the restorecon | 17:26 |
| pradk | rhallisey, so why would the context be wrong to being with? | 17:27 |
| *** pkovar has quit IRC | 17:28 | |
| *** ohamada_ has quit IRC | 17:28 | |
| pradk | rhallisey, we could perhaps run restorecon in the %post of redis rpm spec | 17:28 |
| pradk | thats another option | 17:28 |
| rhallisey | pradk, it depends when it is installed. Puppet or bash script will run a restore con after | 17:28 |
| rhallisey | pradk, i don't think it should be in the redis rpm | 17:29 |
| rhallisey | pradk, when in the process does this occur? | 17:29 |
| *** tosky has quit IRC | 17:29 | |
| rhallisey | deployment? undercloud setup? | 17:29 |
| pradk | rhallisey, deployment.. the overcloud deploy finishes fine, but if we check pcs status .. redis is down | 17:30 |
| rhallisey | pradk, having a restorecon in openstack-selinux might work | 17:33 |
| rhallisey | it might not | 17:33 |
| openstackgerrit | Dan Prince proposed openstack/tripleo-heat-templates: Hiera optimization: use a new hiera hook https://review.openstack.org/384757 | 17:33 |
| rhallisey | but we can try it | 17:33 |
| rhallisey | pradk, would you be able to test it if I can get you a new rpm? | 17:33 |
| pradk | rhallisey, yep i definitely can | 17:34 |
| *** trown is now known as trown|lunch | 17:34 | |
| dtrainor | Why is /etc/hosts modified to point the fqdn of the Undercloud host to 127.0.0.1? | 17:35 |
| bnemec | dtrainor: Because if the fqdn doesn't resolve properly either puppet or rabbit (or both) explodes. | 17:36 |
| slagle | it's only done if the fqdn isn't already in /etc/hosts | 17:37 |
| dtrainor | bnemec++ slagle++ thank you | 17:37 |
| slagle | so you can prevent it from happening by setting it differently | 17:37 |
| dtrainor | i'll make a habit out of modifying /etc/hosts before undercloud install | 17:37 |
| dtrainor | excellent | 17:37 |
| *** shardy has quit IRC | 17:38 | |
| *** pblaho has quit IRC | 17:39 | |
| *** ccamacho has quit IRC | 17:39 | |
| *** cylopez has quit IRC | 17:42 | |
| *** kjw3 has quit IRC | 17:46 | |
| *** jpena is now known as jpena|off | 17:49 | |
| rhallisey | pradk, I have a build for you | 17:51 |
| rhallisey | pradk, try that out and let me know | 17:51 |
| pradk | rhallisey, cool | 17:52 |
| *** mcornea has quit IRC | 17:53 | |
| *** kjw3 has joined #tripleo | 17:54 | |
| *** rhefner has joined #tripleo | 17:58 | |
| *** ebarrera has joined #tripleo | 17:59 | |
| *** amoralej is now known as amoralej|off | 17:59 | |
| *** pblaho has joined #tripleo | 18:01 | |
| *** rbowen has joined #tripleo | 18:03 | |
| *** ebarrera has quit IRC | 18:03 | |
| *** abregman|afk has quit IRC | 18:04 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Move qemu-img installation after repo setup https://review.openstack.org/384628 | 18:09 |
| *** tiswanso has quit IRC | 18:09 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Populate vnc_api_lib.ini on compute nodes with OpenContrail https://review.openstack.org/383446 | 18:09 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Try temprevert of keystone domain change on mitaka https://review.openstack.org/385114 | 18:13 |
| jdennis | is tripleo.org down? it's not responding. | 18:17 |
| bnemec | jdennis: It looks like it. | 18:19 |
| bnemec | dprince: ^ | 18:19 |
| dprince | bnemec: checking | 18:20 |
| dprince | bnemec: I can't get in via ssh either. First time this has ever happened. Let me reboot my precious VM | 18:21 |
| bnemec | dprince: Cool, thanks | 18:21 |
| *** chandankumar has joined #tripleo | 18:24 | |
| *** kjw3 has quit IRC | 18:27 | |
| *** tiswanso has joined #tripleo | 18:29 | |
| *** tiswanso has quit IRC | 18:29 | |
| *** tiswanso has joined #tripleo | 18:29 | |
| *** kjw3 has joined #tripleo | 18:45 | |
| *** trown|lunch is now known as trown | 18:46 | |
| *** chandankumar has quit IRC | 18:53 | |
| *** dsneddon_ has joined #tripleo | 19:03 | |
| *** dsneddon_ has quit IRC | 19:04 | |
| EmilienM | slagle: http://logs.openstack.org/96/385096/1/check/gate-tripleo-ci-centos-7-nonha-multinode/7172c29/console.html#_2016-10-11_17_37_13_050271 | 19:11 |
| EmilienM | slagle: wtf? lol | 19:11 |
| EmilienM | something is trying to use epel here | 19:11 |
| EmilienM | on the subnode2 | 19:12 |
| EmilienM | Repo-id : epel/x86_64 | 19:12 |
| EmilienM | see http://logs.openstack.org/96/385096/1/check/gate-tripleo-ci-centos-7-nonha-multinode/7172c29/logs/subnode-2/var/log/host_info.txt.gz | 19:12 |
| slagle | EmilienM: is epel-release installed? | 19:13 |
| slagle | we should remove that if it is present | 19:13 |
| EmilienM | yes ! | 19:13 |
| EmilienM | it is installed | 19:13 |
| EmilienM | slagle: I thought we removed the element for that | 19:14 |
| EmilienM | slagle: maybe it's in the image | 19:14 |
| slagle | EmilienM: we dont use our images for multinode jobs | 19:15 |
| slagle | that's why we remove epel-release in repo-setup | 19:15 |
| EmilienM | the package is not on undercloud though | 19:15 |
| slagle | subnode2 wouldnt be the undercloud | 19:16 |
| EmilienM | right, the package is on overcloud | 19:17 |
| slagle | we havent removed epel from the image-yaml in tripleo-common | 19:18 |
| EmilienM | nice catch | 19:18 |
| EmilienM | slagle: you want to submit the patch? | 19:19 |
| slagle | NO | 19:19 |
| slagle | b/c there already is one | 19:19 |
| slagle | just hanging out waiting for reviews :) | 19:19 |
| slagle | https://review.openstack.org/#/c/368976/ | 19:19 |
| EmilienM | no more | 19:19 |
| EmilienM | trown++ | 19:19 |
| trown | hehe | 19:20 |
| slagle | our surface area for errors just decreased | 19:22 |
| trown | is that the last remnant of EPEL? | 19:22 |
| slagle | i think so | 19:22 |
| trown | woot | 19:22 |
| mwhahaha | if anyone wants to review a trivial puppet typo fix, https://review.openstack.org/#/c/383934/ | 19:25 |
| openstackgerrit | Merged openstack/tripleo-heat-templates: Actually start the systemd services in step3 of the major-upgrade step https://review.openstack.org/384912 | 19:28 |
| *** kjw3 has quit IRC | 19:35 | |
| thrash | need a final +2: | 19:36 |
| thrash | and... | 19:36 |
| openstackgerrit | Carlos Camacho proposed openstack/tripleo-common: Default the J2 excludes files to safe values. https://review.openstack.org/384321 | 19:36 |
| thrash | for some reason I can't copy/paste. :P | 19:36 |
| thrash | https://review.openstack.org/#/c/381899/ | 19:36 |
| thrash | there... | 19:36 |
| thrash | sheesh | 19:36 |
| *** yamahata has joined #tripleo | 19:44 | |
| *** rbrady-run is now known as rbrady | 19:46 | |
| *** dprince has quit IRC | 19:48 | |
| stevebaker | morning | 19:53 |
| *** Goneri has quit IRC | 19:58 | |
| gfidente | stevebaker if only it wasn't 21.58 here! | 19:59 |
| stevebaker | gfidente: :) | 19:59 |
| panda|afk | any ideas what this could be http://logs.openstack.org/28/384628/4/check-tripleo/gate-tripleo-ci-centos-7-ovb-nonha-mitaka/688da1d/console.html#_2016-10-11_12_29_42_888733 | 20:01 |
| *** mhenkel has quit IRC | 20:02 | |
| rook | lucasagomes: ping any chance you are still around? | 20:05 |
| *** another_larsks is now known as larsks | 20:05 | |
| lucasagomes | rook, I was about to call it aday | 20:06 |
| lucasagomes | rook, but what's up ? | 20:06 |
| rook | lucasagomes: naw man, go for it. | 20:06 |
| rook | lucasagomes: i wanted to see where i could insert some of my logic in my setups | 20:06 |
| lucasagomes | rook, no worries, if it's quick I can help out | 20:06 |
| rook | ie when we are in a deploy and things go from build->active | 20:06 |
| rook | where can i find that in code, so I can look to add something for checking status. | 20:07 |
| rook | status/connectivity | 20:07 |
| lucasagomes | rook, so ironic status goes like: deploying -> deploy wait [-> deploying] -> active | 20:07 |
| lucasagomes | ops I mean | 20:07 |
| rook | building right? | 20:08 |
| lucasagomes | rook, that's in nova | 20:08 |
| lucasagomes | but underneath ironic is going to have those status ^ | 20:08 |
| rook | ok | 20:08 |
| lucasagomes | rook, so the first deploying is just setting up the environemnt to boot the ramdisk, in the code you will see funciotions like prepare_ramdisk() | 20:09 |
| rook | i think the handoff is where we could do some better checking.. Ironic says this node is done building, handing it off from there, but then nothing happens because something went fubar with that node. | 20:09 |
| lucasagomes | rook, oh, like not booting the ramdisk ? | 20:09 |
| rook | so, it boots the ramdisk, installs, but then doesn't reboot. | 20:09 |
| rook | something went fubar -- so the node is still sitting in the PXE | 20:09 |
| rook | but ironic already determined this node is done. | 20:10 |
| lucasagomes | rook, so ironic the node still have the deploy ramdisk booted in it ? Or it rebooted and then nothing happened ? | 20:11 |
| rook | nothing happening. however w/ a nova list/ ironic node-list i see active in nova and active in ironic (no more wait-call-back) | 20:12 |
| rook | lucasagomes: this is in regard to the email i sent this morning | 20:13 |
| lucasagomes | rook, I haven't seem the email, sorry | 20:13 |
| lucasagomes | rook, so yeah, Ironic kinda assumes the node was deployed correct after that stage | 20:13 |
| rook | right. | 20:13 |
| rook | lucasagomes: so it is a hand off issue. | 20:13 |
| lucasagomes | it doesn't actually perform any check whether it has booted correctly or not | 20:14 |
| lucasagomes | yeah | 20:14 |
| lucasagomes | it will happen after the "reboot_to_instance()" method is called | 20:14 |
| * lucasagomes finds in code | 20:14 | |
| rook | so, after the node is deployed/installed, no one checks to see if the desired state is met, ie booted into a OS | 20:14 |
| lucasagomes | rook, https://github.com/openstack/ironic/blob/659f951d72e96f39bb967455a6855682e517ca43/ironic/drivers/modules/agent.py#L312 | 20:14 |
| lucasagomes | rook, nop, cause I don't think we have any means to actually do that | 20:14 |
| rook | lucasagomes well we have all the information to do that. | 20:15 |
| lucasagomes | specially with network isolation | 20:15 |
| rook | each should be in the provisioning network... | 20:15 |
| lucasagomes | rook, right yeah so we probably need to add some checks to it, what do you have in mind ? | 20:15 |
| rook | lucasagomes the check IMHO is a simple ping, after a desired timeout (something that dtantsur|afk and I have been emailing about)... if no connectivity after the timeout, we kick the host -- and if this fails, we re-schedule the host. | 20:16 |
| *** jayg is now known as jayg|g0n3 | 20:17 | |
| rook | this will add a ton of resilience | 20:17 |
| lucasagomes | rook, yeah we probably can think of something like that, would be better to have it performed prior to marking the node as active | 20:18 |
| rook | True. | 20:18 |
| lucasagomes | right after the reboot we could check if the node is up | 20:18 |
| lucasagomes | and if so, reboot | 20:18 |
| lucasagomes | I mean | 20:18 |
| lucasagomes | mark as active | 20:18 |
| * lucasagomes brain's slow | 20:18 | |
| rook | right, no it is late. no problem | 20:18 |
| rook | I just think this will make the babysitting issue of OOO much better. | 20:18 |
| rook | much less handholding when you are doing a ton of nodes. | 20:19 |
| lucasagomes | yeah | 20:19 |
| leifmadsen | please! :)_ | 20:19 |
| rook | hush leifmadsen | 20:19 |
| rook | :P | 20:19 |
| leifmadsen | rook: pfffft | 20:19 |
| rook | lucasagomes head out man, thanks for sticking around to discuss. | 20:19 |
| lucasagomes | rook, would be good to add an RFE for it | 20:19 |
| rook | lucasagomes: i have a email titled Suggestions -- give it a look in your freetime. sent to openstack-dev | 20:19 |
| rook | OOO Suggestions i think | 20:20 |
| lucasagomes | rook, totally+ I will add to my todo list to look at it tomorrow morning | 20:20 |
| lucasagomes | rook, thanks for it! | 20:20 |
| lucasagomes | rook, alright! I'm heading now | 20:20 |
| lucasagomes | see ya | 20:21 |
| *** lucasagomes is now known as lucas-afk | 20:21 | |
| rook | later! | 20:25 |
| openstackgerrit | Dan Sneddon proposed openstack/os-net-config: Add NIC Mapping Reporting Feature https://review.openstack.org/383516 | 20:26 |
| *** gfidente has quit IRC | 20:32 | |
| *** panda|afk is now known as panda | 20:33 | |
| *** bana_k has quit IRC | 20:34 | |
| *** bana_k has joined #tripleo | 20:35 | |
| bnemec | Wow, epel is having a really bad day. | 20:36 |
| openstackgerrit | Pradeep Kilambi proposed openstack/instack-undercloud: Enable telemetry services by default on undercloud https://review.openstack.org/385146 | 20:43 |
| pradk | bnemec, ^^ | 20:43 |
| remix_tj | hello, i see that in newton's tripleo there still isn't neutron-lbaas. Is there any support plan? can be configured in some way? | 20:48 |
| *** trown is now known as trown|outtypewww | 20:51 | |
| *** dsavineau has quit IRC | 20:52 | |
| pradk | can i get a +A on this please -> https://review.openstack.org/#/c/384481/ | 20:53 |
| *** jcoufal_ has quit IRC | 20:56 | |
| *** rbrady is now known as rbrady-afk | 20:59 | |
| *** egafford has quit IRC | 21:02 | |
| *** kbyrne has quit IRC | 21:04 | |
| *** akrivoka has quit IRC | 21:08 | |
| *** kbyrne has joined #tripleo | 21:10 | |
| openstackgerrit | Brad P. Crochet proposed openstack/python-tripleoclient: Downloads templates from swift before processing update https://review.openstack.org/385151 | 21:13 |
| *** rlandy has quit IRC | 21:17 | |
| *** yamahata has quit IRC | 21:26 | |
| *** jeckersb is now known as jeckersb_gone | 21:27 | |
| *** dbecker has quit IRC | 21:28 | |
| *** tdasilva has quit IRC | 21:30 | |
| *** tdasilva has joined #tripleo | 21:33 | |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Default $STABLE_RELEASE in bootstrap-overcloud-full.sh https://review.openstack.org/385159 | 21:34 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Add --setup-nodepool-files option https://review.openstack.org/385160 | 21:34 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Create dummy overcloud-full in --boostrap-subnodes https://review.openstack.org/385161 | 21:34 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Add configurable mtu to --mulitnode-setup https://review.openstack.org/385162 | 21:34 |
| *** yamahata has joined #tripleo | 21:36 | |
| *** fultonj has quit IRC | 21:47 | |
| bnemec | slagle: probably just killed rh1 :-P | 21:49 |
| openstackgerrit | Merged openstack/tripleo-common: Make UpdateManager use a passed-in stack setup https://review.openstack.org/385096 | 21:55 |
| *** zigo has quit IRC | 22:02 | |
| *** zigo has joined #tripleo | 22:05 | |
| *** zigo is now known as Guest50630 | 22:05 | |
| jrist | EmilienM: are you already PTL? | 22:09 |
| jrist | EmilienM: :) | 22:09 |
| *** bana_k has quit IRC | 22:09 | |
| jrist | EmilienM: I wonder if we can get some extra time/space at the PTG for TripleO-UI too? | 22:10 |
| EmilienM | jrist: well, I'll be for the Summit | 22:11 |
| EmilienM | jrist: of course | 22:11 |
| jrist | EmilienM: do you think I should request that separately? | 22:12 |
| EmilienM | jrist: do you pay in popcorn? | 22:12 |
| jrist | only the finest | 22:12 |
| EmilienM | no need, you let me know that's enough | 22:12 |
| EmilienM | I'll ask for enough space | 22:12 |
| EmilienM | depending of the amount of popcorn you'll pay | 22:12 |
| jrist | what flavor? | 22:12 |
| jrist | :) | 22:12 |
| EmilienM | sweet | 22:13 |
| bkero | Sweet popcorn? Kettle corn? | 22:13 |
| *** panda is now known as panda|zZ | 22:14 | |
| jrist | that's wide open, bkero | 22:16 |
| jrist | sweet sweet popcorn | 22:16 |
| EmilienM | bnemec: nice https://review.openstack.org/#/c/385179/ I think it can fix the overcloud pingtest on mitaka Ci | 22:18 |
| bnemec | EmilienM: Yeah, that's hopefully the last blocker to having working stable branch jobs again. :-) | 22:19 |
| EmilienM | nice | 22:19 |
| EmilienM | bnemec: did you check if rspec tests need an update? | 22:19 |
| bnemec | EmilienM: Ah shoot, I forgot. It probably does. | 22:19 |
| thrash | bnemec: have you tried the update job again against https://review.openstack.org/#/c/381899/ | 22:23 |
| thrash | does reverify actually work? Or does one use recheck if the verify fails? | 22:25 |
| bnemec | thrash: Not recently. I might just merge https://review.openstack.org/#/c/374406/ and then you can try it by just running check experimental. | 22:25 |
| bnemec | It's only failing on known issues with the stable branches, and doesn't actually affect any of those jobs anyway. | 22:25 |
| thrash | bnemec: is that non-voting? | 22:26 |
| thrash | I suppose it would be. :P | 22:26 |
| thrash | nm | 22:26 |
| bnemec | thrash: Sort of. The updates job is currently voting experimental. | 22:26 |
| thrash | I thought that did something different. :) | 22:26 |
| bnemec | thrash: Okay, I pushed the button on the updates job fix. Once that merges you should be able to test any patch for updates using check experimental. | 22:28 |
| bnemec | Once we get it working again we can put it back in the regular queue. | 22:29 |
| thrash | bnemec: awesome. | 22:29 |
| *** kjw3 has joined #tripleo | 22:50 | |
| *** tiswanso has quit IRC | 22:58 | |
| *** tiswanso has joined #tripleo | 22:59 | |
| *** bana_k has joined #tripleo | 23:00 | |
| *** bana_k has quit IRC | 23:02 | |
| *** rhallisey has quit IRC | 23:03 | |
| *** anshul has quit IRC | 23:07 | |
| openstackgerrit | Steve Baker proposed openstack/os-collect-config: WIP ec2 collector attempt config-drive before metadata server https://review.openstack.org/369731 | 23:18 |
| openstackgerrit | Steve Baker proposed openstack/os-collect-config: Treat ec2 collector data as immutable https://review.openstack.org/384712 | 23:18 |
| openstackgerrit | Steve Baker proposed openstack/os-collect-config: Use TEST-NET-1 for unit tests, not 127.0.0.1 https://review.openstack.org/385196 | 23:18 |
| *** rajinir has quit IRC | 23:26 | |
| *** morazi has quit IRC | 23:32 | |
| *** pmannidi|brb is now known as pmannidi | 23:35 | |
| *** tiswanso has quit IRC | 23:38 | |
| *** bana_k has joined #tripleo | 23:38 | |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-specs: Propose TripleO Squads policy https://review.openstack.org/385201 | 23:43 |
| *** hrybacki has quit IRC | 23:54 | |
| *** hrybacki has joined #tripleo | 23:55 | |
| *** bana_k has quit IRC | 23:56 | |
| *** bana_k has joined #tripleo | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!