Wednesday, 2018-05-23

*** nurdie has quit IRC00:01
*** nurdie has joined #openstack-ansible00:02
*** cshen has quit IRC00:05
*** nurdie has quit IRC00:06
*** gyee has quit IRC00:07
*** chyka has joined #openstack-ansible00:11
*** tux_ has joined #openstack-ansible00:11
*** cshen has joined #openstack-ansible00:12
*** chyka_ has quit IRC00:14
*** rpittau has quit IRC00:16
*** chyka has quit IRC00:17
*** chyka has joined #openstack-ansible00:17
*** chyka has quit IRC00:19
*** chyka has joined #openstack-ansible00:19
*** chyka has quit IRC00:25
*** chyka_ has joined #openstack-ansible00:25
*** cshen has quit IRC00:28
*** chyka_ has quit IRC00:29
*** chyka has joined #openstack-ansible00:29
*** dave-mcc_ has quit IRC00:30
*** eumel8 has joined #openstack-ansible00:38
*** cshen has joined #openstack-ansible00:38
*** spsurya has quit IRC00:41
*** cshen has quit IRC00:50
*** dcreno has joined #openstack-ansible00:50
*** dcreno has quit IRC00:54
*** chyka_ has joined #openstack-ansible00:59
*** chyka has quit IRC01:01
*** chyka_ has quit IRC01:08
*** markvoelker has joined #openstack-ansible01:17
*** armaan has quit IRC01:20
*** armaan has joined #openstack-ansible01:21
*** rudysp has joined #openstack-ansible01:28
*** rudysetyop has joined #openstack-ansible01:29
*** eumel8 has quit IRC01:29
tux_which playbook install repo packages? because one of my infra node has empty directory in repo container /var/www/repo/os-releases/16.0.1001:31
*** eumel8 has joined #openstack-ansible01:32
*** rudysp has quit IRC01:33
*** dave-mccowan has joined #openstack-ansible01:40
*** nurdie has joined #openstack-ansible01:41
tux_I am getting skipping: no hosts matched   on infra1 and infra201:43
tux_http://paste.openstack.org/show/721681/01:47
tux_I do have entry in openstack_user_config.yml for repo-infra01:47
*** rudysetyop has quit IRC01:51
tux_I have reported bug here https://bugs.launchpad.net/openstack-ansible/+bug/177277802:04
openstackLaunchpad bug 1772778 in openstack-ansible "skipping: no hosts matched" [Undecided,New]02:04
*** nurdie has quit IRC02:05
tux_what could be wrong here?02:08
*** cshen has joined #openstack-ansible02:28
*** cshen has quit IRC02:34
*** cshen has joined #openstack-ansible02:37
openstackgerritShangXiao proposed openstack/openstack-ansible-os_octavia master: Add doc links information to README.rst  https://review.openstack.org/57010503:09
openstackgerritMerged openstack/openstack-ansible-os_nova master: SUSE: Explicitly add all the available libvirt storage drivers  https://review.openstack.org/56976003:21
*** nurdie has joined #openstack-ansible03:47
*** tux_ has quit IRC03:50
*** armaan has quit IRC03:50
*** armaan has joined #openstack-ansible03:50
*** rpittau has joined #openstack-ansible03:52
*** poopcat has quit IRC04:03
*** dxiri has quit IRC04:20
*** dxiri has joined #openstack-ansible04:31
*** udesale has joined #openstack-ansible04:35
*** dxiri has quit IRC04:40
*** DanyC has joined #openstack-ansible04:47
*** cshen has quit IRC04:51
*** chyka has joined #openstack-ansible04:52
*** chyka has quit IRC04:57
*** radeks has joined #openstack-ansible04:58
*** radeks has quit IRC05:08
*** rpittau has quit IRC05:13
*** eumel8 has quit IRC05:15
*** cshen has joined #openstack-ansible05:16
*** cshen has quit IRC05:21
*** Taseer has quit IRC05:23
*** radeks has joined #openstack-ansible05:24
*** dcreno has joined #openstack-ansible05:24
*** cshen has joined #openstack-ansible05:26
*** dcreno has quit IRC05:27
*** DanyC_ has joined #openstack-ansible05:27
*** Taseer has joined #openstack-ansible05:28
*** Taseer has quit IRC05:30
*** DanyC has quit IRC05:31
*** DanyC_ has quit IRC05:32
*** cshen has quit IRC05:33
*** dave-mccowan has quit IRC05:42
*** cshen has joined #openstack-ansible05:42
*** pvradu has joined #openstack-ansible05:45
*** dcreno has joined #openstack-ansible05:45
*** dcreno has quit IRC05:48
*** nurdie has quit IRC05:49
*** yolanda has joined #openstack-ansible05:50
*** cshen has quit IRC05:52
*** cshen has joined #openstack-ansible05:58
*** yolanda has quit IRC06:05
*** cshen has quit IRC06:06
*** nurdie has joined #openstack-ansible06:10
*** nurdie has quit IRC06:10
*** rudysp has joined #openstack-ansible06:14
*** masber has joined #openstack-ansible06:16
*** udesale_ has joined #openstack-ansible06:16
*** udesale has quit IRC06:18
*** nurdie has joined #openstack-ansible06:19
*** yolanda has joined #openstack-ansible06:19
*** masber has quit IRC06:21
*** armaan has quit IRC06:25
*** armaan has joined #openstack-ansible06:25
*** jbadiapa_ has quit IRC06:32
*** _jbadiapa has joined #openstack-ansible06:32
*** pcaruana has joined #openstack-ansible06:33
*** lvdombrkr has joined #openstack-ansible06:36
*** radek__ has joined #openstack-ansible06:42
*** udesale__ has joined #openstack-ansible06:44
*** mma has joined #openstack-ansible06:46
*** dcreno has joined #openstack-ansible06:46
*** udesale_ has quit IRC06:46
*** nurdie_ has joined #openstack-ansible06:48
*** dcreno has quit IRC06:50
*** nurdie has quit IRC06:50
*** nurdie has joined #openstack-ansible06:51
*** udesale__ has quit IRC06:52
*** udesale__ has joined #openstack-ansible06:52
*** armaan has quit IRC06:52
*** armaan has joined #openstack-ansible06:53
*** shardy has joined #openstack-ansible06:54
*** nurdie has quit IRC06:55
*** nurdie_ has quit IRC06:55
*** pmannidi has quit IRC06:57
*** cshen has joined #openstack-ansible06:57
*** threestrands has quit IRC06:57
*** udesale_ has joined #openstack-ansible07:01
*** georgem1 has joined #openstack-ansible07:01
*** cshen has quit IRC07:02
*** rudysp has quit IRC07:03
*** udesale__ has quit IRC07:04
*** mbuil has joined #openstack-ansible07:05
*** udesale_ has quit IRC07:11
*** udesale has joined #openstack-ansible07:12
*** udesale has quit IRC07:12
*** udesale has joined #openstack-ansible07:12
*** georgem1 has left #openstack-ansible07:16
*** epalper has joined #openstack-ansible07:18
*** tomtom001 has quit IRC07:22
*** tomtom001 has joined #openstack-ansible07:23
*** _jbadiapa has quit IRC07:23
*** parasitid has joined #openstack-ansible07:31
parasitidhi07:31
parasitid@all: in the openstack-ansible doc07:31
parasitidit is said one should "git clone master"07:31
parasitidisn't there some "stable" release we should prefer ?07:32
parasitidi'm confused07:32
parasitidbecause i have some issues with master branch07:32
parasitidi don't know if its because it's considered unstable07:32
parasitidor if i've missed somehting07:32
*** sxc731 has joined #openstack-ansible07:33
*** _jbadiapa has joined #openstack-ansible07:38
evrardjpdepends on which doc you read parasitid07:39
parasitidevrardjp: meaning?07:40
parasitidevrardjp: i'm reading "latest"07:41
parasitid;)07:41
parasitidbut i cant find a way to choose "stable"07:41
parasitidhttps://docs.openstack.org/project-deploy-guide/openstack-ansible/latest/deploymenthost.html07:42
evrardjpparasitid: go to https://docs.openstack.org07:42
evrardjpselect deployment guide07:42
evrardjptada07:42
evrardjp:)07:43
evrardjpI am tired, I am done for the day. ttyl everyone!07:43
parasitidhanhan07:44
parasitidthanks07:44
parasitid;)07:44
*** cshen has joined #openstack-ansible07:45
*** udesale_ has joined #openstack-ansible07:51
*** udesale has quit IRC07:54
*** cshen has quit IRC07:56
*** electrofelix has joined #openstack-ansible08:02
*** DanyC has joined #openstack-ansible08:03
*** mbuil has quit IRC08:07
*** gkadam has joined #openstack-ansible08:17
*** chyka has joined #openstack-ansible08:28
*** chyka has quit IRC08:33
*** cshen has joined #openstack-ansible08:37
*** shardy has quit IRC08:39
*** cshen has quit IRC08:41
*** gameon has joined #openstack-ansible08:48
*** shardy has joined #openstack-ansible08:53
parasitid@all again even with queens doc on 17.0.4 branch08:53
parasitidi still hit the same issue08:54
parasitidpip3 is not installed in the ubuntu xenial lxc containers08:54
jrosserparasitid: if you could pastebin your actual error somewhere that would help09:00
jrosserparasitid: are you still needing help with which branch to use?09:00
*** armaan has quit IRC09:04
*** armaan has joined #openstack-ansible09:04
*** sxc731 has quit IRC09:08
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible master: [DNM] - Testing Ceph AIO on openSUSE  https://review.openstack.org/50745109:17
parasitidjrosser: no i've reboostrapped my nodes09:17
parasitidbased on 17.0.4 branch09:18
parasitidto deploy queens09:18
parasitidthe error is : fatal: [myopenstack_shared_infra_2]: FAILED! => {"attempts": 5, "changed": false, "failed": true, "msg": "Unable to find any of pip3 to use.  pip needs to be installed."}09:18
parasitidjrosser: i've setup deployer + target hosts accordingly to the queens doc, based on ubuntu 16.04 hosts09:20
parasitidthe recap is : null_resource.ansible_setup_hosts (remote-exec): myopenstack_cinder_0       : ok=140  changed=49   unreachable=0    failed=009:20
parasitidnull_resource.ansible_setup_hosts (remote-exec): myopenstack_compute_0      : ok=140  changed=49   unreachable=0    failed=009:20
parasitidnull_resource.ansible_setup_hosts (remote-exec): myopenstack_haproxy_0      : ok=140  changed=49   unreachable=0    failed=009:20
parasitidnull_resource.ansible_setup_hosts (remote-exec): myopenstack_shared_infra_0 : ok=163  changed=59   unreachable=0    failed=109:20
parasitidnull_resource.ansible_setup_hosts (remote-exec): myopenstack_shared_infra_1 : ok=164  changed=59   unreachable=0    failed=109:20
parasitidnull_resource.ansible_setup_hosts (remote-exec): myopenstack_shared_infra_2 : ok=165  changed=60   unreachable=0    failed=109:20
parasitiderrors on "shared infra" nodes are all the same: unable to find any of pip3 to use09:21
odyssey4meparasitid nothing is meant to use pip3, so that's very odd09:24
odyssey4mealso please use pastebin or something similar for your logs, not the channel09:24
odyssey4meand  please pastebin more than just the failure output - at least the task name is useful, but a -vvv output of the task that fails would be useful too09:25
*** cshen has joined #openstack-ansible09:26
parasitidodyssey4me: ok sorry for the paste09:26
parasitidhere it is09:29
parasitidodyssey4me: jrosser: https://pastebin.com/XdPfD5dF09:29
parasitidi haven't run with the -vvv09:29
parasitidhas the playbooks are quite long to run09:29
parasitidwill do in a min09:29
*** cshen has quit IRC09:30
odyssey4methat output is quite odd, it doesn't look standard09:30
*** udesale__ has joined #openstack-ansible09:31
odyssey4meyou don't have to use setup-hosts every time, if you look at the contents of that playbook, it's just importing others which you can execute individually too09:32
odyssey4meso if the first passes, try the second, etc09:32
parasitidok09:32
*** yolanda has quit IRC09:32
*** udesale has joined #openstack-ansible09:32
parasitidthing is, i run the playbooks through a terraform provisionner09:32
odyssey4meok, and has the config you've implemented worked before?09:33
parasitidthats why lines are prefixed by "null_resource ..... remote-exec"09:33
odyssey4mehave you tried it outside that provisioner?09:33
parasitidyes09:33
*** udesale_ has quit IRC09:33
parasitidsame result09:33
parasitidand "no ive never managed to get it work"09:33
parasitidalso09:34
odyssey4mealright, well it looks like it's failing to install pip packages in the lxc_hosts role - but that's odd because it should have installed pip before that, so are you sure there isn't a failure higher up?09:34
parasitidi've set a "-e ansible_python_interpreter=/usr/bin/python3"09:34
odyssey4meok, that's not good09:34
odyssey4mewe do not yet support python309:35
odyssey4meansible doesn't fully yet, nor does openstac09:35
parasitidbecause it was previously failing with "/usr/bin/python" not found09:35
parasitidoh09:35
odyssey4meas the docs say, python2 needs to be installed on all the hosts09:35
*** cshen has joined #openstack-ansible09:35
parasitidbut there's no python2 installed in lxc containers09:35
parasitidonly python309:36
*** udesale__ has quit IRC09:36
odyssey4megiven https://github.com/openstack/openstack-ansible-lxc_hosts/blob/stable/queens/vars/ubuntu-16.04.yml#L114 there should be, so if there isn't then you have another problem09:36
parasitidwell09:37
parasitidmaybe because before i was running the master branch09:37
odyssey4memaster also installs it into the containers09:37
odyssey4mebut to be clear here, if you've installed from the master branch, then are using the same hosts for queens, then you're in for a bad time09:38
parasitidi've rebooted the hosts09:38
odyssey4merefresh the hosts back to a clean state, then try the build again09:38
parasitidmeaning "destroyed09:38
parasitidi can share my terraform playbook09:38
parasitidyou'll see i dont do anything particular exectp the -e09:39
odyssey4mewell, take that python interpreter thing out and have another go09:39
odyssey4meif you get errors from that then lets look at it again09:39
parasitidok09:40
*** cshen has quit IRC09:43
*** sxc731 has joined #openstack-ansible09:44
*** yolanda has joined #openstack-ansible09:45
odyssey4meparasitid if you could gist your terraform playbook, I'd be interested to see it09:48
parasitidok09:49
parasitidactually09:49
parasitidi think i'll push it on github09:49
parasitid;)09:49
*** gameon has quit IRC09:50
*** cshen has joined #openstack-ansible09:51
parasitidodyssey4me: https://github.com/yanndegat/bootstrap-openstack/tree/openstack-ansible09:52
parasitidhere it is09:52
parasitidits a wip09:52
parasitiddoc isnt up to date09:53
*** gaudenz has joined #openstack-ansible09:53
gaudenzHi09:54
gaudenzI tried to upgrade my Newton deployment to the latest version of newton before further upgrading to Ocata, but it seems that all the newton-eol tags in the openstack-ansible repositories are gone? What happened to them?09:55
*** cshen has quit IRC09:56
gaudenzAs the stable/newton branches are gone too, I have no way to find out the last commit in the Newton series. Any help would be appreciated.09:57
*** nurdie has joined #openstack-ansible09:57
*** nurdie has quit IRC10:02
odyssey4meparasitid fyi pike is 16.x: https://github.com/yanndegat/bootstrap-openstack/blob/openstack-ansible/variables.tf#L1610:03
odyssey4meah, another issue is that you're using 'ansible-playbook' instead of 'openstack-ansible'... the second is a shell script we implement which makes sure we import the right extra vars and set the right env vars10:05
odyssey4meit's a wrapper around ansible-playbook10:05
odyssey4mewell, this is based on the readme's description...10:06
parasitidodyssey4me: ??10:07
parasitidodyssey4me: could you point me the line10:07
odyssey4meI guess you may need to use ansible-playbook for some initial bootstrapping10:07
odyssey4mehttps://github.com/yanndegat/bootstrap-openstack/blob/openstack-ansible/variables.tf#L1610:07
parasitidbecause in the provisionner i use the openstack-ansible wrapper10:07
odyssey4meah ok, my mistake then10:07
parasitidi've based the repo on a repo that boostrapped openstack on its own ansible playbooks made for educational purpose only10:08
jrosseri hope this is disposable https://github.com/yanndegat/bootstrap-openstack/blob/openstack-ansible/variables.tf#L82 :)10:08
parasitidthat's why there are some parts to rework10:09
* jrosser unmeetings, good morning odyssey4me o/10:09
parasitidjrosser: of course10:10
odyssey4meok, I see https://github.com/yanndegat/bootstrap-openstack/blob/openstack-ansible/main.tf#L262-L271 now - that's probably copying too much, but it should be ok10:10
parasitidjrosser: the password is a legacy from the original fork10:12
odyssey4meyou could probably rather use something like https://github.com/rcbops/rpc-openstack/blob/master/playbooks/configure-release.yml#L122-L132 for that initial openstack_deploy copy to get the structure there10:12
parasitidoh ok, nice10:14
odyssey4meoh, and the syntax check here should probably rather just use the first playbook in setup-hosts: https://github.com/yanndegat/bootstrap-openstack/blob/openstack-ansible/main.tf#L295-L29610:14
parasitidwell i'm running the "docs" commands as is10:14
parasitid;)10:14
odyssey4meyeah, that's fine I guess if it's working :) using openstack-hosts-setup.yml will work just fine too and will take less time10:15
odyssey4meperhaps the docs should change to use that playbook :) you could push up a patch for that10:16
odyssey4mevery interesting - thanks for sharing!10:16
parasitidi'm only trying to setup a "near real conditions" environement to test magnum10:17
parasitid;)10:17
parasitidbut thanks a lot for your feedback10:17
parasitids10:17
jrosserodyssey4me: i have a partially working designate/bind9 integration, but i think some stuff is not quite right in the designate role10:18
odyssey4mejrosser that's probably true, it doesn't get as much attention as I wish it did10:19
jrosseri'd be up for making a worked example in an appendix, but might need a bit of help10:20
odyssey4meif you need clarity on how designate's supposed to work or be configured, the #openstack-dns channel is quite helpful, and mugsie hangs out here too10:20
odyssey4mehe's at the summit this week, so unfortunately will probably not be very responsive10:21
jrosseryeah, i have a feeling the HA config isnt quite right, lots of "designate-producer.log:2018-05-22 16:21:27.728 11452 WARNING designate.coordination [-] No coordination backend configured, assuming we are the only worker. Please configure a coordination backend"10:21
odyssey4mehaving a user story up for configuring this would be great!10:22
jrossermaybe i put something together as a WIP for the ops repo10:24
*** Taseer has joined #openstack-ansible10:25
odyssey4meWell yeah, what we're trying to encourage is that we add a doc page in the user guide describing the 'story' (Implementing BIND DNS with Designate), then have that story link to the ops repo where the actual code is. The story should generally remain fairly static as it describes things at a very high level, and the ops repo content can obviously iterate as time goes by.10:26
*** udesale has quit IRC10:40
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_heat master: defaults: Allow uwsgi to autoload required plugins.  https://review.openstack.org/57016510:49
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_heat master: Add support for using distribution packages for OpenStack services  https://review.openstack.org/57016610:49
*** cshen has joined #openstack-ansible10:57
*** cshen has quit IRC11:01
*** mbuil has joined #openstack-ansible11:02
*** erolg has joined #openstack-ansible11:03
*** cshen has joined #openstack-ansible11:07
*** shardy is now known as shardy_lunch11:08
*** cshen has quit IRC11:16
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_nova master: SUSE: Add missing QEMU drivers for block devices  https://review.openstack.org/57016811:18
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible master: [DNM] - Testing Ceph AIO on openSUSE  https://review.openstack.org/50745111:19
*** cshen has joined #openstack-ansible11:21
*** cshen has quit IRC11:25
jrosserodyssey4me: very rough but its a start https://github.com/jrosser/openstack-ansible-ops/tree/designate-bind/designate-bind11:27
*** pcaruana|afk| has joined #openstack-ansible11:29
*** pcaruana has quit IRC11:29
*** cshen has joined #openstack-ansible11:36
*** cshen has quit IRC11:40
*** armaan has quit IRC11:42
*** armaan has joined #openstack-ansible11:43
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_heat master: Add support for using distribution packages for OpenStack services  https://review.openstack.org/57016611:44
*** cshen has joined #openstack-ansible11:45
*** parasitid has quit IRC11:45
*** nurdie has joined #openstack-ansible11:50
*** tux_ has joined #openstack-ansible11:50
*** nurdie has quit IRC11:54
*** cshen has quit IRC11:57
*** mkucia has joined #openstack-ansible11:58
mkuciaHello. Is there some recommended way to share the same config file across different containers?  nova-compute needs to access  multipath.conf which is used by multipath container by default. I would prefer to avoid creating two templates.11:59
*** parasitid has joined #openstack-ansible12:01
openstackgerritMerged openstack/openstack-ansible-os_neutron master: Use octavia_service_internaluri instead of url  https://review.openstack.org/54926712:01
*** nattanon has joined #openstack-ansible12:01
*** tux_ has quit IRC12:02
*** shardy_lunch is now known as shardy12:13
*** mbuil has quit IRC12:16
*** udesale has joined #openstack-ansible12:17
*** mbuil has joined #openstack-ansible12:17
*** mbuil has quit IRC12:18
*** sxc731 has quit IRC12:19
*** mbuil has joined #openstack-ansible12:20
*** sxc731 has joined #openstack-ansible12:33
parasitidodyssey4me: switching to queens branch and reverting ansible_python_interpreter seems to have fixed my issue ;)12:35
parasitidodyssey4me: thanks.12:35
parasitidodyssey4me: question: is there some playbook that could be used to "packerize" a template image to speed up the playbooks ?12:36
odyssey4meparasitid great news :)12:37
odyssey4meparasitid no, at this stage there's nothing like that - we're busy re-working the whole stack to make that kind of thing possible, and to speed up the deploy - but for now there's nothing like that yet12:37
parasitidok12:38
*** cshen has joined #openstack-ansible12:38
openstackgerritMerged openstack/openstack-ansible master: Move radosgw keystone config tasks to their own playbook  https://review.openstack.org/56570112:40
devxodyssey4me: correct me if i'm wrong but I thought that the nspawn driver creates an image which makes things faster12:41
odyssey4medevx at this point nspawn is master only, but also both lxc and nspawn use a common base image ancestor which has very little in it12:42
devxbut that is in master as well correct?12:42
*** cshen has quit IRC12:42
odyssey4meto move to something which builds container images with the software in it, then only to do runtime config on top when they deploy, is something I've tried before and have received mixed results... we have to change how we execute the deployment for that to work - hence my reply above12:43
devxthat makes sense12:44
odyssey4meto be clear - I tried it before with the current deploy method... basically build the container images in CI, then use them in the deploy... it didn't make it any faster and caused all sorts of hidden issues to show up12:44
odyssey4meso to make that work we actually need to bake it into the deployment process as a normal thing12:44
odyssey4meit's quite hard to do considering we cater for using containers, or not using containers - but we're moving to a place where it will be easier to do at least12:45
parasitidmeaning?12:46
*** cshen has joined #openstack-ansible12:47
*** dave-mccowan has joined #openstack-ansible12:47
odyssey4mewhich part would you like clarity on?12:48
andymccrcloudnull: still gonna have to go with a list or specify ids - with_indexed_items is specific to lists. not sure why we need the container networks to be a dict though12:49
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_heat master: Add support for using distribution packages for OpenStack services  https://review.openstack.org/57016612:50
parasitidwhat kind of "big refacto" could lead to a "place where it will be easier" ?12:50
odyssey4meparasitid we have to change how the software is built and deployed, so https://github.com/openstack/openstack-ansible-specs/blob/master/specs/queens/python-build-install-simplification.rst which is still ongoing12:51
parasitidok12:51
odyssey4meonce that's done, we can change how deployments are managed, so https://github.com/openstack/openstack-ansible-specs/blob/master/specs/queens/deployment-stages.rst which has not yet started because it relies on the former spec12:51
andymccrbut that'll be a big rework to ensure thats always a list12:52
odyssey4meonce that's done we'll revisit the general idea and see what's next - those are multi-cycle initiatives and we currently do not have enough hands helping to 'keep the lights on' for that work to progress more quickly than it currently is12:52
parasitidcould it somehow be a conjoin effort with "helm-openstack" ?12:52
parasitidor is it 2 completely separated initiatives ?12:53
andymccror do some conversion from dict --> list just for the purposes of creating the lxc.net.x ids, which could create issues since dicts arent ordered and could cause changes erroneously. TL;DR i think we should use a list12:53
odyssey4meno, helm is its own orchestrator so it doesn't make sense for the two projects to collaborate except on upstream changes to make life better for all deployment tools12:54
parasitidmaybe they could be based on the same "prepared container images", taking the same env vars12:54
parasitidah ok.12:55
odyssey4methe app container deployment projects can only do builds in CI, and then deploys... one of our strengths is being able to deploy and change at run time12:55
parasitidwell, on other previous intern projects12:56
odyssey4meyeah, so we may look at using the LOCI images (used by openstack-helm) at some point... but even to do that would require adjustments to how we do things... so for now we continue to make those adjustments12:56
*** cshen has quit IRC12:56
*** cshen has joined #openstack-ansible12:56
parasitidi've managed to build fully "immutable infrastructure" deployments12:56
parasitidbuild by CI12:56
parasitidbut as a prepared all the images with ansible playbooks12:56
parasitidi could output my inventory from terraform and reuse the exact same playbooks to upgrade at run time12:57
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_heat master: Add support for using distribution packages for OpenStack services  https://review.openstack.org/57016612:58
parasitidokidoki12:58
*** cshen has quit IRC13:00
odyssey4meusing images would be fine if our deployment process expected images to be used, which it currently does not (with the exception of the base image)13:01
odyssey4meto do that requires that we do the stage separation at the very least, and that we do all the 'build' bits in something that can be executed on its own13:02
*** cshen has joined #openstack-ansible13:02
parasitidok13:02
parasitidi see13:02
parasitidit makes sense13:02
odyssey4meto do that means that we need to having a staged execution across all roles and playbooks, which we do not13:02
parasitid"tag the pip install"13:02
parasitid ;)13:02
odyssey4methe work I'm busy with now is to get the python wheels builds all done on a per role basis in its own stage13:03
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_nova master: SUSE: Add missing QEMU drivers for block devices  https://review.openstack.org/57016813:04
*** dave-mccowan has quit IRC13:04
odyssey4mewe also have a bunch of notifiers which execute when packages change, so we'll have to implement something to switch up how that works13:04
odyssey4methen there's also the extra complexity of the orchestration of upgrades - you have to upgrade a service's software on all hosts before you do a final restart of the service, then execute db migrations13:05
*** kaiokmo has quit IRC13:06
*** kaiokmo has joined #openstack-ansible13:06
odyssey4mewe've been doing all this stuff since icehouse, and while it works well - it's slow and the organic growth has reached a point where things need to change... but to do that while still keeping everything working is a mammoth task, and I happen to be chipping away at it in my spare time13:07
*** armaan has quit IRC13:11
*** dave-mcc_ has joined #openstack-ansible13:11
parasitidto "orchestrate things"13:11
parasitidon "simple" way is to rely on etcd or consul13:12
parasitidand use watchers13:12
odyssey4meso basically, if you're using OSA, it'd help us move faster and do better things if you could help us with pretty much anything: docs maintenance, bug triage, patches to fix bugs, etc13:12
parasitidand declare the watchers as simple systemd services13:12
parasitidok13:12
parasitidwill try to13:12
openstackgerritmelissaml proposed openstack/openstack-ansible-nspawn_container_create master: fix a typo in main.yml  https://review.openstack.org/57018713:13
odyssey4meyeah, we've discussed using etcd/consul for some things - but if we make that a requirement, then our operators have to learn a new system... and our operator feedback has been not to require yet another system13:13
*** esberglu has joined #openstack-ansible13:13
odyssey4mewe've discussed some ideas around making something similar which can use etcd/consul, but not require it13:14
parasitidodyssey4me: that would be tough ;)13:14
odyssey4meit's better to have a good core base, then have opt-in things which make it better13:14
parasitidof course13:14
parasitidbut trying to build things that are somehow "reactive" to the environment13:15
odyssey4mewell, not that tough - we can use native ansible to maintain state using facts on the deploy host or facts on the targets, no other service required for that13:15
*** tux_ has joined #openstack-ansible13:15
parasitidis kinda hard without a central knowlegde base13:16
nattanonHello, Could i ask questions here.13:16
parasitidok ok13:16
odyssey4mewe do some of that already, and using consul/etcd is already possible for anything a deployer wants - any variables can use the etcd lookup, or whatever lookup13:16
tux_Good morning!13:16
tux_Folks any idea what is going on here https://bugs.launchpad.net/openstack-ansible/+bug/177277813:16
openstackLaunchpad bug 1772778 in openstack-ansible "skipping: no hosts matched" [Undecided,New]13:16
odyssey4metux_ most likely a stale fact cache13:17
odyssey4metry running: ansible -m setup all13:17
tux_where i should find that and fix ?13:17
odyssey4methen try again13:17
*** yolanda has quit IRC13:17
nattanonDo OSA have task like "systemctl daemon-reload"??13:17
tux_run on deployment node?13:17
odyssey4metux_ yes, from the /opt/openstack-ansible/playbooks directory13:18
tux_ok let me try13:18
odyssey4menattanon you don't need it - just execute: ansible -m shell -a "systemctl daemon-reload" <target_host_group>13:18
odyssey4menattanon but yes, every role that has a change of template fires a notifier which does a daemon reload13:19
odyssey4meit will execute automatically when the template changes13:19
nattanonodyssey4me Actually, i face through the problem about that command.13:19
MiougeIs there a good way to restrict SSH access to the control plane from a given subnet?13:19
*** yolanda has joined #openstack-ansible13:19
odyssey4menattanon rephrase?13:20
odyssey4meMiouge the best way is to make sure your controll pane is inaccessible from anything not part of the control plane at all - keep the networks isolated at the switch/router/whatever13:21
odyssey4mebut if you must, do some sort of host firewalling - OSA won't do that for you as the best way to do that is somewhat opinionated13:21
nattanonI'am using vadvisor for monitoring instance. After i re-deploy new config i got this "Requested operation is not valid: cgroup CPUACCT controller is not mounted" from libvirt.13:22
nattanonodyssey4me Then i trying to find step to reproduce that error then i found.13:24
odyssey4menattanon I've not seen that personally. I guess you may need to go back and look at the changes you made.13:24
Miougeodyssey4me: My problem is that external_lb_vip_address has SSH open. I already have edge firewalls, but VMs inside OSA can still access that SSH port. So I have to do host level firewall right?13:24
nattanonsystemctl daemon-reload --> service libvirtd restart  --> ERROR13:25
odyssey4meMiouge oh, that's odd - ah, that's probably due to the sshd listening on all addresses? You should have the host sshd only listen on the address you want it to, not all of them13:25
nattanonwithout daemon-reload libvirtd can be restart13:25
*** eumel8 has joined #openstack-ansible13:26
odyssey4menattanon sorry, I can't help13:26
nattanonperfectly13:26
nattanonOkay thank you for your time.13:26
*** nattanon has quit IRC13:27
Miougeodyssey4me: Good idea. I just need to listen on br-mgmt. Thanks a lot!13:29
jrosserhmm i was going to do a utility playbook to go round everything and create a report about all the listening ports13:31
odyssey4mejrosser good idea! perhaps add it to the healthcheck playbook13:32
*** throwsb1 has joined #openstack-ansible13:34
*** jwitko has joined #openstack-ansible13:38
*** jwitko has quit IRC13:40
*** jwitko has joined #openstack-ansible13:41
*** mma has quit IRC13:41
*** mma has joined #openstack-ansible13:41
parasitidodyssey4me: hitting another issue, this time on setup-infrastructure13:43
odyssey4meparasitid yeah?13:44
parasitidodyssey4me: in a first run13:45
*** tux_ has quit IRC13:45
parasitidthere seems to have been connectivy issue on one host13:45
parasitidsince then13:45
parasitidi get13:46
*** tux_ has joined #openstack-ansible13:46
*** mma has quit IRC13:46
parasitid[myopenstack_shared_infra_2_repo_container-141b600c]: FAILED! => {"changed": false, "failed": true, "msg": "file not found: /var/www/repo/o13:47
parasitidpenstackgit/requirements/upper-constraints.txt"}13:47
openstackgerritAndy McCrae proposed openstack/openstack-ansible-lxc_container_create master: Add Bionic testing  https://review.openstack.org/56703213:47
parasitidhere's a pastebin13:47
parasitidhttps://pastebin.com/v9jqp0jj13:48
odyssey4mepas13:48
odyssey4meparasitid GnuTLS recv error (-54): Error in the pull function.13:48
*** throwsb1 has quit IRC13:49
parasitidyes13:49
parasitidthat was the first run13:49
parasitidno i get the "file not found" error13:49
parasitidis there something a could taint?13:50
*** tux_ has quit IRC13:50
odyssey4mehmm, try adding -e repo_build_git_reclone=yes -e repo_build_wheel_rebuild=yes -e repo_build_venv_rebuild=yes13:50
odyssey4melooks like it thinks a stage completed, but it didn't, so you can force them all to happen again13:50
parasitidok13:51
odyssey4meyou can just run the repo-build.yml playbook this time13:51
parasitidthanks13:53
*** sxc731 has quit IRC13:55
*** pvradu has quit IRC13:58
*** pvradu has joined #openstack-ansible13:59
openstackgerritAndy McCrae proposed openstack/openstack-ansible-lxc_container_create master: Add Bionic testing  https://review.openstack.org/56703214:00
andymccr^ cloudnull i think that'd work - im a bit concerned about reordering of the dicts but https://github.com/ansible/ansible/pull/36644 should fix it for 2.6+ although we're not on that for a while.14:10
*** nurdie has joined #openstack-ansible14:10
*** tux_ has joined #openstack-ansible14:12
*** sxc731 has joined #openstack-ansible14:16
*** cjloader has joined #openstack-ansible14:18
guilhermesphello guys. Quick question: is it not totally clear for me in this doc https://docs.openstack.org/openstack-ansible/latest/user/source-overrides/index.html#overriding-other-upstream-projects-source-code14:25
guilhermespsession overriding upstream source code14:25
guilhermespdo I need to create a new file? ex: nova.yml to override the nova git repo?14:25
guilhermespinside ../defaults/repo_packages/14:26
odyssey4meguilhermesp add your overrides to /etc/openstack_deploy/user_variables.yml14:27
odyssey4memake sure you add the git_repo too (even with the same value if need be), otherwise it won't get picked up14:27
odyssey4methe note explains why14:29
*** throwsb1 has joined #openstack-ansible14:29
guilhermespgot it. About the repo, what I did was: I made kind of a "private-fork" because some of the nova source code modifications are not meant to be public yet.14:30
guilhermesp https://gist.github.com/DavideMontersino/810ebaa170a2aa2d2cad14:30
guilhermespI cloned the nova project, made my modifications and pushed to another remote, that is in our internal gitlab14:30
*** rpittau has joined #openstack-ansible14:31
odyssey4meok, as long as the repo container can git clone it, it'll work14:31
guilhermespawesome. At the first run, I'am facing this http://paste.openstack.org/show/721778/ in the task repo_build: wait for the venvs build to complete14:32
*** cshen has quit IRC14:32
guilhermespnot sure if is related to this, coz I didn't go deep into it14:32
*** cmart has joined #openstack-ansible14:33
guilhermespDon't know if you remember odyssey4me but I replicated my production environment to test first. That's why I'm still using an old tag (16.0.0)14:34
guilhermespTo document all the necessary steps to replicate de minor upgrade in my production cloud14:34
guilhermespbut just a quick reminder: we have source code overrides, and I want to control these overrides and do not make changes manually again when I upgrade my environment14:35
odyssey4meguilhermesp it might be, whatever nova sha you're using will have to have a corresponding requirements sha which has a jinja version which matches what nova needs14:35
openstackgerritMerged openstack/openstack-ansible-os_ceilometer master: zuul: Make openSUSE and CentOS voting jobs  https://review.openstack.org/56935414:35
odyssey4meand that may require all the other repositories to also have sha's changed to match14:36
guilhermespyou mean all the other service repositories or just the nova ones?14:37
odyssey4meyeah, it may be required if the requirements across them all don't match14:37
odyssey4methat jinja requirement - is that part of the nova changes you made?14:38
guilhermespno, it's not part of the nova changes14:38
odyssey4meif not, what's the base sha you used for nova before the changes?14:38
*** cshen has joined #openstack-ansible14:38
*** rpittau has quit IRC14:39
guilhermespstupid question: how do I verify the base sha of the git repo that I cloned?14:40
odyssey4mewell, I would guess that the customsations are patched on top of the base nova code - so look at the git log and find the last nova patch before the customisations14:41
guilhermespgot it. Gonna compare here14:41
guilhermesphuuum, in my repo container (/var/www/repo/openstackgit/nova) that last commit is: a98a52d85eb2c695568ac01d5ae7baf1bc20d9e3. The nova repo that I cloned before my changes is d78055df0e39b53fd481c53429ba77f8c1e1ab0914:43
guilhermespone of sep 6 2017 and other may 21 201814:43
guilhermesphahahah14:43
guilhermespbig gap14:44
odyssey4meguilhermesp argh, d78055df0e39b53fd481c53429ba77f8c1e1ab09 comes from nova master (Rocky)14:44
*** mma has joined #openstack-ansible14:44
odyssey4mewhat're you supposed to be deploying - Pike, or Queens?14:44
odyssey4mewhich branch are you deploying from for OSA?14:45
guilhermespah yeah yeah, I did some s**t14:45
odyssey4meyou should take the SHA pin for the OSA release you're working with, and base your patch on top of that SHA14:46
guilhermespI cloned and forgot to change the branch =/14:46
guilhermespmy mistake14:46
odyssey4methen every time you minor upgrade, take the new OSA sha and rebase your changes on top of it14:46
odyssey4medownstream forks are fun ;)14:46
guilhermespyes, gonna catch the opportunity to this14:47
guilhermesp:)14:47
guilhermespa new thing to learn14:47
evrardjp[m]good morning folks14:48
evrardjp[m]how are things ?14:48
guilhermespso, assuming that I will be in the correct branch, do I need to match the exactly sha as it is in my repo container?14:48
*** pvradu_ has joined #openstack-ansible14:48
evrardjp[m]everything good ?14:48
*** mma has quit IRC14:49
*** pvradu_ has quit IRC14:49
*** dcreno has joined #openstack-ansible14:51
*** cshen has quit IRC14:51
*** pvradu has quit IRC14:52
*** chyka has joined #openstack-ansible14:55
*** dcreno has quit IRC14:55
*** chyka has quit IRC15:00
*** markvoelker has quit IRC15:00
*** markvoelker has joined #openstack-ansible15:01
*** jproulx has joined #openstack-ansible15:01
*** cshen has joined #openstack-ansible15:02
*** markvoelker has quit IRC15:05
*** cshen has quit IRC15:06
*** cshen has joined #openstack-ansible15:07
openstackgerritJonathan Rosser proposed openstack/openstack-ansible master: Add utility playbook to report all listening ports  https://review.openstack.org/57021115:10
jrosserodyssey4me: feel free to improve that if you have any ideas ^^ - i fought it a bit trying to make a more machine readable output.....15:11
*** chyka has joined #openstack-ansible15:13
*** cshen has quit IRC15:15
tomtom001Hello, trying to run-upgrade from pike to queens but the scripts keeps dying at: nova cleanup complaining about containers that it destroyed,  I've verified that they're not there, cleaned up ansible facts, is there anything else I can do?15:17
*** chyka has quit IRC15:17
*** eumel8 has quit IRC15:20
*** DanyC has quit IRC15:21
*** lvdombrkr has quit IRC15:22
*** sxc731 has quit IRC15:23
*** dave-mcc_ has quit IRC15:23
*** jproulx has quit IRC15:24
*** openstacking_123 has joined #openstack-ansible15:25
openstackgerritJonathan Rosser proposed openstack/openstack-ansible master: Add utility playbook to report all listening ports  https://review.openstack.org/57021115:29
odyssey4meguilhermesp whatever you set as your git_branch override is set to is what the repo container will checkout; if the sha doesn't exist in that repo then it will try to fetch from the git_repo override is set to15:31
odyssey4metomtom001 perhaps those scripts could so with a bit of extra work to make them idempotent... I don't think they were made such as they're a one-off execution15:32
tux_odyssey4me: this didn't help me ansible -m setup all :(15:33
odyssey4mepatches are most certaonly welcome15:33
odyssey4mejrosser thanks - will star it for review later15:33
*** pcaruana|afk| has quit IRC15:33
tux_http://paste.openstack.org/show/721807/15:33
tux_odyssey4me: look at this15:33
odyssey4metux_ do you actually have repo containers at all?15:34
odyssey4meoh I see - you've limited it to the controller - don't do that15:34
tux_i have repo container but its has no data in /var/www/repo :(15:34
odyssey4meyou're telling it to run the repo-build on the controller, but that controller is not in the repo group15:34
*** dcreno has joined #openstack-ansible15:35
tux_what do you mean?15:35
odyssey4methe controller (host) is not in the repo_all group, only containers are15:35
tux_how do i find its in repo group or not?15:35
*** pvradu has joined #openstack-ansible15:35
odyssey4mejust run without the limit15:35
tux_it did install repo only on infra3 node (ostack-controller-03)15:36
tux_but infra1 and infra2 isn't in match list15:36
tux_odyssey4me: openstack-ansible repo-build.yml  doing this now15:36
*** cmart has quit IRC15:37
tux_odyssey4me: can you explain me what is repo group?15:37
odyssey4metunthe group 'repo_all' contains all the repo containers, and that group is what the playbook targets15:38
tux_where that defined ?15:38
odyssey4meif you use a limit, then it excludes all hosts in the inventory except those you limit to - and the host you limited your playbook to is not in that group, so there were no hosts15:38
odyssey4methe playbook is where the target hosts are defined, and the dynamic inventory makes that group - you can see the groups by using the inventory-manage.py script15:39
*** dcreno has quit IRC15:39
tux_believe me i have run 100 time setup-infra without any limit and it just targeting only  infra3 but not touching 1/215:39
*** pvradu has quit IRC15:40
tux_right now i am running without --limit and in output all i am seeing infra3  but nothing happening at infra1 and infra215:40
odyssey4metux_ then your inventory doesn't have the other infra nodes in the host group for the repo15:41
tux_Yes it does15:41
tux_i have check multiple time.. let me paste here again15:41
odyssey4methen I dunno, sorry - I'm trying to get some other work done so I can't help right now15:41
hwoarangodyssey4me: do you have any clue why the haproxy_server role doesn't log anything?15:42
odyssey4mehwoarang no idea?15:42
hwoarangok then i will investigate. i was hoping to be a known issue15:42
*** pvradu has joined #openstack-ansible15:42
tux_odyssey4me: openstack-ansible repo-build.yml just finished and again infra3 got updated but didn't update infra1 and infra2 they are untouched15:43
tux_odyssey4me:  here you go see my inventory json file http://paste.openstack.org/show/721808/15:44
odyssey4metux_ check your inventory, verify that all the hosts are in the right host group - also verify that your facts are fresh as the playbook uses them to organise the group into multi distro/arch groups for execution15:44
odyssey4megoing offline now to concentrate, bbl folks15:45
tux_come back :)15:45
tux_i need you15:45
jrossertux_: start from the beginning. check you have defined the repo hosts properly like this https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/openstack_user_config.yml.prod.example#L93-L10015:46
*** mkucia has quit IRC15:47
tux_jrosser: Yes i do have that block in my config file.15:47
jrosseri think more than once you have been given advice to use the inventory manage tool as well15:47
-tux_- # repository (apt cache, python packages, etc)15:47
-tux_- repo-infra_hosts:15:47
-tux_- ostack-controller-01:15:47
-tux_- ip: 192.168.100.115:47
-tux_- ostack-controller-02:15:48
-tux_- ip: 192.168.100.215:48
-tux_- ostack-controller-03:15:48
-tux_- ip: 192.168.100.315:48
jrossertux_: pastebin :/15:48
tux_hold on let me do that15:48
tux_jrosser: http://paste.openstack.org/show/721809/15:49
tux_take a look15:49
*** sxc731 has joined #openstack-ansible15:49
*** cmart has joined #openstack-ansible15:50
*** jproulx has joined #openstack-ansible15:51
jrossercheck that the groups are making sense as well /opt/openstack-ansible/scripts/inventory-manage.py -G15:51
*** gyee has joined #openstack-ansible15:51
tux_jrosser: they are in group check this out http://paste.openstack.org/show/721810/15:53
tux_does that looks good to you?15:54
*** dave-mccowan has joined #openstack-ansible15:54
*** rpittau has joined #openstack-ansible15:55
*** rpittau has quit IRC15:55
jrossertux_: well it all makes total sense15:56
jrosseryou're running repo build15:56
tux_Yes15:56
jrosserit targets the first repo server in the group15:56
jrosserwhich as inventory_manage says is infra315:56
jrosserand the code is here https://github.com/openstack/openstack-ansible/blob/master/playbooks/repo-build.yml#L29-L4315:56
openstackgerritAndy Smith proposed openstack/openstack-ansible-os_keystone master: Update to use oslo.messaging service for RPC and Notify  https://review.openstack.org/56202615:57
tux_what do you mean target first repo server ?15:58
*** dave-mccowan has quit IRC15:58
tux_jrosser: what about infra1/2 ??15:58
*** cshen has joined #openstack-ansible15:58
jrosserthe repo is built on one of the repo servers and synced onto the other two15:58
tux_oh!!15:59
tux_how that will get sync to other two?15:59
jrosserplease can you check that this patch is applied to your repo_server role and if not, fix up as needed https://github.com/openstack/openstack-ansible-repo_server/commit/14254375b1cb2623be0cce7a21ea5925cd19f0c516:00
tux_is this a patch?16:02
*** rpittau has joined #openstack-ansible16:02
jrosserif your actual issue is that the repo server is not being synchronised across the infra nodes, then that could be your issue16:02
jrosseryou'll need to investigate16:02
tux_jrosser: this is what my file saying, its very different http://paste.openstack.org/show/721811/16:03
jrosserunfortunatly i can't fix this for you myself, you'll need to debug16:04
tux_:(16:04
tux_problem is there is very less document for openstack-ansible stuff and i new in ansible so taking long time to get all pieces to understand16:05
*** dcreno has joined #openstack-ansible16:05
tux_can you tell me what trigger sync? so i can try manual sync and see16:05
tux_or i can open bug16:06
*** udesale has quit IRC16:07
*** erolg has quit IRC16:10
jrossertux_: i can only advise you read the playbooks, and then the roles if they are called16:10
jrosseryou will then end up finding how these things work, so for the repo sync you might end up somewhere like this https://github.com/openstack/openstack-ansible-repo_build/blob/master/tasks/repo_package_sync.yml16:10
openstackgerritAndy Smith proposed openstack/openstack-ansible-os_neutron master: Update to use oslo.messaging service for RPC and Notify  https://review.openstack.org/56375816:11
jrosserthen check out the relevant logs on your hosts to see why things have failed16:11
tux_jrosser: Ok reading it now16:11
tux_jrosser: also i am using CentOS7 so may be it could be a bug too16:12
*** rpittau has quit IRC16:12
jrosserremember to look at the code for the branch/tag you are using16:13
*** epalper has quit IRC16:13
tux_ok i am using 16.0.1016:14
tux_can i execute this specific task on repo container?16:15
tux_in worst case i will do manual sync between all repo container16:16
jamesdentonis there a reason why you're using 16.0.10 vs stable/pike?16:18
jamesdentonthere are some bugs fixed in stable (16.0.14)16:18
tux_i just pick random one for LAB testing :(16:19
tux_does 16.0.10 is not stable?16:19
jamesdentonyeah, that's probably not doing you favors at the moment.16:19
jamesdentoni ran into some issues yesterday (ubuntu)16:20
jamesdentonit's not that it's not stable, but things change outside the control of OSAand those fixes are wrapped into new tags16:20
jamesdentonyou might beat your head against the wall trying to get that version working when it's already fixed16:20
jrosserplus important bugfixes off master get backported16:20
tux_should i use https://github.com/openstack/openstack-ansible/tree/stable/pike16:21
jrosserhere are the instructions for a minor upgrade https://docs.openstack.org/openstack-ansible/pike/user/minor-upgrade.html16:21
jamesdentoni would, yes16:21
tux_do i need to start from scratch ?16:22
jamesdentonno16:22
tux_or just update my branch ?16:22
jamesdentonyou can check out that URL16:22
tux_ok let me do that16:22
*** dave-mccowan has joined #openstack-ansible16:22
jamesdentonit mentions 16.0.13, but i should choose stable over that, too16:23
tux_also do i need to re-run all my playback again or just resume from where i stuck at this point?16:23
jamesdentonrerun16:23
jamesdentonincluding bootstrap16:23
jrosserfollow the instructions on the url i gave you16:23
jrosserjust checkout stable/pike instead of 16.0.13 that it says16:24
*** cshen has quit IRC16:25
*** dave-mccowan has quit IRC16:27
*** dave-mccowan has joined #openstack-ansible16:29
*** cshen has joined #openstack-ansible16:29
*** pcaruana|afk| has joined #openstack-ansible16:29
*** dxiri has joined #openstack-ansible16:31
tux_running git clone https://git.openstack.org/openstack/openstack-ansible /opt/openstack-ansible16:32
tux_will do then git checkout stable/pike16:32
tux_it saying Cloning into '/opt/openstack-ansible'... but nothing happening... intesting..16:33
*** radek__ has quit IRC16:33
*** cshen has quit IRC16:34
*** radeks has quit IRC16:34
*** tacoboy has joined #openstack-ansible16:39
tux_done16:39
tux_now i need to copy /etc/openstack_deploy right ?16:42
*** dave-mccowan has quit IRC16:42
tux_or just copy roles and keep my all yml config as it is16:42
*** cshen has joined #openstack-ansible16:43
jrosserdo what the instuctions say :)16:43
*** DanyC has joined #openstack-ansible16:43
tux_ok! :)16:43
jrosserin particular if you mess up your current inventory you are in trouble16:43
*** dave-mccowan has joined #openstack-ansible16:43
jrosserso just follow the minor upgrade guide16:43
tux_i will take backup of inventory and overwite other stuff16:44
*** pvradu has quit IRC16:44
*** DanyC_ has joined #openstack-ansible16:44
*** chyka has joined #openstack-ansible16:44
*** chyka has quit IRC16:44
*** jproulx has quit IRC16:45
*** chyka has joined #openstack-ansible16:45
*** DanyC has quit IRC16:47
*** dave-mccowan has quit IRC16:48
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-haproxy_server master: [DNM] tasks: Disable rsyslog syncing for haproxy log files  https://review.openstack.org/57023116:48
*** ivve has joined #openstack-ansible16:48
*** rpittau has joined #openstack-ansible16:49
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible master: [DNM] - Testing Ceph AIO on openSUSE  https://review.openstack.org/50745116:49
*** chyka has quit IRC16:49
*** pcaruana|afk| has quit IRC16:51
*** jproulx has joined #openstack-ansible16:51
*** chyka_ has joined #openstack-ansible16:52
tux_re-running my setup-infra playbook now and fingers cross16:52
*** nurdie has quit IRC16:52
tux_I am not running setup-hosts.yml at this point16:52
*** nurdie has joined #openstack-ansible16:53
TahvokShouldn't this start the gate jobs?16:57
Tahvokhttps://review.openstack.org/#/c/548359/16:57
*** nurdie has quit IRC16:57
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-ops master: Strip underscores from journalbeat fields  https://review.openstack.org/56975116:58
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-ops master: Add filebeats setup for designate  https://review.openstack.org/57023416:58
*** chyka_ has quit IRC17:02
*** chyka has joined #openstack-ansible17:13
*** chyka has quit IRC17:18
*** rpittau has quit IRC17:20
throwsb1Hi, I am running into an haproxy issue with the internal IP.  This is a new deploy in a clustered environment using all of the same configs of the previous build.  All worked fine.  All servers are rebuilt except for deploy.17:22
*** mouchon has joined #openstack-ansible17:23
throwsb1The issue I am having is that HA looks healthy accoring to hatop and the repo_container is online.  As far as networking, I can ping and connect with out issues.17:23
throwsb1The kicker is on my 3 infra nodes, I see the internal IP listening, but there is no IP mapped to it if I run ip a.17:24
throwsb1This is Queens 17.05.  Is there something that I might be missing?17:24
*** lemouchon has joined #openstack-ansible17:25
*** mouchon has quit IRC17:26
throwsb1The deploy dies at connecting to the repo server to pull software, which it tried to connect to the internal LB IP on 8181.  I can curl direct to each of the repo servers with out a problem.17:26
*** rpittau has joined #openstack-ansible17:26
*** lbragstad has joined #openstack-ansible17:26
*** lemouchon has quit IRC17:26
throwsb1:$17:27
*** jproulx has quit IRC17:31
*** chyka has joined #openstack-ansible17:33
*** chyka has quit IRC17:35
*** chyka has joined #openstack-ansible17:37
*** dcreno has quit IRC17:38
*** cshen has quit IRC17:38
*** chyka has quit IRC17:41
*** nurdie has joined #openstack-ansible17:45
*** rpittau has quit IRC17:49
*** chyka has joined #openstack-ansible17:52
*** cshen has joined #openstack-ansible17:52
*** dcreno has joined #openstack-ansible17:53
*** lbragstad has quit IRC17:53
*** lbragstad has joined #openstack-ansible17:54
*** pcaruana has joined #openstack-ansible17:55
*** jproulx has joined #openstack-ansible17:55
*** hamzy has joined #openstack-ansible17:56
*** chyka has quit IRC17:56
*** dcreno has quit IRC17:57
*** eumel8 has joined #openstack-ansible17:58
tux_jamesdenton: I am still seeing same error even after stable/pike18:00
*** lbragstad has quit IRC18:02
tux_my repo directory is still not getting sync18:02
tux_ /var/www/repo/os-releases/16.0.14/ still empty18:02
tux_my problem is its not getting sync with infra318:04
tux_let me do manual sync and see what is going on18:04
tux_also opening BUG18:04
*** dave-mccowan has joined #openstack-ansible18:05
*** armaan has joined #openstack-ansible18:05
tux_https://bugs.launchpad.net/openstack-ansible/+bug/170005118:07
openstackLaunchpad bug 1700051 in openstack-ansible "lsyncd don't work in centos7" [High,Incomplete]18:07
tux_seems my issue18:07
tux_but i don't have that daemon installed18:08
tux_lsyncd18:08
tux_oh wait its dead on infra3 node18:09
*** poopcat has joined #openstack-ansible18:13
*** chyka has joined #openstack-ansible18:14
tux_now my repo getting sync18:20
tux_damn ittttt18:20
*** chyka has quit IRC18:22
jrossertux_: if you know what/why it is broken please comment of the bug or even batter make a patch :)18:24
*** chyka_ has joined #openstack-ansible18:24
jrosser*on the bug....18:24
tux_i did on that comments18:26
tux_after restart lsyncd it fixed and start syncing repo18:26
tux_I would love to see these kind of stuff on Openstack-ansible official documents like how small-small components are connected and how they works18:26
*** chyka_ has quit IRC18:28
throwsb1I was able to pinpoint my haproxy floating IP for internal issue to the keepalived.  I am getting an error stating that it can not find an IP to use and dies.18:30
throwsb1Still digging to see if it is a mis-config on my side or a bug,18:31
tux_throwsb1: i had ip duplication in past related Haproxy18:32
throwsb1tux_: what do you mean by a duplication?  What version of OSA are you running?18:33
tux_throwsb1: By mistake was i forgot to put IP in used_ip list and lxc container grab that IP and assigned to someone, later i found my haproxy was dead because IP was used by someone else18:34
*** hamzy has quit IRC18:35
throwsb1tux_: Ahhh, that is not happening for me, as I am doing a rebuild with same scripts.  I even fully removed haproxy on infra nodes and started again.18:36
*** chyka_ has joined #openstack-ansible18:37
*** nicolasbock has joined #openstack-ansible18:38
throwsb1here is the error I am seeing.  http://paste.openstack.org/show/721813/18:40
*** lbragstad has joined #openstack-ansible18:42
*** hamzy has joined #openstack-ansible18:44
*** dcreno has joined #openstack-ansible18:44
*** dave-mccowan has quit IRC18:46
*** rpittau has joined #openstack-ansible18:46
*** dcreno_ has joined #openstack-ansible18:48
*** dcreno has quit IRC18:48
tux_throwsb1:  track script haproxy_check_script not found ???18:51
tux_why there are missing files18:51
throwsb1don't know18:51
tux_that is the issue i think, keepalived isn't configured properly18:51
throwsb1do you think it is something in the user_vars?18:52
tux_post it here18:52
throwsb1which?18:53
tux_This is what i have in user_variables.yml file18:53
-tux_- haproxy_keepalived_external_vip_cidr: "{{external_lb_vip_address}}/32"18:53
-tux_- haproxy_keepalived_internal_vip_cidr: "{{internal_lb_vip_address}}/32"18:53
-tux_- haproxy_keepalived_external_interface: ens19218:53
-tux_- haproxy_keepalived_internal_interface: br-mgmt18:53
*** markvoelker has joined #openstack-ansible18:54
tux_run openstack-ansible haproxy-install.yml  playbook and see18:54
throwsb1haproxy_keepalived_external_vip_cidr: "10.9.198.32/24"18:54
throwsb1haproxy_keepalived_internal_vip_cidr: "172.29.236.5/22"18:54
throwsb1haproxy_keepalived_external_interface: br-vlan18:54
throwsb1haproxy_keepalived_internal_interface: br-mgmt18:54
*** lbragstad has quit IRC18:54
throwsb1used this config before with out issues18:55
tux_did you run openstack-ansible haproxy-install.yml18:55
throwsb1no I did a full install since I was rebuilding18:55
throwsb1this is a fresh install.  only diff is I am moving from lvm to ceph18:56
throwsb1I would expect to have issues with ceph since it is new.18:56
tux_all i can say something is missing in keepalived or misconfigured18:57
throwsb1yea.18:57
tux_throwsb1: check /etc/keepalived/keepalived.conf18:58
throwsb1tux_: I am digging in now.  I am new to haproxy and using it in a clustered config.  working backwards to see what I am missing.18:59
*** markvoelker has quit IRC19:00
throwsb1tux_: do you know where the track scripts live?  in the config they list haproxy_check_script and pingable_check_script19:00
tux_its defined in /etc/keepalived/keepalived.conf file19:01
tux_vrrp_script haproxy_check_script19:01
throwsb1yup just saw that.  lol19:02
tux_its resources and it has bunch of attributes19:02
tux_it will just doing ping test to make sure stuffs are OK etc..19:02
tux_you will figure out i am sure...19:03
throwsb1yup, going through now19:03
*** lbragstad has joined #openstack-ansible19:03
*** chyka has joined #openstack-ansible19:07
*** nicolasbock has quit IRC19:07
jrosserthrowsb1: in my config the *_vip_cidr variables are both /3219:09
*** chyka_ has quit IRC19:10
throwsb1jrosser: I tried that in my previous install and it failed.  Setting the subnet to match the IP worked.19:10
*** sxc731 has quit IRC19:11
throwsb1jrosser: I can try it.19:11
tux_jrosser: did you ever seen this issue ? http://paste.openstack.org/show/721815/19:13
tux_now i am stuck with mysql cluster :(19:13
tux_FAILED! => {"attempts": 1, "changed": false, "cmd": "/usr/bin/mysqladmin  extended-status19:13
*** gkadam has quit IRC19:13
throwsb1jrosser: did not work.  :(19:15
*** dcreno_ has quit IRC19:20
*** dcreno has joined #openstack-ansible19:20
*** mbuil has quit IRC19:22
*** chyka has quit IRC19:22
*** lbragstad has quit IRC19:22
*** dcreno has quit IRC19:24
*** dcreno has joined #openstack-ansible19:25
*** jproulx has quit IRC19:27
*** radeks has joined #openstack-ansible19:27
*** chyka has joined #openstack-ansible19:28
tux_[Note] WSREP: (88b2d9f1, 'tcp://0.0.0.0:4567') connection to peer 88b2d9f1 with addr tcp://192.168.100.246:4567 timed out, no messages seen in PT3S19:29
*** radeks_ has joined #openstack-ansible19:29
*** hamzy has quit IRC19:30
*** dcreno has quit IRC19:30
*** cshen has quit IRC19:31
*** sxc731 has joined #openstack-ansible19:32
*** radeks has quit IRC19:33
*** chyka has quit IRC19:33
*** cshen has joined #openstack-ansible19:35
tux_Do i need to run "galera_new_cluster" manually ?19:37
*** rpittau has quit IRC19:37
*** chyka has joined #openstack-ansible19:38
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-os_designate master: Create rndc key.  https://review.openstack.org/57025419:39
*** cshen has quit IRC19:41
*** eumel8 has quit IRC19:41
*** chyka has quit IRC19:43
*** armaan has quit IRC19:45
*** cshen has joined #openstack-ansible19:49
tux_now i stuck at galera_container :(19:51
tux_cluster is not coming up and playbook is failing19:51
tux_seems WSREP check failed19:52
*** chyka has joined #openstack-ansible19:53
*** cshen has quit IRC19:53
tux_why this is empty in /etc/mysql/conf.d/server.cnf file  #wsrep_cluster_address=19:54
*** chyka has quit IRC19:57
tux_| wsrep_ready                                                  | OFF20:02
tux_why its off20:03
*** chyka has joined #openstack-ansible20:04
*** bauruine has quit IRC20:07
*** bauruine has joined #openstack-ansible20:14
*** jproulx has joined #openstack-ansible20:19
*** cshen has joined #openstack-ansible20:20
*** hamzy has joined #openstack-ansible20:22
*** sxc731 has quit IRC20:22
*** mma has joined #openstack-ansible20:22
*** mma has quit IRC20:23
*** mma has joined #openstack-ansible20:24
*** chyka_ has joined #openstack-ansible20:25
*** tacoboy has quit IRC20:26
*** mma has quit IRC20:28
*** chyka has quit IRC20:28
*** pcaruana has quit IRC20:28
*** openstacking_123 has quit IRC20:32
*** brad[] has quit IRC20:35
*** brad[] has joined #openstack-ansible20:37
*** cshen has quit IRC20:44
*** cshen has joined #openstack-ansible20:44
*** tux__ has joined #openstack-ansible20:45
*** chyka_ has quit IRC20:45
*** openstacking_123 has joined #openstack-ansible20:46
*** dcreno has joined #openstack-ansible20:47
*** dcreno has quit IRC20:48
*** dcreno has joined #openstack-ansible20:48
*** tux_ has quit IRC20:48
*** eumel8 has joined #openstack-ansible20:49
*** lbragstad has joined #openstack-ansible20:52
*** tux_ has joined #openstack-ansible20:54
*** chyka has joined #openstack-ansible20:55
tux_I have destroy container and re-create fixed mysql cluster issue20:55
*** tux__ has quit IRC20:56
*** chyka has quit IRC20:57
*** chyka has joined #openstack-ansible20:57
*** chyka_ has joined #openstack-ansible20:59
*** radeks_ has quit IRC21:00
*** chyka has quit IRC21:02
*** dcreno has quit IRC21:04
*** dcreno has joined #openstack-ansible21:10
*** nurdie has quit IRC21:10
*** tux__ has joined #openstack-ansible21:11
*** nurdie has joined #openstack-ansible21:11
*** dcreno has quit IRC21:14
*** tux__ has quit IRC21:14
*** tux_ has quit IRC21:14
*** tux_ has joined #openstack-ansible21:14
*** esberglu has quit IRC21:15
*** rpittau has joined #openstack-ansible21:18
*** tux_ has quit IRC21:18
*** tux_ has joined #openstack-ansible21:18
*** throwsb1 has quit IRC21:20
*** dcreno has joined #openstack-ansible21:25
*** tux__ has joined #openstack-ansible21:25
*** jamesdenton has quit IRC21:26
*** dcreno has quit IRC21:26
*** dcreno has joined #openstack-ansible21:27
*** tux_ has quit IRC21:28
*** chyka has joined #openstack-ansible21:29
*** chyka has quit IRC21:31
*** jproulx has quit IRC21:31
openstackgerritAndy Smith proposed openstack/openstack-ansible master: ([WIP] Add qdrouterd role for messaging backend deployment  https://review.openstack.org/57026121:32
*** chyka_ has quit IRC21:33
*** chyka has joined #openstack-ansible21:34
*** mma has joined #openstack-ansible21:35
*** lbragstad has quit IRC21:37
*** eumel8 has quit IRC21:37
*** markvoelker has joined #openstack-ansible21:38
*** cmart has quit IRC21:38
*** lbragstad has joined #openstack-ansible21:38
*** jwitko_ has joined #openstack-ansible21:38
*** tux_ has joined #openstack-ansible21:39
*** jproulx has joined #openstack-ansible21:41
*** jwitko has quit IRC21:41
*** tux__ has quit IRC21:42
*** eumel8 has joined #openstack-ansible21:43
*** jwitko_ has quit IRC21:43
*** nurdie has quit IRC21:45
*** nurdie has joined #openstack-ansible21:45
*** mma has quit IRC21:46
*** dave-mccowan has joined #openstack-ansible21:50
*** nurdie has quit IRC21:50
*** cjloader_ has joined #openstack-ansible21:57
*** cjloader has quit IRC21:57
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_horizon stable/queens: Download and Install neutron-fwaas-dashboard  https://review.openstack.org/57026622:00
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_horizon stable/pike: Download and Install neutron-fwaas-dashboard  https://review.openstack.org/57026722:01
*** cjloader_ has quit IRC22:01
*** tux__ has joined #openstack-ansible22:02
*** chyka_ has joined #openstack-ansible22:02
*** openstacking_123 has quit IRC22:03
*** tux_ has quit IRC22:06
*** chyka has quit IRC22:06
*** tux_ has joined #openstack-ansible22:06
*** tux___ has joined #openstack-ansible22:09
*** tux__ has quit IRC22:09
*** markvoelker_ has joined #openstack-ansible22:10
*** jwitko has joined #openstack-ansible22:11
*** markvoelker has quit IRC22:11
*** tux_ has quit IRC22:13
*** tux___ has quit IRC22:18
*** jproulx has quit IRC22:19
*** markvoelker_ has quit IRC22:21
*** dave-mccowan has quit IRC22:22
*** hamzy has quit IRC22:25
*** chyka_ has quit IRC22:25
*** DanyC_ has quit IRC22:30
*** DanyC has joined #openstack-ansible22:30
*** eumel8 has quit IRC22:32
*** jproulx has joined #openstack-ansible22:32
*** lbragstad has quit IRC22:33
*** DanyC has quit IRC22:34
*** nurdie has joined #openstack-ansible22:41
*** chyka has joined #openstack-ansible22:42
*** chyka has quit IRC22:47
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-lxc_container_create master: Add Bionic testing  https://review.openstack.org/56703222:52
*** eumel8 has joined #openstack-ansible22:52
*** dcreno has quit IRC22:59
*** dcreno has joined #openstack-ansible23:03
*** eumel8 has quit IRC23:06
*** dcreno has quit IRC23:07
*** dcreno has joined #openstack-ansible23:09
*** cshen has quit IRC23:09
*** parasitid has quit IRC23:09
*** cshen has joined #openstack-ansible23:10
*** hamzy has joined #openstack-ansible23:10
*** cshen has quit IRC23:16
*** dcreno has quit IRC23:16
*** jproulx has quit IRC23:18
*** cshen has joined #openstack-ansible23:27
*** nurdie has quit IRC23:27
*** nurdie has joined #openstack-ansible23:28
*** parasitid has joined #openstack-ansible23:28
*** rpittau has quit IRC23:31
*** tux_ has joined #openstack-ansible23:32
*** nurdie has quit IRC23:32
*** nurdie has joined #openstack-ansible23:34
*** rpittau has joined #openstack-ansible23:35
*** dcreno has joined #openstack-ansible23:36
*** tux_ has quit IRC23:36
*** dcreno_ has joined #openstack-ansible23:40
*** dcreno has quit IRC23:40
*** cshen has quit IRC23:41
*** pmannidi has joined #openstack-ansible23:41
*** lbragstad has joined #openstack-ansible23:42
*** mma has joined #openstack-ansible23:47
*** tux_ has joined #openstack-ansible23:49
*** cshen has joined #openstack-ansible23:51
*** mma has quit IRC23:52
*** tux_ has quit IRC23:55

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!