Monday, 2018-09-10

*** hamzy has quit IRC00:43
*** hamzy has joined #openstack-ansible00:44
*** spatel has joined #openstack-ansible00:59
*** nicolasbock has quit IRC01:35
*** gregoryo has joined #openstack-ansible01:52
openstackgerritAgate Wang proposed openstack/openstack-ansible-os_aodh master: [Trivial Fix] Replace Chinese punctuation with English punctuation  https://review.openstack.org/60114002:14
openstackgerritAgate Wang proposed openstack/openstack-ansible-os_ceilometer master: [Trivial Fix] Replace Chinese punctuation with English punctuation  https://review.openstack.org/60114602:46
openstackgerritAgate Wang proposed openstack/openstack-ansible-os_cinder master: [Trivial Fix] Replace Chinese punctuation with English punctuation  https://review.openstack.org/60114702:50
*** spatel has quit IRC03:13
*** spatel has joined #openstack-ansible03:13
*** spatel has quit IRC03:13
*** dave-mccowan has quit IRC03:33
*** udesale has joined #openstack-ansible03:58
*** johnny101 has quit IRC04:00
*** Bhujay has joined #openstack-ansible04:01
*** Bhujay has quit IRC04:02
*** johnny101 has joined #openstack-ansible04:02
*** Bhujay has joined #openstack-ansible04:02
*** radeks has joined #openstack-ansible04:36
*** ethfci has joined #openstack-ansible04:40
*** ethfci has quit IRC04:40
*** radeks has quit IRC04:52
*** ethfci has joined #openstack-ansible05:07
*** shyamb has joined #openstack-ansible05:12
*** shyamb has quit IRC05:17
*** liangqi has quit IRC05:21
*** threestrands has joined #openstack-ansible05:45
*** threestrands has quit IRC05:45
*** threestrands has joined #openstack-ansible05:45
*** shyamb has joined #openstack-ansible05:56
*** shyamb has quit IRC06:07
*** shyamb has joined #openstack-ansible06:08
*** lvdombrkr has joined #openstack-ansible06:20
*** lvdombrkr89 has joined #openstack-ansible06:23
*** lvdombrkr has quit IRC06:26
*** rgogunskiy has joined #openstack-ansible06:31
*** shyamb has quit IRC06:34
*** shyamb has joined #openstack-ansible06:34
*** chkumar|out is now known as chkumar|ruck06:40
*** ppetit has joined #openstack-ansible06:52
*** jbadiapa has joined #openstack-ansible06:52
*** Bhujay has quit IRC06:52
*** ppetit has quit IRC07:02
*** olivierb has joined #openstack-ansible07:03
*** ppetit has joined #openstack-ansible07:19
*** Pradiprwt has joined #openstack-ansible07:23
PradiprwtHi Everyone, I am getting this issue while running neutron palybook..I have enabled OVS, http://paste.openstack.org/show/729747/07:24
*** threestrands has quit IRC07:25
*** pcaruana has joined #openstack-ansible07:31
*** Bhujay has joined #openstack-ansible07:36
*** mschuppert has joined #openstack-ansible07:42
*** hamzaachi has joined #openstack-ansible07:43
*** shyamb has quit IRC07:48
*** Bhujay has quit IRC07:50
*** Bhujay has joined #openstack-ansible07:51
*** Bhujay has quit IRC07:52
*** Bhujay has joined #openstack-ansible07:52
*** lvdombrkr89 has quit IRC07:52
*** lvdombrkr has joined #openstack-ansible07:53
*** mbuil_ has joined #openstack-ansible07:57
*** mbuil is now known as mbuil_07:58
*** DanyC has joined #openstack-ansible08:14
*** electrofelix has joined #openstack-ansible08:54
*** lvdombrkr89 has joined #openstack-ansible08:57
*** lvdombrkr has quit IRC09:00
openstackgerritMerged openstack/openstack-ansible-os_blazar stable/rocky: Remove blazar_requirements_git_* vars  https://review.openstack.org/60107009:03
*** gregoryo has quit IRC09:06
*** shyamb has joined #openstack-ansible09:09
*** noonedeadpunk has joined #openstack-ansible09:14
*** radeks has joined #openstack-ansible09:28
*** radeks_ has joined #openstack-ansible09:29
*** radeks has quit IRC09:33
openstackgerritMerged openstack/openstack-ansible-os_keystone master: Set default ssl protocols according to used webserver  https://review.openstack.org/56774609:34
*** radeks__ has joined #openstack-ansible09:37
*** radeks_ has quit IRC09:39
*** radeks_ has joined #openstack-ansible09:41
*** radeks__ has quit IRC09:44
*** suggestable has joined #openstack-ansible10:03
suggestableGood morning10:04
noonedeadpunkmorning10:04
suggestableIs anyone able to help with Swift not deploying on stable/queens?10:04
noonedeadpunkunfortunatelly, I have no experience with swift, and all cores most likely are on PTG10:05
suggestablePTG?10:05
noonedeadpunkhttps://www.openstack.org/ptg/10:05
suggestableAhh. OK.10:06
*** hamzaachi_ has joined #openstack-ansible10:14
*** hamzaachi has quit IRC10:16
andymccrsuggestable, what's the issue?10:19
suggestableHi andymccr. When the playbook reaches os_swift: Copy swift config, it fails.10:20
andymccrsuggestable, do you have a log available?10:20
suggestableError from the playbook run is: "FAILED! => {"failed": true, "msg": "'None' has no attribute 'index'"}" for every occurrence.10:20
noonedeadpunkwith what error? Can you paste it?10:20
noonedeadpunklooks like problem with some veriable, which is undefined...10:21
noonedeadpunk*variable10:21
andymccrsuggestable, which task is it that fails in the run? it should say like [os_swift] Run this task - at the top of the bit where it fails10:22
suggestableOK. Is there a doc somewhere that explains exactly what it's looking for in the swift.yml file?10:22
andymccror if possible, paste a larger portion of the log to like paste.openstack.org or something10:22
suggestableTASK [os_swift : Copy swift config]10:23
andymccrsuggestable, can you paste your swift.yml to paste.openstack.org or a similar paste site?10:24
suggestableandymccr: sure. Give me a moment.10:26
andymccrsuggestable, based on the template files being copied it has to be the storage policies configuraiton - probably just a typo/format issue. we can confirm though. im trying to find a doc on the swift bits for reference10:28
suggestablehttp://paste.openstack.org/show/D9TnM23EfVBRmu8Iw1wm/10:29
andymccrahh yeah10:29
andymccrjust indent them slightly10:29
andymccrhttps://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/conf.d/swift.yml.example#L227-L24410:30
andymccrso after policy: add 2 spaces for all the other settings.10:30
*** shyamb has quit IRC10:30
suggestableArgh! Why is Ansible so fussy with indentation?10:30
andymccryaml - yeah :( but its like a list (the -'s) and then an item in the list which has attributes10:31
noonedeadpunkis more about yaml actually) the same thing with python, to be honest10:31
andymccrthat specifically is probably not the optimum design but changing it at this point would make all existing deploys break10:31
andymccr(so probably my bad for doing it that way!)10:32
*** dave-mccowan has joined #openstack-ansible10:32
andymccrsuggestable, but yeah give that a go hopefully it'll work out ok this time! tl;dr for how i debugged it - the error says its missing an atribute called index in one of 3 possible template files, so i checked the template files and fortunately only one bit has a .index which is the storage_policies.10:33
suggestableandymccr, phew! OK. I'm trying to make that change now and then we will see if it works.10:34
suggestableThank you!10:34
andymccrno problem!10:35
*** Bhujay has quit IRC10:45
*** jamesdenton has quit IRC10:52
*** hwoarang has quit IRC11:00
*** hwoarang has joined #openstack-ansible11:03
*** FuzzyFerric has joined #openstack-ansible11:06
*** shyamb has joined #openstack-ansible11:18
*** hwoarang has quit IRC11:21
*** hwoarang has joined #openstack-ansible11:24
*** udesale has quit IRC11:26
*** radeks_ has quit IRC11:30
kaiokmomornings11:32
noonedeadpunkmorning kaiokmo11:33
smoke87morning11:35
*** shyamb has quit IRC11:36
mnasero/11:38
suggestableandymccr, Thank you! That indentation change has allowed Swift to build successfully!11:42
suggestableThe next issue I'm facing is:11:42
suggestableTASK [os_tempest : Gather variables for each operating system]11:42
suggestablefatal: [os-controller3_utility_container-267379bf]: FAILED! => {"failed": true, "msg": "No file was found when using with_first_found. Use the 'skip: true' option to allow this task to be skipped if no files are found"}11:42
andymccrsuggestable, hmm thats odd - that task will find the correct vars file in os_tempst/vars/ that matches your distro - what distro are you using?11:43
noonedeadpunkhave you ran lxc_destroy of the container before?11:44
suggestableUbuntu 16.0411:44
*** shyamb has joined #openstack-ansible11:44
suggestablenoonedeadpunk, I've run lxc_destroy multiple times in the past when deployments have failed.11:44
smoke87suggestable. Hey I got this error too one time11:45
noonedeadpunkhave you added lxc_hosts while containers creation to the --limit list of the hosts?11:45
smoke87Check nyloc messages in irc log: http://eavesdrop.openstack.org/irclogs/%23openstack-ansible/%23openstack-ansible.2017-05-09.log.html11:45
suggestablenoonedeadpunk, I haven't used --limit.11:46
suggestableI'm trying to complete my first osa deployment.11:46
*** radeks_ has joined #openstack-ansible11:47
*** lvdombrkr89 has quit IRC11:48
andymccrsuggestable, try running the os-tempest-install.yml and see if that does the trick too. it looks like perhaps the facts arent generated proprely there11:48
suggestableWill do.11:48
*** lvdombrkr has joined #openstack-ansible11:48
smoke87hey guys, Do you know why when create orchestration's(heat) containers.. the container have no interfaces?11:49
smoke87*containers11:49
admin0smoke87, you have to define it in the yml11:51
admin0sorry ..11:51
*** ansmith has quit IRC11:51
smoke87In the openstack_user_config.yml I added:11:52
smoke87# heat11:52
smoke87orchestration_hosts: *infrastructure_hosts11:52
*** lvdombrkr89 has joined #openstack-ansible11:52
*** lvdombrkr has quit IRC11:52
*** jamesdenton has joined #openstack-ansible11:52
smoke87When I run the lxc-containers-create.yml playbook.. the containers are created with no interfaces just lo11:53
smoke87so it fail in "Execute first script" step11:53
*** ppetit has quit IRC11:54
suggestableandymccr, It seems that running the tempest playbook manually has helped, but now it's failing to upload the Cirros image to Glance.11:54
suggestableGlance on this deployment is configured to be backed by Cinder, which is using an EMC Unity SAN as its backend.11:55
andymccrsuggestable, hmm ok that sounds more legitimate as an issue.11:55
admin0smoke87,  .. lxc-attach and then ip link to see if its defined or not11:56
andymccris the cinder backend working properly? i've not really played around with that before11:56
suggestableandymccr, Not entirely sure how to verify that. Is it straightforward?11:56
andymccrsuggestable, you could try an openstack volume create or similar command from the utility host (remember to source /root/openrc).11:57
admin0suggestable,  config where you are using glance -> cinder backend ?11:57
andymccri imagine if glance cant create images on the backend then tempest will fail to upload images :)11:57
suggestableadmin0, in user_variables.yml: glance_default_store: cinder11:59
admin0and when you upload images, what does the glance logs say ?11:59
*** lvdombrkr has joined #openstack-ansible11:59
admin0on why they fail11:59
*** lvdombrkr89 has quit IRC12:01
suggestableadmin0, Do you have any pointers for where to actually find the logs? This is my first time using LXC containers.12:02
*** ppetit has joined #openstack-ansible12:03
smoke87root@openstack-01_heat_api_container-5a45e124:/# ip link12:04
smoke871: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 112:04
smoke87    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:0012:04
smoke87suggestable the log path is: /openstack/logs12:05
smoke87sorry  /openstack/log12:05
admin0when you upload a glane image, you get a id ..  copy that id .. login to your rsyslog container .. cd /var/log/log-storage .. then grep -ri uuid  *12:06
suggestableIf it helps, these are the two errors I received while running the playbook:12:08
suggestablefailed: [os-controller3_utility_container-267379bf] (item={u'url': u'http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img', u'checksum': u'sha256:e137062a4dfbb4c225971b67781bc52183d14517170e16a3841d16f962ae7470', u'name': u'cirros', u'format': u'qcow2'}) => {"attempts": 5, "changed": false, "extra_data": null, "failed": true, "item": {"checksum": "sha256:e137062a4dfbb4c225971b67781bc52112:08
suggestable83d14517170e16a3841d16f962ae7470", "format": "qcow2", "name": "cirros", "url": "http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img"}, "msg": "(500) Server Error for url: http://10.10.16.2:9292/v2/images/95eae66f-0556-42bb-865a-c10d2eaf37f2/file Internal Server Error"}12:08
suggestableand12:08
suggestablefailed: [os-controller3_utility_container-267379bf] (item={u'url': u'http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img', u'checksum': u'sha256:e137062a4dfbb4c225971b67781bc52183d14517170e16a3841d16f962ae7470', u'name': u'cirros', u'format': u'qcow2'}) => {"attempts": 5, "changed": false, "extra_data": null, "failed": true, "item": {"checksum": "sha256:e137062a4dfbb4c225971b67781bc52112:08
suggestable83d14517170e16a3841d16f962ae7470", "format": "qcow2", "name": "cirros", "url": "http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img"}, "msg": "Image creation failed: Unable to establish connection to http://10.10.16.2:9292/v2/images/78dfc459-a649-4fc4-9a09-1360ef92306c/file: ('Connection aborted.', error(32, 'Broken pipe'))"}12:08
admin0don't paste here .. use pastebin12:09
admin0check if you can create volumes from horizon12:12
admin0suggestable, lxc has nothing to do with  your errors ..  it is just used to keep the apis more managable12:13
*** lvdombrkr89 has joined #openstack-ansible12:16
smoke87admin0, the output of ip link: http://paste.openstack.org/show/729760/12:16
smoke87Thanks for your help!12:16
*** lvdombrkr has quit IRC12:17
suggestableadmin0, cinder-volume is showing as down on all three storage nodes.12:18
*** hamzaachi_ has quit IRC12:21
*** nicolasbock has joined #openstack-ansible12:26
admin0suggestable, you need to fix that first12:26
admin0get cinder up, and make sure you can create volumes without error5s12:27
admin0errors ..12:27
admin0smoke87, paste your full config somewhere12:28
admin0is this for all containers or just heat12:28
admin0suggestable, and if you have issues or get overwhelmed with logs,  shut down 2x  cinder containers .. keep only one running .. and then  from the log container tail -f *cinder*/*12:31
admin0so see logs12:31
*** shyamb has quit IRC12:34
*** olivierb has quit IRC12:46
*** olivierb has joined #openstack-ansible12:47
openstackgerritValentin Boucher proposed openstack/openstack-ansible-os_nova master: Compute: Enable vGPU in Nova if exist on the host  https://review.openstack.org/59718312:47
smoke87admin0, just heat12:54
smoke87I'm adding heat to the infrastructure12:55
*** ansmith has joined #openstack-ansible12:59
mgariepygood morning everyone13:00
smoke87morning13:06
*** nicolasbock has quit IRC13:06
*** radeks_ has quit IRC13:12
*** radeks_ has joined #openstack-ansible13:14
*** spatel has joined #openstack-ansible13:17
*** spatel_ has joined #openstack-ansible13:18
*** hamzaachi_ has joined #openstack-ansible13:19
*** spatel has quit IRC13:22
*** rgogunskiy has quit IRC13:34
*** goldenfri has joined #openstack-ansible13:37
*** canori01 has joined #openstack-ansible13:41
*** hamzaachi_ has quit IRC13:44
*** hamzaachi has joined #openstack-ansible13:45
*** spatel_ has quit IRC13:46
olivierbGood morning everyone. I have a set of machines which are working perfectly fine until end of OSA queens deployment where some (but not all) of them do not answer to commands like ansible MACH1 -i inventory -m setup13:48
olivierbit appears that only the setup module is impacted (tried ping, shell, command, ...) as I can get other commands to succeed13:48
olivierbany clue on why this happens and how to fix/investigate this ?13:48
*** goldenfri has quit IRC13:48
olivierbplease also note that I also still have administrative ssh access to the machines13:48
*** goldenfri has joined #openstack-ansible13:53
*** markvoelker has joined #openstack-ansible13:54
*** goldenfri has quit IRC13:57
*** markvoelker has quit IRC13:59
*** spatel has joined #openstack-ansible14:02
*** markvoelker has joined #openstack-ansible14:06
*** spatel has quit IRC14:12
*** markvoelker has quit IRC14:21
olivierb   in fact digging a little bit more into this, seems like it finishes to return (after 1 mn or more) which is unusual and number of tap devices on infra nodes keeps growing constanty: ~ 10 new itfs per minute14:22
*** markvoelker has joined #openstack-ansible14:23
mgariepyolivierb, it is a new deployment or an upgrade?14:27
olivierbbrand new deployment14:27
*** Leo_m has joined #openstack-ansible14:29
*** goldenfri has joined #openstack-ansible14:32
*** radeks_ has quit IRC14:40
*** skiedude has joined #openstack-ansible14:42
ppetitHi there!14:42
mgariepyolivierb, do you know what is spawning the interface?14:43
olivierbcurrently trying to dig into this but suspecting one the the centos network script run post veth12 for br-vlan14:45
*** hamzaachi_ has joined #openstack-ansible14:49
*** hamzaachi has quit IRC14:50
*** lvdombrkr89 has quit IRC14:54
*** priteau has joined #openstack-ansible14:54
ppetitSomething strange his happening in my OSA deployment... Several character device files (including /dev/tty[1-4], /dev/ptmx) are modified as regular files or symlinnks to lxc->tty  when I run setup-openstack.yaml.14:54
*** markvoelker has quit IRC14:55
*** markvoelker has joined #openstack-ansible14:55
ppetit I couldn't catch the exact task when this happening but it seems it's around TASK [Extra lxc config]14:56
ppetitHave guys already seen that behaviour?14:57
*** markvoelker has quit IRC14:58
*** radeks_ has joined #openstack-ansible14:58
ppetitFor example, lrwxrwxrwx 1 root root        8 Sep 10 14:45 /dev/tty1 -> lxc/tty114:59
*** radeks__ has joined #openstack-ansible15:00
*** macza has joined #openstack-ansible15:00
*** markvoelker has joined #openstack-ansible15:00
*** lbragstad has joined #openstack-ansible15:00
*** radeks_ has quit IRC15:03
*** macza has quit IRC15:04
openstackgerritMerged openstack/openstack-ansible-os_masakari master: Clean up role tests  https://review.openstack.org/60078415:05
*** macza has joined #openstack-ansible15:05
*** hwoarang has quit IRC15:09
*** hwoarang has joined #openstack-ansible15:12
*** arxcruz has joined #openstack-ansible15:34
*** radeks__ has quit IRC15:36
*** mschuppert is now known as mschuppert|afk15:38
*** radeks__ has joined #openstack-ansible15:43
*** gyee has joined #openstack-ansible15:44
mnaserhttps://pypi.org/project/ansible-merge-vars/15:46
*** DanyC has quit IRC15:53
cloudnullgood mornings !16:01
devxbonjour16:08
*** hamzaachi_ has quit IRC16:14
*** markvoelker_ has joined #openstack-ansible16:14
*** FuzzyFerric has left #openstack-ansible16:15
*** markvoel_ has joined #openstack-ansible16:17
d34dh0r53moin16:17
*** markvoelker has quit IRC16:18
jrossero/16:18
*** spatel has joined #openstack-ansible16:19
*** markvoelker_ has quit IRC16:20
*** suggestable has quit IRC16:21
jrossercloudnull: i tried the wazuh stuff....... can't quite make it work yet16:23
spateljrosser: i think cloudnull is out for openstackPTG right?16:24
*** olivierb has quit IRC16:24
jrosserspatel: yes16:24
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-plugins master: Add retry to execution commands  https://review.openstack.org/60130116:24
cloudnullspatel jrosser o/16:24
spateloh boy!!!16:24
cloudnullyup here a the PTG, sitting with d34dh0r53 devx mnaser odyssey4me logan- :)16:25
spateli was watching your twitter feed and thought you won't be here for sometime :)16:25
spatelyou guys are lucky!!!16:25
cloudnullyou'll have to make it to the next one spatel16:25
spatelanyway have fun... i have few question for cloudnull but i think you guys anyway busy!! will talk when you back..16:25
guilhermespany chances to provide an online conference to the PTG discussions? :)16:25
* jrosser says hi to the PTG peeps16:25
spatelDo i need to pay for that?16:26
kaiokmocloudnull: my compliments to you all16:26
cloudnulljrosser yea the wazuh stuff isn't 100% quite yet16:26
jrosseri got very very close16:26
cloudnulli punted it over just before i went off on holiday for the last week16:26
jrosseri did engineer out the nodejs "magic install script" though16:26
cloudnullkaiokmo o/16:26
spatelcloudnull: i want to build ELB (loadbalancer) for 5 million connection.. so i started reading about octivia...16:27
cloudnullspatel there is a ticket for the PTG16:27
cloudnullspatel meet xgerman_16:27
cloudnullhe's working on octavia a lot16:27
spateli will try next year :)  once my cloud will reach to 500 compute nodes.16:27
spatelcloudnull: thanks i will contact xgerman_16:28
* cloudnull hasn't done much with that at this point 16:28
spatelxgerman_: ping me if you free.. wanted to talk about Octivia and how do i fix that with OSA16:29
spatelcloudnull: my boss is happy with OSA :) and he is going to buy more compute node to add in cluster :)16:30
cloudnullspatel nice!16:31
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-nspawn_container_create master: Correct bionic jobs to run bionic  https://review.openstack.org/60094416:32
jrossercloudnull: wazuh tinkering http://paste.openstack.org/show/729792/16:32
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-nspawn_container_create master: Use tests repo for common role test requirements  https://review.openstack.org/57955816:32
jrosseri think you also got node 6.x <> 8.x mixup possibly16:32
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-nspawn_container_create master: Use the TESTING_BRANCH env var for constraints  https://review.openstack.org/59249416:32
spatelcloudnull: & odyssey4me look like horizon developer accept my request and they issued patch https://bugs.launchpad.net/horizon/+bug/178932516:34
openstackLaunchpad bug 1789325 in OpenStack Dashboard (Horizon) "Remove network name from instance > IP address column " [Undecided,In progress] - Assigned to Akihiro Motoki (amotoki)16:34
*** DanyC has joined #openstack-ansible16:37
openstackgerritMerged openstack/openstack-ansible-os_monasca-agent master: Clean up role tests  https://review.openstack.org/60078616:44
*** lbragstad has quit IRC16:44
*** lbragstad has joined #openstack-ansible16:48
*** ppetit has quit IRC16:52
skiedudeif I change a network setting for host_bind_override in the user cfg, do I need to rerun all 3 playbooks? or just specific ones16:54
logan-odyssey4me: no file dropped like kvm_intel but nested virt is enabled by default it looks like http://paste.openstack.org/raw/729795/16:58
*** mashuser_ has joined #openstack-ansible16:58
*** mashuser_ has left #openstack-ansible16:58
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-plugins master: Add retry to execution commands  https://review.openstack.org/60130117:00
*** strattao has joined #openstack-ansible17:00
*** strattao has left #openstack-ansible17:01
*** Darcidride has joined #openstack-ansible17:02
openstackgerritMerged openstack/openstack-ansible-os_neutron stable/rocky: Change install_method by odl_install_method  https://review.openstack.org/59700317:03
*** mashuser_ has joined #openstack-ansible17:07
*** mashuser_ has left #openstack-ansible17:08
*** markvoel_ has quit IRC17:09
*** markvoelker has joined #openstack-ansible17:10
*** strattao_ has joined #openstack-ansible17:10
openstackgerritMerged openstack/openstack-ansible-os_heat master: Clean up role tests  https://review.openstack.org/60077817:10
*** strattao_ has quit IRC17:11
*** mashuser_ has joined #openstack-ansible17:12
*** mashuser_ has left #openstack-ansible17:12
*** markvoelker has quit IRC17:14
cloudnulljrosser nice!17:17
cloudnullmind just adding that to that PR ?17:17
cloudnullfeel free to push to that patch or make a dependent one17:17
* cloudnull never minds if folks add to a given pr 17:17
jrosserCool, np17:17
jrosserWhat do you think about node 6x vs 8x?17:18
openstackgerritMerged openstack/openstack-ansible-os_gnocchi master: Clean up role tests  https://review.openstack.org/60077417:18
jrosserAs far as I could see the wazuh install instructions wanted 8.x17:18
cloudnullyea I think it needs to be node 8x17:18
jrosserOk I can fix that too.17:19
cloudnullthanks!17:19
jrosserIt all installed, though I simultaneously put elk 6.4.0 on too which came with a bit of fun17:19
cloudnullha!17:19
cloudnullyea there were a few things that changed in 6.4 but i believe i caught all of the upgrade bits.17:20
jrosserI have a PR for logstash only seeming to listen on IPv6 which is like a second view on17:20
jrosserI half wonder if I actually had other brok too17:20
* cloudnull goes to have a look 17:20
jrossercloudnull: ah there’s another thing, I ran out of the sysctl “file watchers” limit on my log nodes and that creates all sorts of weird with systemd17:23
odyssey4med34dh0r53 cloudnull reviews if you have a few mins please? https://review.openstack.org/60081817:24
cloudnulljrosser oh ?17:25
cloudnullsomething you had to raise?17:26
cloudnulldid you have to increase  fs.inotify.max_user_watches ?17:26
jrosserYes17:26
openstackgerritMerged openstack/openstack-ansible-os_horizon master: Clean up role tests  https://review.openstack.org/60077917:26
cloudnullmaybe something we just need to set17:27
cloudnull?17:27
jrosserI think it’s in openstack_hosts (just on my mobile so can’t check)17:27
jrosserBut really look like it should be in the elk roles too if we want them portable elsewhere17:27
cloudnullfor elk we could set it the elk roles17:27
cloudnullwhat did you need to set it to ?17:28
jrosserI think I set it to 1M then stuff was good17:28
jrosserif you run out then systemctl can’t restart services17:28
cloudnullfs.epoll.max_user_watches = 166920117:28
cloudnullfs.inotify.max_user_watches = 3686417:28
cloudnullthats the default17:29
jrosserYes, that’s certainly not enough on the elastic_logstash nodes17:29
cloudnullok17:29
jrosserAs far as the rest of the wazuh stuff went it’s all gone on, but the registration of the agents is broken17:30
openstackgerritMerged openstack/openstack-ansible-tests master: Resolve E004 File did not end with a newline  https://review.openstack.org/60096917:31
*** lvdombrkr has joined #openstack-ansible17:34
openstackgerritMerged openstack/openstack-ansible-ops master: Move bootstrap-ansible and passwords to run_osa tag  https://review.openstack.org/60087517:35
openstackgerritMerged openstack/openstack-ansible-os_barbican master: Clean up role tests  https://review.openstack.org/60076117:35
openstackgerritGerman Eichberger proposed openstack/openstack-ansible-os_octavia master: Fixes too low security group rules quota  https://review.openstack.org/60082617:41
openstackgerritVictor Palma proposed openstack/openstack-ansible-galera_server master: fix bootstrap of galera 10.2+ clusters  https://review.openstack.org/59253117:43
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Set the max user watches to 1M  https://review.openstack.org/60133017:43
cloudnulljrosser ^17:43
jrossercloudnull: copy paste gremlins there :)17:46
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-plugins master: Add retry to execution commands  https://review.openstack.org/60130117:49
odyssey4mehttps://github.com/openstack/openstack-ansible-plugins/commit/f56d1242c99f68659e575aa610cdefff73e7707a17:52
odyssey4mecloudnull logan- ^17:52
*** strattao_ has joined #openstack-ansible17:52
*** strattao_ has left #openstack-ansible17:53
odyssey4mecloudnull logan- https://review.openstack.org/#/q/project:openstack/openstack-ansible-plugins+status:open17:53
jrossercloudnull: it’s all coming back to me.... :) there’s ELK memory issues too that need looking at17:53
*** strattao_ has joined #openstack-ansible17:54
odyssey4melogan- wanna review https://review.openstack.org/#/c/601301/3 ?17:58
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-tests stable/rocky: Resolve E004 File did not end with a newline  https://review.openstack.org/60133717:58
*** poopcat has joined #openstack-ansible18:01
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Set the max user watches to 1M  https://review.openstack.org/60133018:02
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-plugins stable/ocata: Correct severe performance problem  https://review.openstack.org/58220418:04
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-plugins stable/ocata: Correct severe performance problem  https://review.openstack.org/58220418:04
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-plugins stable/pike: Correct severe performance problem  https://review.openstack.org/60134218:05
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-plugins stable/queens: Correct severe performance problem  https://review.openstack.org/60134318:05
cloudnulllogan- ^18:06
d34dh0r53odyssey4me, cloudnull, et al: https://review.openstack.org/#/c/586612/ pretty please18:10
*** electrofelix has quit IRC18:11
openstackgerritMerged openstack/openstack-ansible-os_trove master: Clean up role tests  https://review.openstack.org/60106218:18
*** lbragstad has quit IRC18:20
*** radeks__ has quit IRC18:21
*** radeks__ has joined #openstack-ansible18:22
openstackgerritMerged openstack/openstack-ansible-plugins master: Add retry to execution commands  https://review.openstack.org/60130118:25
*** priteau has quit IRC18:29
*** priteau has joined #openstack-ansible18:30
*** priteau has quit IRC18:30
*** priteau has joined #openstack-ansible18:30
*** priteau has quit IRC18:32
*** hamzaachi has joined #openstack-ansible18:33
*** lvdombrkr has quit IRC18:34
*** DanyC has quit IRC18:36
*** DanyC has joined #openstack-ansible18:36
*** priteau has joined #openstack-ansible18:38
*** lbragstad has joined #openstack-ansible18:39
*** DanyC has quit IRC18:41
*** lbragstad has quit IRC18:41
*** radeks__ has quit IRC18:42
*** priteau has quit IRC18:42
*** spatel has quit IRC18:46
*** markvoelker has joined #openstack-ansible18:57
*** Leo_m_ has joined #openstack-ansible18:59
*** Leo_m has quit IRC19:02
strattao_Hoping someone can give me a hand with my weird error… I’ve completed a successful deployment and try to log in to the dashboard. When I try logging in with admin credentials (or the demo credentials created with tempest) the login page sits an thinks for a while, then the page just refreshes.19:02
strattao_Checking the horizon error logs, I see: The timeout specified has expired: [client 192.168.215.3:44978] AH01991: SSL input filter read failed.19:02
strattao_My little digging around tells me that I may have an issue with the apache config, but I didn’t immediately see any issues and am not sure what else to check… Any thoughts?19:05
strattao_(This is on a 6 node test cluster, 3 controllers, 2 computes, 1 loghost)19:06
*** spatel has joined #openstack-ansible19:10
strattao_hmmmm, well - I was just able to log in, but it hit a different controller this time, so I must have something out of sync in the haproxy config or something… Will do some more digging. Thx.19:18
*** DanyC has joined #openstack-ansible19:23
*** priteau has joined #openstack-ansible19:27
*** spatel has quit IRC19:28
*** priteau has quit IRC19:28
*** lbragstad has joined #openstack-ansible19:31
*** hamzaachi has quit IRC19:36
*** Darcidride has quit IRC19:38
*** priteau has joined #openstack-ansible19:43
*** priteau has quit IRC19:47
*** priteau has joined #openstack-ansible19:48
*** priteau has quit IRC19:48
*** strattao_ has quit IRC19:49
*** priteau has joined #openstack-ansible19:49
*** strattao has joined #openstack-ansible19:57
*** spatel has joined #openstack-ansible20:04
*** hamzaachi has joined #openstack-ansible20:06
*** dw has quit IRC20:07
*** pcaruana has quit IRC20:08
*** dw has joined #openstack-ansible20:09
*** radeks__ has joined #openstack-ansible20:12
*** markvoelker has quit IRC20:14
*** arxcruz is now known as arxcruz|ptg20:23
*** SimAloo has joined #openstack-ansible20:30
*** strattao has quit IRC20:31
*** hamzaachi has quit IRC20:34
*** markvoelker has joined #openstack-ansible20:36
*** lbragstad has quit IRC20:39
*** ansmith has quit IRC20:45
*** radeks__ has quit IRC20:45
*** lbragstad has joined #openstack-ansible20:46
*** radeks__ has joined #openstack-ansible20:47
*** luksky has joined #openstack-ansible20:57
*** markvoelker has quit IRC21:02
*** savvas has quit IRC21:06
*** DanyC has quit IRC21:06
*** DanyC has joined #openstack-ansible21:06
*** markvoelker has joined #openstack-ansible21:07
*** nicolasbock has joined #openstack-ansible21:09
*** DanyC has quit IRC21:11
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Set the max user watches to 1M  https://review.openstack.org/60133021:12
*** savvas has joined #openstack-ansible21:13
*** spatel has quit IRC21:13
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Enforce no_proxy when setting up ELK dashboards and rollups  https://review.openstack.org/60000821:14
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Ensure logstash listens on ipv4 address  https://review.openstack.org/60000921:14
nicolasbockHi everybody. I just deployed a brand new AIO on Leap 42.3 (I used 17.1.0). It ran great until I decided to reboot it.21:17
*** luksky has quit IRC21:17
nicolasbockOn boot I am getting lots of errors such as: parsing [/etc/haproxy/haproxy.cfg:30] : 'bind 38.108.68.9:8776' : unable to load SSL private key from PEM file '/etc/ssl/private/haproxy.pem'.21:17
nicolasbockHas anyone seen this?21:17
nicolasbockDoes anyone have a suggestion on how to fix it?21:17
openstackgerritVictor Palma proposed openstack/openstack-ansible-galera_server master: fix bootstrap of galera 10.2+ clusters  https://review.openstack.org/59253121:18
*** savvas has quit IRC21:18
cloudnullnicolasbock maybe the key was removed?21:19
*** hamzaachi has joined #openstack-ansible21:19
jrossernicolasbock: look at that pem file and check it is valid, particularly the way that the three parts are concatented together. Lack of carriage returns in the right place can break that21:19
cloudnull^21:20
*** spatel has joined #openstack-ansible21:21
*** nicolasbock has quit IRC21:22
*** savvas has joined #openstack-ansible21:23
*** hamzaachi has quit IRC21:27
*** savvas has quit IRC21:27
*** nicolasbock has joined #openstack-ansible21:28
*** savvas has joined #openstack-ansible21:32
*** spatel has quit IRC21:35
*** ansmith has joined #openstack-ansible21:35
*** savvas has quit IRC21:36
*** savvas has joined #openstack-ansible21:41
*** lbragstad has quit IRC21:53
*** SimAloo has quit IRC21:54
*** skiedude has quit IRC22:01
*** goldenfri has quit IRC22:02
openstackgerritDavid Wilde proposed openstack/openstack-ansible-ops master: Deploy ELK in MNAIO  https://review.openstack.org/58661222:04
openstackgerritMerged openstack/openstack-ansible-ops master: Ensure logstash listens on ipv4 address  https://review.openstack.org/60000922:10
openstackgerritMerged openstack/openstack-ansible-plugins stable/ocata: Correct severe performance problem  https://review.openstack.org/58220422:10
nicolasbockjrosser, cloudnull, it ended up being apparmor22:12
*** savvas has quit IRC22:14
openstackgerritMerged openstack/openstack-ansible-ops master: Enforce no_proxy when setting up ELK dashboards and rollups  https://review.openstack.org/60000822:15
openstackgerritMerged openstack/openstack-ansible-plugins stable/queens: Correct severe performance problem  https://review.openstack.org/60134322:17
openstackgerritMerged openstack/openstack-ansible-plugins stable/pike: Correct severe performance problem  https://review.openstack.org/60134222:17
*** savvas has joined #openstack-ansible22:20
cloudnullnicolasbock orly122:21
cloudnullIs it something that we need to update in the profile?22:21
nicolasbockYes, possibly22:21
nicolasbockThere is an apparmor profile for haproxy22:21
nicolasbockWhen I disable apparmor alltoghether (which is a bit hackish)22:22
nicolasbockit works22:22
nicolasbockI am running into a new error though :(22:22
cloudnullah. so maybe we just need to tweak it to respect that pem file22:22
cloudnullwhat's the new error ?22:22
nicolasbockfatal: [aio1_galera_container-a67c200f]: UNREACHABLE!22:23
nicolasbocklxc-ls shows the container though22:23
cloudnullno ips ?22:23
nicolasbockWhen I 'lxc-attach' I get: lxc-attach: attach.c: lxc_attach: 755 Failed to get init pid.22:23
cloudnullah is the container running ?22:24
cloudnullmaybe it needs to be killed and restarted22:24
nicolasbockHmm, and there are hardly network devices now22:24
cloudnullsomething like `lxc-stop -fn $name`22:24
nicolasbockI think the networking wasn't set up22:24
cloudnulloh!22:24
nicolasbockWhat service is that?22:25
cloudnullsystemd-networkd22:25
cloudnullwell, which release?22:25
nicolasbockLeap 42.322:26
nicolasbock17.1.022:26
cloudnullah yea, that should be networkd on the aio22:26
cloudnullmaybe it needs to be restarted ?22:26
nicolasbockInteresting, that service is disabled22:26
nicolasbockIs that a wicked conflict?22:26
cloudnullit shouldnt conflict at all22:27
cloudnullits just creating dummy devices on the aio22:27
nicolasbockok22:27
cloudnullI dont think we enable the service in the aio, we just start it and run the networks22:27
cloudnullthough we could start it to help make the AIO survive a reboot.22:27
nicolasbockI just restarted networkd22:27
nicolasbockAnd don't see any additional devices22:28
nicolasbockI'll enable the service and reboot22:28
cloudnullhum ..22:28
cloudnulldo you have networkd service files ?22:28
cloudnullin /etc/systemd/networkd?22:28
cloudnullsorry /etc/systemd/network/22:29
nicolasbockYes, there are a bunch of files in there22:30
nicolasbockDo you have a minute to hack this together?22:30
cloudnullsure22:34
nicolasbockWhere are you?22:34
nicolasbockWhere I talked to you earlier22:35
cloudnull we're in the hall by registration22:35
nicolasbock?22:35
nicolasbockOk, I am walking over22:35
cloudnullthere's power here :)22:35
*** lbragstad has joined #openstack-ansible22:38
prometheanfireish22:48
*** savvas has quit IRC22:53
openstackgerritMerged openstack/ansible-role-python_venv_build master: Add option for default pip package install  https://review.openstack.org/57454422:56
*** savvas has joined #openstack-ansible22:59
*** macza has quit IRC23:05
*** macza has joined #openstack-ansible23:05
openstackgerritDavid Wilde proposed openstack/openstack-ansible-ops master: Deploy ELK in MNAIO  https://review.openstack.org/58661223:14
*** lbragstad has quit IRC23:14
*** pmannidi has joined #openstack-ansible23:33
*** savvas has quit IRC23:33
*** savvas has joined #openstack-ansible23:38
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-haproxy_server master: Fix haproxy when there's an haproxy apparmor profile  https://review.openstack.org/60146723:56
*** evrardjp has quit IRC23:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!