Friday, 2018-02-09

*** acormier has joined #openstack-ansible00:17
*** hamzy_ has joined #openstack-ansible00:17
*** acormier has quit IRC00:21
*** dave-mccowan has joined #openstack-ansible00:22
*** threestrands has joined #openstack-ansible00:27
*** threestrands has quit IRC00:27
*** threestrands has joined #openstack-ansible00:27
*** acormier has joined #openstack-ansible00:48
*** acormier has quit IRC00:49
*** acormier has joined #openstack-ansible00:49
*** weezS has joined #openstack-ansible01:01
*** weezS has quit IRC01:07
*** gfa has quit IRC01:10
*** gfa has joined #openstack-ansible01:11
*** vnogin has joined #openstack-ansible01:15
*** vnogin has quit IRC01:20
logan-mnaser: ceph_client https://github.com/openstack/openstack-ansible-ceph_client01:20
*** woodard has quit IRC01:21
*** woodard has joined #openstack-ansible01:21
*** tux_ has joined #openstack-ansible01:27
*** threestrands has quit IRC01:29
*** Jeffrey4l has joined #openstack-ansible01:34
openstackgerritzhongshengping proposed openstack/openstack-ansible-os_nova master: Adjust deprecated nova console variables  https://review.openstack.org/53283601:44
*** Jeffrey4l_ has joined #openstack-ansible01:46
*** Jeffrey4l has quit IRC01:47
openstackgerritJames E. Blair proposed openstack/openstack-ansible-rabbitmq_server stable/pike: Zuul: Remove project name  https://review.openstack.org/54253301:47
*** rama_y has quit IRC01:48
*** TxGirlGeek has quit IRC01:50
*** tux_ has quit IRC01:51
*** tux_ has joined #openstack-ansible01:56
*** tux_ has quit IRC01:57
mnaserlogan-: that role doesnt seem to include any repos or add them02:01
mnaserjust install ceph client side things02:01
logan-well for example there are repos added here https://github.com/openstack/openstack-ansible-ceph_client/blob/master/tasks/ceph_preinstall_apt.yml02:03
logan-which distro are you working on?02:04
logan-rhel/centos?02:04
logan-and I guess reading into what you said above a little more, the issue is with the gnocci wheel build right? so something that should be installed in the repo container02:04
logan-so maybe you'd want to add something to the repo_build role (where the wheel is built) and also os_gnocchi (where the wheel is installed/used)02:06
logan-or alternatively ceph_client instead of os_gnocchi if it is a general ceph client need not specific to gnocchi02:07
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_aodh stable/newton: Zuul: Remove project name  https://review.openstack.org/54254102:08
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_almanach stable/pike: Zuul: Remove project name  https://review.openstack.org/54254502:28
*** hamzy_ has quit IRC02:53
*** tux_ has joined #openstack-ansible02:55
*** hamzy_ has joined #openstack-ansible02:58
*** jwitko has quit IRC03:03
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_swift stable/ocata: Zuul: Remove project name  https://review.openstack.org/54255803:08
tux_Ceilometer eating lots of CPU on my AIO box03:10
tux_show do i keep shutdown service so i can enable when i need?03:10
tux_so do i keep shutdown service so i can enable when i need?03:10
tux_i just ceilometer container for now03:21
*** tux_ has quit IRC03:28
*** gkadam has joined #openstack-ansible03:34
*** acormier has quit IRC03:43
*** dave-mccowan has quit IRC03:48
*** chhagarw has joined #openstack-ansible03:57
*** udesale has joined #openstack-ansible04:02
*** bhujay has joined #openstack-ansible04:03
openstackgerritJames E. Blair proposed openstack/openstack-ansible-tests stable/newton: Zuul: Remove project name  https://review.openstack.org/54258104:08
*** chyka has joined #openstack-ansible04:41
*** rama_y has joined #openstack-ansible04:44
*** chyka has quit IRC04:46
*** poopcat has quit IRC05:05
*** gkadam_ has joined #openstack-ansible05:10
*** vnogin has joined #openstack-ansible05:15
*** vnogin has quit IRC05:20
*** rama_y has quit IRC05:24
*** Deagon has joined #openstack-ansible05:26
*** Deagon has quit IRC05:28
*** rama_y has joined #openstack-ansible05:32
*** ivveh has quit IRC05:33
*** dariko has joined #openstack-ansible05:41
*** rama_y has quit IRC05:47
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249105:48
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_ceilometer stable/pike: Zuul: Remove project name  https://review.openstack.org/54260105:49
*** dariko has quit IRC05:58
*** chyka has joined #openstack-ansible06:10
*** chyka has quit IRC06:14
openstackgerritJames E. Blair proposed openstack/openstack-ansible-lxc_container_create stable/pike: Zuul: Remove project name  https://review.openstack.org/54262106:29
jafehagood morning06:30
hwoaranggood morning06:30
*** chyka_ has joined #openstack-ansible06:34
openstackgerritMerged openstack/openstack-ansible-galera_server stable/ocata: Remove release note for yum mirror change  https://review.openstack.org/54219106:38
openstackgerritMerged openstack/openstack-ansible-openstack_hosts stable/pike: Zuul: Remove project name  https://review.openstack.org/54246006:39
openstackgerritMerged openstack/openstack-ansible-galera_client master: Remove release note for yum mirror change  https://review.openstack.org/54218906:43
openstackgerritMerged openstack/openstack-ansible-galera_client stable/pike: Remove release note for yum mirror change  https://review.openstack.org/54219206:43
openstackgerritMerged openstack/openstack-ansible-galera_client stable/ocata: Remove release note for yum mirror change  https://review.openstack.org/54219006:43
*** acormier has joined #openstack-ansible06:44
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249106:47
openstackgerritJames E. Blair proposed openstack/openstack-ansible-lxc_hosts stable/newton: Zuul: Remove project name  https://review.openstack.org/54263406:49
*** acormier has quit IRC06:49
openstackgerritMerged openstack/openstack-ansible-os_ceilometer master: Updated from global requirements  https://review.openstack.org/53505107:08
openstackgerritMerged openstack/openstack-ansible-os_rally master: Updated from global requirements  https://review.openstack.org/53506907:10
openstackgerritMerged openstack/openstack-ansible-os_barbican stable/pike: Zuul: Remove project name  https://review.openstack.org/54245707:13
tescraam i to add the ips of infra nodes for HA proxy in file/etc/openstack_deploy/conf.d/haproxy.yml for production environment?07:14
tescrain pike07:14
tescraalso create respective roles?07:15
tescrafor HA07:15
tescraor it is installed by defalt?07:15
jafehatescra, you'd usually define internal and external loadbalancer vips in the openstack_user_config.yml07:15
*** sar has joined #openstack-ansible07:16
jafehahaproxy is installed to all your infra hosts by default, so it's HA by default07:17
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249107:17
jafehayou can choose to use your own loadbalancer though07:17
tescrathanks @ jafeha07:19
*** chyka_ has quit IRC07:19
*** taseer1 has joined #openstack-ansible07:21
openstackgerritMerged openstack/openstack-ansible-rabbitmq_server stable/ocata: Zuul: Remove project name  https://review.openstack.org/54212007:21
*** Taseer has quit IRC07:22
*** taseer2 has joined #openstack-ansible07:22
openstackgerritMerged openstack/openstack-ansible-os_nova master: Adjust deprecated nova console variables  https://review.openstack.org/53283607:25
openstackgerritMerged openstack/openstack-ansible-os_glance stable/ocata: tests: test-glance-upgrades.sh: Remove the exit trap  https://review.openstack.org/54147107:25
*** taseer2 is now known as Taseer07:25
*** eumel8 has joined #openstack-ansible07:26
*** taseer1 has quit IRC07:26
openstackgerritMerged openstack/openstack-ansible-os_trove master: Update the Tempest plugin to point towards the new plugin  https://review.openstack.org/54127207:28
evrardjptescra: on top of that if you want dedicated nodes it's also possible.07:30
jafehagood morning evrardjp07:31
openstackgerritMerged openstack/openstack-ansible-os_trove master: Zuul: Remove project name  https://review.openstack.org/54030807:35
openstackgerritMerged openstack/openstack-ansible-os_trove master: Updated from global requirements  https://review.openstack.org/53507407:35
evrardjpgood morning jafeha07:35
evrardjpFYI that's what your colleagues have seen: https://jira.mariadb.org/browse/MDEV-1525407:35
*** pcaruana has joined #openstack-ansible07:35
evrardjpthat's why rsync was the fix07:36
jafehai have seen you guys had real fun yesterday with the whole galera fun07:36
jafehayeah, that sounds like the stuff we were suffering.07:37
jafehastill recovereing on some fronts as it looks07:37
*** bauruine has joined #openstack-ansible07:38
jafehaevrardjp, do you already have a plan how to deal with this? like version pinning or sth like that?07:38
evrardjpjafeha: yes, we have a series of patches to use 10.1.30 for now, setting to repo to use 10.1.30 instead of a 10.1 repo07:39
evrardjpall deployers having 10.1.31 should uninstall it, and re-run the playbooks.07:39
evrardjpI haven't bumped the galera_client and galera_server role for the stable branches, and I will do it today07:39
jafehamkay07:39
evrardjpin the meantime, I am trying to find a long term fix07:40
tescraTASK [lxc_hosts : Ensure that the LXC cache has been prepared] ******************************************************* Thursday 08 February 2018  23:39:13 -0800 (0:00:00.538)07:40
tescrafails07:40
evrardjpbut if you moved to use rsync instead of xtrabackup-v2 for sst, everything should be alright for you07:40
tescrafor pike07:40
evrardjpnetworking issue tescra ?07:41
jafehai need another look at our galera setup, as we're using rsync to sync right now07:41
evrardjpyeah rsync works07:41
tescraFAILED - RETRYING: Ensure that the LXC cache has been prepared (111 retries left). FAILED - RETRYING: Ensure that the LXC cache has been prepared (111 retries left). FAILED - RETRYING: Ensure that the LXC cache has been prepared (111 retries left). FAILED - RETRYING: Ensure that the LXC cache has been prepared (11007:41
evrardjpif you want to rollback to xtrabackup-v2 that would fail with 10.1.3107:41
jafehaic.07:41
evrardjptescra: that's normal07:41
evrardjpit's an asynchronous event07:42
evrardjpit will retry until everything has succeed.07:42
evrardjpat the end if the last line is a fail, you have a hard failure07:42
evrardjpbut right now your task hasn't failed in itself07:42
jafehai'll have another look how broken things are, i can't point my finger to it, but i think our galera cluster has some more issues than just the replication07:43
evrardjpoh i see07:43
evrardjpwell rsync is slow07:43
evrardjpso if you have a network connectivity issue, it will freeze and rsync, which can disrupt the write cycles I guess07:43
evrardjpbut don't trust my word on that, I am definitely a noob07:43
jafehanetworking "should" be fine07:44
jafehaat least no saturation there07:44
tescraevrardjp it changed after sometime that's OK?07:45
tescraThanks for explain07:46
evrardjptescra: yeah basically we start these command in the background, with ansible async . Then we test until these commands have completed or a timeout is exceeded.07:48
jafehaevrardjp, are there any outstanding cinder issues? getting a HTTP 500 on instance creation07:49
evrardjpnot that I am aware of07:49
evrardjpcheck your lb ?07:49
evrardjpjafeha: you're on ubuntu, right?07:50
jafehayep07:51
evrardjpk07:51
jafehaloadbalancer seems fine07:51
openstackgerritMerged openstack/openstack-ansible-os_magnum master: Update the Tempest plugin to point towards the new plugin  https://review.openstack.org/54126907:51
openstackgerritMerged openstack/openstack-ansible-os_magnum master: Zuul: Remove project name  https://review.openstack.org/53862707:51
openstackgerritMerged openstack/openstack-ansible-os_magnum master: Updated from global requirements  https://review.openstack.org/53506107:51
evrardjpwell it should report the 50007:51
evrardjpshouldn't it ? :p07:52
*** mbuil has joined #openstack-ansible07:52
evrardjphello mbuil07:52
Taseerevrardjp: can this now be merged ? => https://review.openstack.org/#/c/522491/07:52
evrardjpTaseer: woot! I will have a look07:52
evrardjpI see green I will check if it's doing proper tests and all07:53
jafehaevrardjp, on a closer look there seems to be something broken07:53
evrardjpyup07:54
Taseerok07:54
*** hamza21 has joined #openstack-ansible07:59
openstackgerritMerged openstack/openstack-ansible-ops stable/pike: Zuul: Remove project name  https://review.openstack.org/54191308:01
Taseerevrardjp: what did you mean by "change functional tests in a non-voting fashion" ?08:03
jafehaevrardjp, seems ok now. had to restart one cinder-api container.08:04
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249108:04
evrardjpTaseer: done08:04
evrardjpTaseer: we are setting false expectations by showing functional tests that are green, while there is nothing that functionally tests it.08:05
evrardjpso I am moving these tests to basic functional tests, and we'll able to merge even if it doesn't pass test.08:05
Taseerokay08:05
evrardjpwith the hope this will get fixed as soon as possible in another commit08:05
evrardjpthis commit has lasted too long already08:06
mbuilhello evrardjp08:06
evrardjphow are things?08:07
openstackgerritMerged openstack/openstack-ansible-ceph_client stable/newton: Zuul: Remove project name  https://review.openstack.org/54191908:08
openstackgerritJames E. Blair proposed openstack/openstack-ansible-memcached_server stable/newton: Zuul: Remove project name  https://review.openstack.org/54270708:09
*** hw_wutianwei has quit IRC08:11
*** hw_wutianwei has joined #openstack-ansible08:11
evrardjphwoarang: sorry I didn't babysit the https://review.openstack.org/#/c/538933/ patch. will keep under my nose today08:11
hwoarangevrardjp: no worries i am also keeping an eye on it08:12
evrardjpthis mariadb has taken some time, and I had quite a series of meetings yesterday :/08:12
openstackgerritMerged openstack/openstack-ansible-ceph_client stable/ocata: Zuul: Remove project name  https://review.openstack.org/54186908:13
openstackgerritMerged openstack/openstack-ansible-rsyslog_client stable/ocata: Correct task tag  https://review.openstack.org/54102508:17
*** peri has joined #openstack-ansible08:18
tescrahi for some reason my putty connection got disconnected?08:18
tescraand server is at reomte location08:19
*** sar has quit IRC08:20
tescracan i restart the session and rerun the playbook openstack-ansible setup-hosts.yml or it will cause discrepancy08:24
tescraTASK [lxc_container_create : Container network includes] was in progress08:27
tescracan i start executing play book from here ?08:28
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_ironic stable/newton: Zuul: Remove project name  https://review.openstack.org/54271308:29
openstackgerritMerged openstack/openstack-ansible-memcached_server stable/newton: Zuul: Remove project name  https://review.openstack.org/54270708:30
tescrais there any tag?08:34
*** sar has joined #openstack-ansible08:34
openstackgerritMerged openstack/openstack-ansible-os_ceilometer master: Remove pip_install dependency  https://review.openstack.org/52554608:36
*** olivierbourdon38 has joined #openstack-ansible08:42
tescracan anyone suggest what to do?08:43
jafehatescra, i'd suggest to use tmux, screen or some other terminal multiplexer to avoid those disconnects08:46
jafehaor issues with them08:46
tescrabut what to do now replay the play book?08:47
jafehai'd probably do a complete re-run of the setup-hosts08:47
tescraok but it had executed few plays so the changes made will be overwritten or cause error?08:49
tescrathanks jafeha08:50
*** gameon has joined #openstack-ansible08:50
jafehayou should see a lot of green then, ansible won't execute tasks that do no change08:50
gameonHi all - I have a question which I'm struggling to find any info on. I have a Kilo deployment, and have a provider network which is set as external. This network is NOT visible from within tenant projects. However, using a new Pike deplyoment, the provider network IS visible (and deleteable even!!) from tenant projects. Has anyone got a solution to this? RBAC didn't seem to do anything, as soon as I set the network to extern08:52
gameon(I realise this isn't strictly OSA related but I'm curious if anyone else has seen this behavior. This is a 16.0.6 deployment)08:53
hwoarangevrardjp: could you quickly get https://review.openstack.org/#/c/542458/ in ?08:53
hwoarangi broke stuff ;p08:53
tescrathanks again Jafeha08:53
*** armaan has joined #openstack-ansible08:54
evrardjphwoarang: oh yeah I remember we talked about it yesterday.08:54
evrardjpSorry I have missed it, I am talking with maria ppl to find out what's wrong08:54
evrardjpI think you can help I've narrowed it down to 2 shell scripts.08:55
evrardjp:D08:55
hwoarangheh08:56
openstackgerritMerged openstack/openstack-ansible-tests stable/pike: Zuul: Remove project name  https://review.openstack.org/54184808:59
openstackgerritMerged openstack/openstack-ansible-galera_client master: Revert "RedHat: Ensure that conflicting MariaDB packages are removed"  https://review.openstack.org/54245809:04
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249109:05
openstackgerritMerged openstack/openstack-ansible-os_nova stable/pike: If debug is enabled, provide verbose output for host discovery  https://review.openstack.org/54228709:07
openstackgerritOpenStack Proposal Bot proposed openstack/openstack-ansible-os_monasca-agent stable/pike: Updated from OpenStack Ansible Tests  https://review.openstack.org/54273509:07
openstackgerritMerged openstack/openstack-ansible-os_magnum master: Replace virtualenv-tools by a script  https://review.openstack.org/53365709:10
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_trove master: Updated from OpenStack Ansible Tests  https://review.openstack.org/53644109:15
*** pbandark has joined #openstack-ansible09:17
*** Jeffrey4l_ has quit IRC09:17
*** Jeffrey4l has joined #openstack-ansible09:18
Miougeevrardjp: any tips on how I can add tests to check that the ceph-rgw is working? Also swift and ceph-rgw should probably be mutually exclusive since they use the same HaProxy port?09:18
evrardjpMiouge: agreed.09:18
evrardjpdoes the tempest swift testing suite apply?09:19
evrardjpelse maybe we should change anything, and it's already functionally tested then, we could just change the conditional to say if ceph and swift, only deploy one of those09:19
evrardjpwhich is a scenario thing09:19
evrardjpand a group vars thing09:20
evrardjpI don't know how you test it rn09:20
MiougeSince swift and rgw are mutually exclusive, I wonder if openstack-ansible-deploy-ceph-ubuntu-xenial install swift as well. That test currently passes and install rgw, but if swift is also there it should fail.09:23
*** acormier has joined #openstack-ansible09:24
tescrahi on replay of the playbook it seems to be stuck at same task where the session disconnected...any idea?09:24
*** acormier has quit IRC09:24
*** acormier has joined #openstack-ansible09:25
*** sar has quit IRC09:25
tescra@ jafeha evrardjp09:26
*** acormier has quit IRC09:26
*** acormier has joined #openstack-ansible09:27
evrardjpMiouge: that's a scenario, so we can adapt it. I think we should adapt the scenario to not deploy swift, but use ceph + rgw09:31
evrardjpI can change that on your patch if you want09:31
*** acormier has quit IRC09:31
evrardjptescra: that's too vague, not sure I can help. Re-Running playbooks should not be a problem09:31
tescraok thanks09:32
evrardjptescra: maybe the output could be helpful?09:33
tescrano output the session terminated again at same play09:34
evrardjplack of ramn?09:34
tescrain deploy host?09:35
tescraor infra ?09:35
Miougeevrardjp: if you know how/where to change that, go ahead. I greped the repo, that didnā€™t help me09:36
evrardjptescra: well that's too vague, but I'd say deploy right now.09:36
evrardjpshow me a log and I can probably help09:36
evrardjpMiouge: no worries, will handle it.09:37
tescraok 1 minute09:37
evrardjptescra: thanks :)09:37
tescrasyslog:Feb  7 21:13:25 deploy-node systemd[1]: Started Set console keymap. Feb  7 21:13:25 deploy-node systemd-modules-load[430]: Inserted module 'ib_iser' Feb  7 21:13:25 deploy-node systemd[1]: Started Load Kernel Modules. Feb  7 21:13:25 deploy-node systemd[1]: Starting Apply Kernel Variables... Feb  7 21:13:25 deploy-node systemd[1]: Mounting FUSE Control File System... Feb  7 21:13:25 deploy-node systemd[1]: Mounted F09:39
Taseerevrardjp: the api verification failed09:40
odyssey4meevrardjp I think it might be time to just merge https://review.openstack.org/#/c/522491/, then inspect afterwards and register bugs for things that need to be done?09:40
Taseerbut the patch can be merged, I guess :)09:40
tescraauthlog:Feb  9 01:40:28 deploy-node sshd[24735]: pam_unix(sshd:session): session closed  for user osadmin Feb  9 01:40:28 deploy-node su[24829]: pam_unix(su:session): session closed for  user root Feb  9 01:40:28 deploy-node sudo: pam_unix(sudo:session): session closed for use r root09:41
evrardjpTaseer: yes, but you'll have to fix it later.09:42
openstackgerritMerged openstack/openstack-ansible-galera_client stable/newton: Zuul: Remove project name  https://review.openstack.org/54247109:42
openstackgerritMerged openstack/openstack-ansible-rsyslog_client stable/pike: Fix pep8 doc config lint failures  https://review.openstack.org/54184709:42
odyssey4meotherwise I can review whats there, and we go through many more review rounds?09:42
evrardjpodyssey4me: that's exactly my point: let's stop here, and iterate09:43
evrardjptescra: could you run everything as root? It would make your life simpler. Use ssh keys.09:44
evrardjpelse you'd have to pass things in the cli, to force the user, the method, etc.09:44
evrardjptescra: but I don't see the full flog there09:45
odyssey4mealright, 86 iterations already - how about we go for 100 ;)09:45
evrardjpodyssey4me: haha09:45
tescra update-alternatives 2018-02-07 22:20:02: run with --install /bin/nc nc /bin/nc.traditional 10 --slave /bin/netcat netcat /bin/nc.traditional --slave /usr/share/man/man1/nc.1.gz nc.1.gz /usr/share/man/man1/nc.traditional.1.gz --slave /usr/share/man/man1/netcat.1.gz netcat.1.gz /usr/share/man/man1/nc.traditional.1.gz09:47
openstackgerritMerged openstack/openstack-ansible-lxc_hosts stable/newton: Zuul: Remove project name  https://review.openstack.org/54263409:47
openstackgerritMerged openstack/openstack-ansible-rsyslog_client stable/pike: Correct task tag  https://review.openstack.org/54102409:47
tescraam root  users09:47
tescrawhile executing playbooks09:47
evrardjptescra: I'd need a playbook run log09:48
evrardjpnot a system log09:48
*** shardy has joined #openstack-ansible09:49
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_nova stable/newton: Zuul: Remove project name  https://review.openstack.org/54274709:50
Taseerevrardjp: I will fix it. I just need some help.09:52
Taseerfrom a person in congress09:53
openstackgerritMerged openstack/openstack-ansible-os_almanach stable/pike: Zuul: Remove project name  https://review.openstack.org/54254509:53
tescraevrardjp path for playbook run log? can't find in /var/log of depoly node09:55
openstackgerritMerged openstack/openstack-ansible-os_keystone stable/pike: Use the good configuration directive for memcache servers  https://review.openstack.org/54093109:57
evrardjpTaseer: yeah, but we can do it in a later commit, let's just start by inclusion in osa09:59
evrardjptescra: just re-run it and capture the output?10:00
evrardjpelse it's in your ansible log folder location maybe /var/log/somethingansible?10:00
openstackgerritMerged openstack/openstack-ansible-lxc_container_create stable/pike: Zuul: Remove project name  https://review.openstack.org/54262110:01
openstackgerritMerged openstack/openstack-ansible-os_cinder stable/pike: Zuul: Remove project name  https://review.openstack.org/53854410:05
*** nsingh has joined #openstack-ansible10:06
*** niraj_singh has quit IRC10:08
Taseerevrardjp: so I guess, it just needs your approval with the long awaited +2 :)10:08
openstackgerritMerged openstack/openstack-ansible-os_aodh stable/newton: Zuul: Remove project name  https://review.openstack.org/54254110:09
openstackgerritMerged openstack/openstack-ansible-tests stable/newton: Zuul: Remove project name  https://review.openstack.org/54258110:09
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-galera_client master: Set proper yum repo priority on CentOS 7  https://review.openstack.org/54276210:15
tescrano file for ansible /var/log$ ls -a .                 auth.log       dmesg     installer  ntpstats ..                bootstrap.log  dpkg.log  kern.log   syslog alternatives.log  btmp           faillog   lastlog    unattended-upgrades apt               dist-upgrade   fsck      lxd        wtmp10:16
tescraalso my session terminates before i can see play recap10:17
openstackgerritMerged openstack/openstack-ansible-rabbitmq_server stable/pike: Zuul: Remove project name  https://review.openstack.org/54253310:17
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_nova master: tests: test-nova-upgrades.sh: Drop Ansible verbosity  https://review.openstack.org/53737910:20
Miougetescra: are you runnining that in a screen?10:20
*** Smeared_Beard has quit IRC10:22
openstackgerritMerged openstack/openstack-ansible-os_swift stable/ocata: Zuul: Remove project name  https://review.openstack.org/54255810:22
*** vnogin has joined #openstack-ansible10:25
odyssey4meTaseer ok, it looks like a good start there in the os_congress role - a few things to clean up and update... the non-voting functional test is not a huge concern from my standpoint right now - but I'd like to try and get it at least to a good quality place and up to a minimum standard10:25
odyssey4meevrardjp Taseer given that the os_congress role already has some stuff in it which is broken, we need to try and get something better in soon to replace the current broken, IMO... so Taseer if you could apply my suggestions then we should be good to go and can work on making it functional afterwards in subsequent patches10:26
tescrashall  i execute setupeverything instead of setup-hosts.yml10:33
tescrasame thing repeated 3thrice..10:34
tescrano logs for ansible no play recap10:35
openstackgerritMerged openstack/openstack-ansible-os_monasca-agent stable/pike: Updated from OpenStack Ansible Tests  https://review.openstack.org/54273510:40
*** SmearedBeard has joined #openstack-ansible10:41
Taseerodyssey4me: ok, I will address your comments10:46
*** gkadam_ has quit IRC10:46
*** armaan has quit IRC10:46
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_keystone stable/pike: Zuul: Remove project name  https://review.openstack.org/54277710:50
tescraok: [infra3_cinder_api_container-a9cf1875 -> 10.40.100.23] => (item={'value': {'netmask': '255.255.252.0', 'interface': 'eth2', 'bridge': 'br-storage', 'address': '10.40.244.144', 'type': 'veth'}, 'key': 'storage_address'})10:51
tescrano progress hereafter10:51
tescraTASK [lxc_container_create : Run container veth wiring script]10:52
evrardjptescra: maybe you have networking issue10:53
*** vnogin has quit IRC10:59
*** vnogin has joined #openstack-ansible11:01
*** hw_wutianwei has quit IRC11:02
odyssey4methanks Taseer - apologies for not getting to review that sooner11:02
odyssey4meevrardjp your thoughts please? https://review.openstack.org/#/c/542288/ passed again on a recheck, but it's based on https://review.openstack.org/#/c/541463/ which isn't passing... shall we just consolidate and see how that goes?11:06
odyssey4mehwoarang did you see my comment in https://review.openstack.org/537381 and the others like it?11:08
evrardjpodyssey4me: afk for a few minutes, will review.11:10
openstackgerritManuel Buil proposed openstack/openstack-ansible master: l3_agent should only appear in computes when dvr  https://review.openstack.org/54131011:11
openstackgerritMerged openstack/openstack-ansible-os_nova stable/newton: Revert "Switch the LXD compute driver test to non-voting"  https://review.openstack.org/54137211:16
openstackgerritMerged openstack/openstack-ansible-os_trove master: Updated from OpenStack Ansible Tests  https://review.openstack.org/53644111:16
openstackgerritMerged openstack/openstack-ansible-os_ironic stable/newton: Zuul: Remove project name  https://review.openstack.org/54271311:19
*** olivierbourdon38 has quit IRC11:22
hwoarangodyssey4me: not yet got busy with soem opnfv bits11:23
odyssey4mehwoarang no rush - just letting you know 'cos my vote's hidden behind a +2 :)11:24
*** vnogin has quit IRC11:25
tescraok: [infra3_cinder_api_container-a9cf1875 -> 10.40.100.23] => (item={'value': {'type': 'veth', 'address': '10.40.244.91', 'netmask': '255.255.252.0', 'interface': 'eth2', 'bridge': 'br-storage'}, 'key': 'storage_address'})11:27
evrardjpodyssey4me: imo they don't depend on each other.11:27
evrardjpthey do for your use case11:28
odyssey4meyes, they don't - but that didn't pass until I put them on top of each other11:28
evrardjptescra: please paste the whole log into a pastebin.11:28
odyssey4meI'll rebase to verify11:28
evrardjpI think it's not linked, but like you said, a more insidious cause: a race condition11:29
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/ocata: If debug is enabled, provide verbose output for host discovery  https://review.openstack.org/54228811:29
*** udesale has quit IRC11:35
*** armaan has joined #openstack-ansible11:39
armaanevrardjp: Hello!11:39
*** stuartgr has joined #openstack-ansible11:43
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/pike: Document known issue with galera 10.1.31  https://review.openstack.org/54279011:47
*** vnogin has joined #openstack-ansible11:47
evrardjparmaan: hey11:49
armaanevrardjp: how are you?11:50
evrardjpgood and you ? Fixing the mariadb issues accross all branches, thanks to their terrible packaging!11:50
evrardjplong story short, I'd wait a little before upgrading :(11:50
evrardjpone after another...11:51
armaanevrardjp: struggling with ceph performance issues, damn latency :/11:51
evrardjpoh yeah that's definitely linked.11:51
evrardjpodyssey4me: what do you think of https://review.openstack.org/#/c/542790/ ?11:51
evrardjpalso, me terribad with rst.11:52
taccohi there11:52
evrardjphello tacco11:52
armaanevrardjp: ok then, i will wait :), thanks for letting me know...11:52
evrardjpwhen I pinged you, you got a bell?11:52
evrardjpthat's a terrible joke.11:53
armaan:P11:53
evrardjparmaan: yeah, pay attention to your upgrades too11:53
evrardjpif you're doing package updates on your system --- meh! you lose!11:53
taccoyeah this mariadb xtrabackup stuff is always a bit unstable thats why i would prefer to use rsync repl. method even if this is a bit less performant11:53
odyssey4meevrardjp mind if I tweak it?11:53
evrardjpgo ahead.11:53
evrardjptacco: yeah, and you get read locks11:54
armaanevrardjp: ohh, i will take extra care!11:54
taccoyes11:54
*** sar has joined #openstack-ansible11:54
evrardjpwhich is no go for many ppl. And on top of it, it's not recommended by mariadb ppl :)11:54
evrardjpbut yes, at least rsync works :)11:54
taccoyes but if you are in big trouble, you better have some read locks instead of a dead cluster11:55
tacco:D11:55
evrardjpfair11:55
evrardjpI just think they shouldn't have released a broken thing.11:55
evrardjp:p11:55
taccotrue11:55
evrardjpodyssey4me: building the docs on my local machine to see what it looks like11:56
evrardjphope they're gonna be pretty.11:56
evrardjpMaybe I should get a crash course to writing rst instead of md11:56
evrardjpmd is so convenient11:56
evrardjptacco: oh btw11:58
evrardjpWELCOME!11:59
evrardjp(you have to hear the brasilian party music in the background)11:59
taccothanks a lot11:59
evrardjpodyssey4me: FYI tacco is a colleague of jafeha :)11:59
evrardjpwhat a good team!12:00
taccohope that we can contribute some good stuff12:00
evrardjpthat's already started, jafeha helped people quite a lot in this chan, plus we have the scenario for ovs on the way12:01
taccogreat12:03
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/pike: Document known issue with galera 10.1.31  https://review.openstack.org/54279012:04
odyssey4meevrardjp fixed up12:04
*** sar has quit IRC12:04
odyssey4meevrardjp in future, https://docs.openstack.org/reno/latest/user/usage.html#note-file-syntax is helpful :)12:05
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/pike: Document known issue with galera 10.1.31  https://review.openstack.org/54279012:05
odyssey4mekilled the whitespace12:05
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Integrate OVS AIO Scenario  https://review.openstack.org/53431812:07
evrardjpodyssey4me: bookmarked!12:08
hwoarangodyssey4me: ok got your comments now i am going to update the open requests12:08
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_swift master: tests: test-swift-upgrades.sh: Drop Ansible verbosity  https://review.openstack.org/53738112:09
evrardjpbut ara doesn't show the file loading order easily!12:10
*** sar has joined #openstack-ansible12:11
evrardjp:)12:11
evrardjpodyssey4me: thanks for the work done in the last days on metal.12:11
openstackgerritMerged openstack/openstack-ansible-os_neutron master: tasks: Ensure Open vSwitch is started for all providers that need it  https://review.openstack.org/53893312:17
*** rodolof has joined #openstack-ansible12:19
*** acormier has joined #openstack-ansible12:26
*** acormier has quit IRC12:27
*** acormier has joined #openstack-ansible12:27
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Ceph RadosGW integration  https://review.openstack.org/51785612:29
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Ceph RadosGW integration  https://review.openstack.org/51785612:29
*** acormier has quit IRC12:31
odyssey4meevrardjp perhaps that's best given as feedback to dmsimard ;)12:34
odyssey4me(the file load ordering)12:34
odyssey4meevrardjp and sure - happy to have helped, and very glad we have that example demonstrated now...12:35
evrardjpyup, will do it :)12:35
evrardjpI think we definitely improved coverage this cycle, which allow us great refactors in the future.12:36
*** hamza21 has quit IRC12:41
*** serverascode has quit IRC12:48
*** serverascode has joined #openstack-ansible12:49
*** rodolof has quit IRC12:54
*** armaan has quit IRC12:56
evrardjpodyssey4me: has super fancy rst skillz12:57
openstackgerritMerged openstack/openstack-ansible-os_nova stable/newton: Zuul: Remove project name  https://review.openstack.org/54274712:59
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible stable/ocata: Document known issue with galera 10.1.31  https://review.openstack.org/54280013:01
evrardjpodyssey4me: ^ ocata one13:01
evrardjpdoing the newtonz now13:01
evrardjpoh there is no newtonz.13:02
evrardjpgood!13:02
*** acormier has joined #openstack-ansible13:05
openstackgerritMerged openstack/openstack-ansible-os_keystone stable/pike: Zuul: Remove project name  https://review.openstack.org/54277713:06
*** acormier has quit IRC13:06
*** acormier has joined #openstack-ansible13:06
openstackgerritPeriyasamy Palanisamy proposed openstack/openstack-ansible master: Make Opendaylight as the BGP speaker using Quagga  https://review.openstack.org/52390713:07
openstackgerritMaxime Guyot proposed openstack/openstack-ansible master: Ceph RadosGW integration  https://review.openstack.org/51785613:09
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_trove stable/pike: Zuul: Remove project name  https://review.openstack.org/54281113:11
*** fdegir has quit IRC13:14
*** fdegir has joined #openstack-ansible13:15
*** kklimonda has quit IRC13:15
*** kklimonda has joined #openstack-ansible13:16
*** woodard has quit IRC13:18
*** woodard has joined #openstack-ansible13:18
mgariepymorning everyone13:18
mhaydenmorning13:18
evrardjpmorning mgariepy mhayden13:19
evrardjpmnaser: for gnocchi that needs ceph, it's probable that ceph bindings aren't installed into the venv13:19
mnaserevrardjp: so how should I add it?  Any other role to follow?13:19
evrardjpfor example for cinder we do this to install basic ceph stuff on the container: https://github.com/openstack/openstack-ansible-os_cinder/blob/f807cdccfbc148fc335a2fd89d13ec8ea85aa6ba/tasks/main.yml#L79-L8913:20
evrardjpthen I am not sure where we have an example for linking the ceph c bindings13:20
evrardjpmhayden: you played with that recently, right?13:21
evrardjpmnaser: then, if you need cradox, just add cradox under the list of packages to install with pip, if that's a pip package13:21
evrardjpmnaser: oh the linking is done in the role already: https://github.com/openstack/openstack-ansible-ceph_client/blob/master/tasks/ceph_install_python_libs.yml#L26-L3313:22
mnaserevrardjp: right but in developer mode install on the gate wouldnā€™t that not matter?13:22
evrardjpit wouldn't matter for being built, but you still have to stay to install the package if it's not in pip package's setup.py/requirements.13:23
evrardjpI am not aware of how gnocchi does it13:23
evrardjplet me quickly check13:23
evrardjpmhayden: do you know why we aren't using RDO packages for mariadb?13:24
openstackgerritMerged openstack/openstack-ansible-os_ceilometer master: fix misspelling of 'configuration' and 'list'  https://review.openstack.org/53419513:25
mhaydenevrardjp: i was using the mariadb repos for centos 7 so that it would match the other distros13:26
mhaydenRDO's mariadb is a little older13:26
mhaydeni was worried about configuration conflicts and such13:26
mhaydendid we ever get to the bottom of what's up with 10.1.31? or is it just wait and see13:27
evrardjpI stopped this morning. I was into deep, but I had to stop for other things.13:28
evrardjpI can hand that over to you, or we can do a debugging session together.13:28
openstackgerritMerged openstack/openstack-ansible-openstack_hosts master: Ensure openstack_hosts sets nodepool variables  https://review.openstack.org/54235213:29
evrardjpmhayden: I did like you did, asking #maria. didn't really help13:29
odyssey4meevrardjp looks like it might be a race - the same patch, two different attempts: http://logs.openstack.org/88/542288/1/check/openstack-ansible-upgrade-ubuntu-xenial/7a05b52/job-output.txt.gz#_2018-02-08_17_15_22_049899 and http://logs.openstack.org/88/542288/2/check/openstack-ansible-upgrade-ubuntu-xenial/6ddf7fe/job-output.txt.gz#_2018-02-08_19_57_37_17089713:29
evrardjpI don't like this.13:30
evrardjpintroducing a sleep 20 is the solution!13:30
evrardjpoh but it's in tempest13:31
odyssey4meevrardjp we already have a form of a sleep13:31
odyssey4mewe wait until the compute node registers13:31
odyssey4meso clearly, for an upgrade, this is not enough13:31
odyssey4meI'll keep looking into it.13:31
odyssey4meand no, this is not in tempest - tempest fails because there are no compute nodes in cell1 so it can't schedule the instance build and fails13:32
evrardjpoh I might have an idea13:32
evrardjpis it from O to P?13:33
evrardjpor from N to O ?13:33
odyssey4meN to O13:33
odyssey4methis is the 'sleep': https://github.com/openstack/openstack-ansible-os_nova/blob/stable/ocata/handlers/main.yml#L67-L7613:34
evrardjpodyssey4me: could you display, in your tests, the result of a task  shell: "{{ _db_nova_bin }}/nova-manage cell_v2 list_cells  ?13:37
*** dariko has joined #openstack-ansible13:37
odyssey4mecould do - but the output there shows the two cells already, cell0 and cell113:39
odyssey4mesee http://logs.openstack.org/88/542288/2/check/openstack-ansible-upgrade-ubuntu-xenial/6ddf7fe/job-output.txt.gz#_2018-02-08_19_57_37_170711 and http://logs.openstack.org/88/542288/1/check/openstack-ansible-upgrade-ubuntu-xenial/7a05b52/job-output.txt.gz#_2018-02-08_17_15_22_04940513:40
*** kstev has joined #openstack-ansible13:40
*** mubix has quit IRC13:41
evrardjpodyssey4me: yeah I wasn't on the right line, it took a while to load13:42
*** mubix has joined #openstack-ansible13:42
*** wspthr has quit IRC13:42
*** kstev has quit IRC13:44
*** wspthr has joined #openstack-ansible13:45
*** kstev has joined #openstack-ansible13:46
*** dave-mccowan has joined #openstack-ansible13:46
*** dave-mcc_ has joined #openstack-ansible13:54
*** dave-mccowan has quit IRC13:55
*** hamza21 has joined #openstack-ansible13:58
*** acormier has quit IRC14:01
*** acormier has joined #openstack-ansible14:02
*** johnsom has quit IRC14:03
*** johnsom has joined #openstack-ansible14:04
*** acormier_ has joined #openstack-ansible14:05
*** acormier has quit IRC14:06
openstackgerritJames E. Blair proposed openstack/openstack-ansible-memcached_server stable/pike: Zuul: Remove project name  https://review.openstack.org/54284214:11
openstackgerritMerged openstack/openstack-ansible-os_swift master: tests: test-swift-upgrades.sh: Drop Ansible verbosity  https://review.openstack.org/53738114:12
*** vnogin has quit IRC14:12
*** wspthr has quit IRC14:17
*** dave-mcc_ has quit IRC14:17
*** vnogin has joined #openstack-ansible14:18
openstackgerritMaxime Guyot proposed openstack/openstack-ansible master: Ceph RadosGW integration  https://review.openstack.org/51785614:19
* mnaser is in the office again14:19
*** adcpdk has joined #openstack-ansible14:19
mnaserokay so, os_gnocchi installs librados2-devel14:20
*** wspthr has joined #openstack-ansible14:20
mnaserand centos ships hammer release if we don't install storage sig14:20
*** acormier_ has quit IRC14:21
*** acormier has joined #openstack-ansible14:22
adcpdkidlemind thank you so much for such a nice troubleshooting yesterday. I finally got my networking setup and working as a flat network! Such a nice feeling when there are people out there to help and go out with you and fight with monsters you were scared of))) It was my first experience of using IRC channels in my life and I won't forget that. I will try to do my best to be around to help the newbies like me ;-)14:23
mnaserso my small question is, what role can i add to dependencies to make sure it adds the ceph modern repos14:25
openstackgerritMerged openstack/openstack-ansible-os_ceilometer master: Updated from OpenStack Ansible Tests  https://review.openstack.org/51890914:25
*** dave-mccowan has joined #openstack-ansible14:29
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_nova stable/ocata: Zuul: Remove project name  https://review.openstack.org/54285114:31
tescrahi my userconfig file is ok asper yamllint but openstack-ansible --syntax-check shows no plays14:39
*** dave-mccowan has quit IRC14:40
openstackgerritMerged openstack/openstack-ansible stable/ocata: Fix logic to check for insecure Keystone  https://review.openstack.org/53930514:43
*** berendt has quit IRC14:46
*** berendt has joined #openstack-ansible14:46
mnaserhttps://github.com/openstack/openstack-ansible/blob/master/playbooks/ceph-install.yml#L35-L5214:51
mnaserdoes this happen in another role?14:51
*** hogepodge has quit IRC14:51
mnaseror should i add that to the ceph gnocchi role?14:51
*** hogepodge has joined #openstack-ansible14:52
*** acormier has quit IRC14:57
*** adrianreza has quit IRC14:57
*** adrianreza has joined #openstack-ansible14:57
*** sar has quit IRC14:58
*** dave-mccowan has joined #openstack-ansible15:03
*** vadim_ has joined #openstack-ansible15:04
*** rodolof has joined #openstack-ansible15:04
*** acormier has joined #openstack-ansible15:04
*** acormier has quit IRC15:06
*** acormier has joined #openstack-ansible15:06
*** bhujay has quit IRC15:07
logan-mnaser: https://github.com/openstack/openstack-ansible-ceph_client/blob/master/tasks/ceph_preinstall_yum.yml probably needs to be updated15:10
ashakI tried running a bunch of the OSA playbooks earlier after I made a change to some variable that I hoped would allow me to be able to upload image files via horizon directly to glance... but during that run, something tried to restart mariadb on one of my two galera containers... Since then i've failed to be able to recover that galera container. I followed everything here15:10
ashakhttps://docs.openstack.org/openstack-ansible/newton/developer-docs/ops-galera-recovery.html (because I couldn't find a similar one for ocata or pike). But whatever happens mariadb is never able to start on the container it failed on.15:10
ashakEven when that entire container is rebuilt15:10
*** rama_y has joined #openstack-ansible15:10
vadim_hi, I think https://github.com/openstack/openstack-ansible-galera_server/commit/f2bfbd38513ac8d61ba4e02a4d5ef6cbbca259cc commit is missing from ocata branch15:11
cloudnullashak: there was recently an issue with the mariadb (galera) package which broke galera clustering15:12
cloudnullrecent prs went in to fix that15:12
cloudnullbut you'll have to pull in the pr using the branch instead of using a tag for now15:13
cloudnullthat's the pr to fix it15:13
cloudnullhttps://github.com/openstack/openstack-ansible-galera_server/commit/8a42facf6b98f77ef3cd04ca619e3c5173ab39e915:13
cloudnullfor the server15:13
cloudnullhttps://github.com/openstack/openstack-ansible-galera_client/commit/3524b8352f0728b2955a002540a4fc464615913215:13
cloudnullfor the client15:13
ashakcloudnull: arse. Ok, thanks15:14
*** olivierbourdon38 has joined #openstack-ansible15:16
mnaserlogan-: i guess i can include_role: from_tasks: there15:16
logan-right15:17
logan-thats how glance/cinder/nova work right now15:17
logan-https://github.com/openstack/openstack-ansible-os_nova/blob/9e44a4dd6dd8cdef984d61afc8b68cffc45472ea/tasks/main.yml#L112-L12315:17
logan-and https://github.com/openstack/openstack-ansible-os_cinder/blob/master/tasks/main.yml#L79-L8915:17
logan-for example15:17
logan-it looks like gnocchi does the ceph_client include at the playbook level https://github.com/openstack/openstack-ansible/blob/8224fbcede862078d2a3914863f06339ca6c85cb/playbooks/os-gnocchi-install.yml#L50-L5715:18
mnaserlogan-: i guess that is the problem, it needs to happen in the role or otherwise the functional testing is failing15:19
mnaserill get it moved into the role and out of openstack-ansible15:20
*** chhagarw has quit IRC15:20
logan-also interesting that no venv path is being sent into ceph_client for linking15:20
*** chhagarw has joined #openstack-ansible15:20
mnaserlogan-: the thing is i think ceph_client won't do anything if no ceph_mons are defined15:20
logan-yeah15:21
vadim_yes, socket path needs to be fixed even for .30 mariadb15:21
mnaserso gnocchi absolutely needs cradox even if you don't use ceph15:21
mnaser:<15:21
logan-i tihnk there might be a workaround for that if you feed it a ceph.conf dump directly?15:21
logan-i can't remember15:21
logan-but it normally copies ceph.conf from one of the mons15:21
mnaserlogan-: right but in the gate functional testing we're installing gnocchi with file storage15:21
mnaserjust to test if the role is working ok15:22
logan-ah15:22
mnaserim thinking include_role with the tasks from here https://github.com/openstack/openstack-ansible-ceph_client/blob/master/tasks/main.yml#L27-L3315:22
mnaserthat will make sure whatever dependencies are in15:22
logan-hmmm yeah if gnocchi needs some package regardless of whether ceph is enabled then this doesnt sound like a ceph_client change/issue15:22
mnaserbecause the os_gnocchi role installs librados itself so i guess it should make sure that it has fresh repos too15:23
mnaserif you think of it independently15:23
mgariepyvadim_, you are probably better deploying pike on centos.15:23
vadim_why?15:24
mgariepyocata was the first release that ""supported"" centos for most of the openstack-service. why do you want to deploy ocata ?15:24
vadim_it's deployed already. and I plan to upgrade to pike soon.15:25
mgariepyho15:25
mgariepyout of curiosity how is it going ?15:26
*** chhagarw has quit IRC15:26
vadim_15.1.10 was ok for a while. 15.1.14 broke galera. 15.1.15 did not fix it. mariadb 31 broke galera again;) overall ok15:28
*** dariko has quit IRC15:28
*** dariko has joined #openstack-ansible15:29
mnaserok15:30
mnaserfound it.15:30
mnaserhttps://github.com/openstack/openstack-ansible-os_gnocchi/blob/master/defaults/main.yml#L166-L17215:31
mnaserthe ",ceph" there should be dependent if the storage driver is ceph15:31
mnaseri think i'll refactor it so that it only installs the extra which it needs (ex: ceph or swift or file depending on driver)15:31
openstackgerritJames E. Blair proposed openstack/openstack-ansible stable/pike: Zuul: Remove project name  https://review.openstack.org/54288815:32
tescrahi now syntax check says dynamic inventory Failed to parse /opt/openstack-ansible/inventory/dynamic_inventory.py with script plugin: Inventory script (/opt/openstack-ansible/inventory/dynamic_inventory.py) had an execution error: Traceback (most recent call last): File "/opt/openstack-ansible/inventory/dynamic_inventory.py", line 80, in <module>     output = generate.main(**all_args)   File "/opt/openstack-ansible/osa_tool15:35
tescrano host found15:36
openstackgerritMarc GariĆ©py (mgariepy) proposed openstack/openstack-ansible-galera_server stable/ocata: Fix Galera socket for RedHat  https://review.openstack.org/54289415:44
mgariepyvadim_, can you test this patch ^^15:45
vadim_sure15:47
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_gnocchi master: Install only the required storage driver extras  https://review.openstack.org/54289815:49
mgariepythanks15:49
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_gnocchi master: Switch to Keystone extra  https://review.openstack.org/54290015:50
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_aodh master: Update the Tempest plugin to point towards the new plugin  https://review.openstack.org/54127015:50
ashakcloudnull: /me grumbles, seems I cannot cherry-pick those commits into my current checkout of tag 16.0.7 for each of those ansible roles due to conflicts. Checking out master for each role contains them but when I run the OSA playbooks it causes other problems.15:54
ashakShould I be doing it some other way? Or are these commits likely to make it into a new tag soon enough that I can just let OSA bootstrapping take care of it for me?15:56
cloudnullyou could cherry pick that change using something like `git fetch https://git.openstack.org/openstack/openstack-ansible refs/changes/90/542790/3 && git cherry-pick FETCH_HEAD`15:59
cloudnullfrom here https://review.openstack.org/#/c/542790/15:59
cloudnullor just make those two changes in the role requirements file15:59
ashakoh I see15:59
ashakI assumed you'd meant patch the two roles directly16:00
cloudnullyou could do either16:00
cloudnullI'd just pull the updated roles down wholesale16:00
ashakI did, they don't work :)16:00
cloudnullthe updated roles didn't work ?16:00
ashakfatal: [infra1_galera_container-02a31604]: FAILED! => {"changed": false, "failed": true, "msg": "missing required arguments: name"}16:00
ashakin TASK [galera_server : Reload systemd service files]16:01
*** pcaruana has quit IRC16:01
ashakBut I checkout out master for both roles16:01
ashakchecked out16:01
*** dariko has quit IRC16:01
cloudnullafter a fetch ?16:01
*** dariko1 has joined #openstack-ansible16:01
ashakYes16:01
cloudnullyou may be too far ahead using the HEAD of master.16:01
cloudnullfor pike16:02
odyssey4megit checkout master; git pull; ./scripts/bootstrap-ansible.sh16:02
odyssey4meif you want to use pike, then just grab https://review.openstack.org/542790 - or wait for it to merge16:02
ashakwhen might it merge?16:02
odyssey4mein the next 90 mins or so if all goes well16:03
odyssey4meyou can also just use your current pike, make the changes to the ansible-role-requirements.yml file in that review, then bootstrap-ansible and go16:03
ashakOk, thanks16:05
tescraroot@deploy-node:/opt/openstack-ansible# git branch -a   remotes/origin/stable/pike * stable/pike16:05
tescradynamic inventory Failed to parse16:06
tescranone of playbook is executing16:07
tescrashall i run boothstrap again16:07
prometheanfireevrardjp: when requirements branches/unfreezes you will start getting rocky requirements updates to your master(queens) branch, either don't merge them or get a +1 from a requirements core (please)16:08
odyssey4meevery time you change the git ref, ansible should be bootstrapped again - otherwise you're not using the role designed to work with the repo16:08
tescraodyssey4me is that ofr me?16:08
odyssey4metescra yes16:08
*** tux_ has joined #openstack-ansible16:09
odyssey4metescra flipping from master to pike is not supported though16:09
evrardjp[m]prometheanfire: yup i know.16:09
evrardjp[m]the usual16:09
prometheanfireevrardjp[m]: cool, just was looking for an ack before branching16:10
tescraodyssey4me where shall i place the git head16:10
tescrai started with pike16:10
evrardjp[m]it's today ?16:10
*** jamesdenton has joined #openstack-ansible16:10
evrardjp[m]wow16:10
odyssey4metescra what are you trying to deploy?16:10
tescraproduction env16:10
tescrawith pike16:10
tescraon bare metal16:10
odyssey4meif you're trying to deploy pike, then use stable/pike and add that patch on top of it16:11
tescrahow?16:11
odyssey4megit checkout stable/pike; git fetch https://git.openstack.org/openstack/openstack-ansible refs/changes/90/542790/3 && git cherry-pick FETCH_HEAD16:11
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_designate stable/pike: Zuul: Remove project name  https://review.openstack.org/54290916:12
tescrathanks16:12
*** armaan has joined #openstack-ansible16:13
odyssey4metescra - perhaps better is: git fetch --all; git checkout origin/stable/pike; git fetch https://git.openstack.org/openstack/openstack-ansible refs/changes/90/542790/3 && git cherry-pick FETCH_HEAD16:13
tescrait syas Already on 'stable/pike' remote: Counting objects: 18, done. remote: Compressing objects: 100% (10/10), done. remote: Total 11 (delta 7), reused 2 (delta 1) Unpacking objects: 100% (11/11), done. From https://git.openstack.org/openstack/openstack-ansible  * branch            refs/changes/90/542790/3 -> FETCH_HEAD error: could not apply 0e0819b... Document known issue with galera 10.1.31 hint: after resolving the co16:13
*** olivierbourdon38 has quit IRC16:13
tescraUnpacking objects: 100% (82/82), done. From https://git.openstack.org/openstack/openstack-ansible    6a906e7..8224fbc  master     -> origin/master    eb357e2..30301c2  stable/ocata -> origin/stable/ocata    00d84d9..01368fc  stable/pike -> origin/stable/pike ansible-role-requirements.yml: needs merge error: you need to resolve your current index first From https://git.openstack.org/openstack/openstack-ansible  * branch16:15
tescraansible-role-requirements.yml: needs merge error: you need to resolve your current index first From https://git.openstack.org/openstack/openstack-ansible  * branch            refs/changes/90/542790/3 -> FETCH_HEAD error: cherry-pick is not possible because you have unmerged files. hint: Fix them up in the work tree, and then use 'git add/rm <file>' hint: as appropriate to mark resolution and make a commit. fatal: cherry-pi16:16
odyssey4metescra remove your customisations16:19
odyssey4megit reset --hard origin/stable/pike16:19
*** vnogin has quit IRC16:20
odyssey4megit fetch --all; git reset --hard origin/stable/pike; git fetch https://git.openstack.org/openstack/openstack-ansible refs/changes/90/542790/3 && git cherry-pick FETCH_HEAD16:21
tescraodyssey4me op was HEAD is now at 01368fc Use native packages to deploy keepalived16:21
odyssey4menow cherry-pick on top16:21
tescrashall i go for cherrypick cmd?16:21
tescranow?16:22
odyssey4mebut that's old too16:22
tescrak16:22
tescra??16:22
tescraold?16:22
odyssey4meah no, 01368fc is the head16:23
*** weezS has joined #openstack-ansible16:23
odyssey4meso cherry-pick on top of that16:23
*** vnogin has joined #openstack-ansible16:23
ashakodyssey4me: I did that but now Get Modern PIP is failing with trying to download http://172.29.236.3:8181/os-releases/16.0.8/ubuntu-16.04-x86_64/get-pip.py from my repo containers. Am I supposed to run something else after the bootstrap script to update the repo containers?16:23
odyssey4meashak yes, you'll need to now run setup-hosts.yml, setup-infrastructure.yml; setup-openstack.yml... you're effectively doing a minor upgrade now16:24
tescraHEAD is now at 01368fc Use native packages to deploy keepalived root@deploy-node:/opt/openstack-ansible# git fetch https://git.openstack.org/openstack/openstack-ansible refs/changes/90/542790/3 && git cherry-pick FETCH_HEAD From https://git.openstack.org/openstack/openstack-ansible  * branch            refs/changes/90/542790/3 -> FETCH_HEAD  *** Please tell me who you are.  Run    git config --global user.email "you@exampl16:24
tescrais that ok?16:25
ashakodyssey4me: oh of course16:25
odyssey4mehttps://docs.openstack.org/openstack-ansible/pike/user/minor-upgrade.html16:25
tescra git config --global user.name "Your Name"  to set your account's default identity. Omit --global to set the identity only in this repository.  fatal: unable to auto-detect email address (got 'root@deploy-node.(none)')16:25
ashakDamn, that's going to take ages16:25
odyssey4metescra no, your git is not configured so it hasn't done the pick16:25
odyssey4meyou need to configure git16:25
tescraby?16:26
odyssey4metescra I'm sure that google can help you.16:26
tescraproviding email?16:26
tescraok16:26
odyssey4methe stderr is telling you what to do already16:27
tescraonly for deploy host or all servers?16:27
odyssey4mejust deploy host, and only because you're cherry-picking a change that hasn't merged16:27
ashakYou guys must love us untrained people and our inabilities to work some of this stuff out/find the correct documents :)16:28
odyssey4methat's where we all started ;)16:28
ashakI'm still struggling to work out where you got refs/changes/90/542790/3 to use with the git fetch command you suggested16:29
odyssey4metescra ashak the head of the branch is the proposed next release, we test it for two weeks before releasing it... so be aware that you're trying to use the bleeding edge ;)16:30
ashakI constantly feel like i'm using the bleeding edge with this stuff ;)16:30
odyssey4meashak if you open https://review.openstack.org/542790 and select 'download' on the top right16:30
odyssey4menext to 'cherry-pick' is the text showing how to cherry-pick it, and next to that is a 'copy to clipboard icon'16:31
ashakodyssey4me: How would I have worked that out from the two commits you linked originally?16:31
dcdamienHi! I'm doing first fresh deploy after a break and noticed that os_cinder tries to install open-iscsi package on api *container*16:31
mnaser:O16:31
mnaseros_aodh successfully installed os_gnocchi on centos.. is this finally it..16:32
odyssey4meashak well, you'd have to watch the review queue in this case... but if you look at https://github.com/openstack/openstack-ansible/blob/stable/pike/ansible-role-requirements.yml#L16 you'll see the 'version' value is a git SHA16:32
ashakJeez.... i'm never going to remember all this :)16:32
odyssey4meif you then look at https://github.com/openstack/openstack-ansible-galera_client/commits/stable/pike (the repo being referred to), you'll find that sha there16:32
odyssey4methat's how a role is pinned for a release16:33
logan-anyone have an upstream bug link handy for the mariadb 10.1.31 galera issue16:33
odyssey4meif you ever want to use a different SHA, then you change https://github.com/openstack/openstack-ansible/blob/stable/pike/ansible-role-requirements.yml#L16 to match the SHA you want to use16:33
odyssey4melogan- see https://review.openstack.org/#/c/542790/3/releasenotes/notes/mariadb-10.1.31-package-issue-78816cff4e9f2de9.yaml16:33
ashakOf course. I'd not thought about it from the point of view of OSA itself. Just the two roles with the problem.16:34
logan-ahhh ok thanks16:34
logan-missed the reno.. was looking in the commit msg :)16:34
odyssey4meashak yep - the openstack-ansible repo is an integration of all the roles with pinning of the SHA's so that a tagged release always uses the same thing - the sha's are updated at least every two weeks for every release16:34
dcdamienapt fails on postinst: http://paste.openstack.org/show/666944/ and as I know it's because of open-iscsi can't work properly in the container16:35
ashakodyssey4me: ok, nice one. Thanks for all the info.16:35
odyssey4medcdamien hmm, I think I've seen that before - there's a bug I think16:36
*** sar has joined #openstack-ansible16:36
dcdamienI'm curious how gates passes runs on CI :P16:36
odyssey4mehttps://bugs.launchpad.net/openstack-ansible/+bug/170416116:36
openstackLaunchpad bug 1704161 in openstack-ansible "open-iscsi post-installation script fails in cinder-api-container" [High,Fix committed]16:36
odyssey4meah yes, we pushed up a change in gate and it now tests that continually... so yes, it's curious why it works in gate and not in your env16:37
* ashak yawns, is there a way to speed up running all the things involved in a minor upgrade?16:37
odyssey4meashak you can change the serial values if you want, but then your api will go down as each service updates16:38
evrardjpI am le back!16:38
odyssey4mebut it is a bit slow, and we're hoping to improve that over the next few cycles16:38
ashakthe serial values?16:38
tux_Any idea what has been included in Minor upgrade?  https://docs.openstack.org/openstack-ansible/pike/user/minor-upgrade.html16:39
mnaser"functional: commands succeeded"16:40
mnaser!!!16:40
openstackmnaser: Error: "!!" is not a valid command.16:40
ashakoh, ansible's serial stuff to only allow the changes to happen on a certain percentage of hosts at once?16:40
*** sar has quit IRC16:40
evrardjpashak: yes.16:40
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_aodh master: Zuul: Remove project name  https://review.openstack.org/54117216:42
odyssey4meashak each service has a _serial var which you can use, for example: openstack-ansible setup-openstack.yml -e 'cinder_serial=100%' -e 'glance_serial=100%' -e 'keystone_serial=100%' -e 'nova_serial=100%'16:43
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_aodh master: Updated from global requirements  https://review.openstack.org/53504916:43
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_aodh master: Replace virtualenv-tools by a script  https://review.openstack.org/53364516:43
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_aodh master: Standardize services list and package installation  https://review.openstack.org/49258016:44
odyssey4medcdamien I wonder if it is perhaps working in the gate because that package is installed on the host in the image we use?16:44
openstackgerritMaxime Guyot proposed openstack/openstack-ansible master: Ceph RadosGW integration  https://review.openstack.org/51785616:45
tescraevrardjp  I am at stable pike16:45
ashakodyssey4me: is there a place that lists them all? ;)16:45
tescrabut had root@deploy-node:/opt/openstack-ansible# git fetch https://git.openstack.org/openstack/openstack-ansible refs/changes/90/542790/3 && git cherry-pick FETCH_HEAD From https://git.openstack.org/openstack/openstack-ansible  * branch            refs/changes/90/542790/3 -> FETCH_HEAD On branch stable/pike You are currently cherry-picking commit 0e0819b.16:46
tescrais it stable?16:46
tescrafor prod16:47
odyssey4meashak yes, grep for serial in the integrated repo and explore the code that comes out of it :)16:47
ashakHah16:47
mnasercan i get a second core to have an eye on this? https://review.openstack.org/#/c/541270/ finally got it working, so i can figure out stable branches16:47
mnaser(and the dependencies, tho most have merged)16:48
odyssey4metescra there are over 100 live production deployments using OSA, from juno all the way through to queens (unreleased) - so yes, it's stable for production... as long as you understand it's an open source project which requires feedback and help from its users16:48
dcdamienodyssey4me: Probably. Can it be tested another way to make it similar to real deployment?16:48
tescrathanks odyssesy16:48
odyssey4medcdamien well, if you can confirm if it works with the package installed on the host then we have the issue triaged and can figure out a fix - our tests only use what we're given, so 'production-like tests' require our users to feedback :)16:49
dcdamienodyssey4me: ok, I'll test it16:50
odyssey4methanks dcdamien - it'd be nice to put that one to bed16:50
tescrabut will the previous values of backup_openstack_inventory.tar  be overwritten ? How?16:51
tescramy actuall conern was that the playbooks were unable to connect to dynamic inverntory16:53
*** armaan has quit IRC16:54
tescraall of a sudden  none of the host were reachable16:54
mnaserthanks logan-16:55
mnaserand finally the dependency on this - https://review.openstack.org/#/c/542898/16:55
mnaserjust to kick it into the gate :>16:55
mnaseroh okay i noticed you did a check experimental16:55
mnasermy bad :<16:55
odyssey4metescra not sure what you're talking about, but if the dynamic inventory fails then it's usually a configuration error16:56
logan-mnaser: yeah hopefully check experimental does an integrated build like I think it does16:57
mnaserlogan-: it did according to zuul.o.o16:57
logan-great16:57
odyssey4meIIRC there's a --debug flag, you can run the inventory script on its own, this will do a check of your config: /opt/ansible-runtime/bin/python /opt/openstack-ansible/playbooks/inventory/dynamic_inventory.py --check16:57
odyssey4me--debug will do debug information16:57
logan-just want to make sure the py_pkgs lookup doesnt fumble the package name due to the templating. https://github.com/openstack/openstack-ansible/blob/8224fbcede862078d2a3914863f06339ca6c85cb/playbooks/repo-build.yml#L56-L6216:58
*** armaan has joined #openstack-ansible16:58
*** armaan has quit IRC16:59
*** armaan has joined #openstack-ansible17:00
tescrano logs for ansible no play recap17:00
tescraoddessy4me17:04
tescra File "/opt/openstack-ansible/playbooks/inventory/dynamic_inventory.py", line 80, in <module>     output = generate.main(**all_args)   File "/opt/openstack-ansible/osa_toolkit/generate.py", line 1141, in main     user_defined_config   File "/opt/openstack-ansible/osa_toolkit/generate.py", line 747, in container_skel_load     address_prefix=p_net.get('address_prefix')   File "/opt/openstack-ansible/osa_toolkit/generate.py",17:06
tescraon checking dynamic nventory17:06
tescrait saya infr4 but am having 3 infra only17:07
odyssey4melogan- mnaser yep, that'll be a problem17:07
logan-odyssey4me: yea im thinking the same thing re: group_vars/all making this go17:07
odyssey4metescra check your config - and also, perhaps you previously had other entries... you can use inventory-manage.py to remove hosts from your inventory, or hack them out manually... or if this is a new environment and nothing's deployed you can delete it and start fresh17:08
ashakodyssey4me: Meh, even with the updates, restart mysql is failing on the galera container on my first infra node. I guess it's somehow got itself into a broken state that these changes can't fix17:08
*** rama_y has quit IRC17:08
odyssey4melogan- mnaser it probably makes sense to set that var in group_vars/all anyway, because group_vars understand the inventory, and can therefore provide the list of the right things to add to it17:09
tescraodyssey4me thanks17:09
ashakGuess i'd best learn a whole bunch more about galera17:09
odyssey4meashak if you had already installed, you'd have to remove first and reinstall - or just wipe the ocntainers if this is a new build17:09
odyssey4meotherwise, yes - get to know more about galera :)17:09
*** sar has joined #openstack-ansible17:09
ashakit's reasonably new and mostly in testing. But i'm trying to treat it like a production system because it always helps to now what fun you might face when it's actually in use :)17:10
mnaserodyssey4me: logan- im a bit confused now so what should i be doing ideally for this?17:10
*** gameon has quit IRC17:13
openstackgerritMerged openstack/openstack-ansible stable/ocata: Document known issue with galera 10.1.31  https://review.openstack.org/54280017:13
*** vnogin has quit IRC17:15
*** vnogin has joined #openstack-ansible17:16
*** weezS has quit IRC17:17
*** weezS has joined #openstack-ansible17:19
openstackgerritMaxime Guyot proposed openstack/openstack-ansible master: Ceph RadosGW integration  https://review.openstack.org/51785617:20
*** peri has quit IRC17:23
*** vnogin has quit IRC17:25
dmsimardevrardjp, mnaser: btw I know you already know that but feel free to reach out if you'd like help on RDO things17:31
mnaserthanks dmsimard :D17:31
*** armaan has quit IRC17:35
*** armaan has joined #openstack-ansible17:36
*** armaan has quit IRC17:37
*** armaan has joined #openstack-ansible17:38
*** rama_y has joined #openstack-ansible17:41
*** rama_y has quit IRC17:42
*** rama_y has joined #openstack-ansible17:42
eumel8evrardjp: still around?17:42
openstackgerritMerged openstack/openstack-ansible stable/pike: Document known issue with galera 10.1.31  https://review.openstack.org/54279017:45
openstackgerritLogan V proposed openstack/openstack-ansible master: Add gnocchi_storage_driver var to all group  https://review.openstack.org/54295717:48
*** mbuil has quit IRC17:49
openstackgerritLogan V proposed openstack/openstack-ansible master: Add gnocchi_storage_driver var to all group  https://review.openstack.org/54295717:50
*** shardy has quit IRC17:51
evrardjpeumel8: I am sorry I have to go for today. I am not forgetting the simplification of the job, in fact, something just merged to improve it.17:54
evrardjpdmsimard: :D17:55
evrardjpsee you everyone online next week!17:55
eumel8evrardjp: alright, was only just wondering that the job now runs successful: http://zuul.openstack.org/builds.html?job_name=openstack-ansible-deploy-translations-ubuntu-xenial17:56
*** dave-mccowan has quit IRC17:57
*** dave-mccowan has joined #openstack-ansible17:58
*** weezS has quit IRC18:01
*** vnogin has joined #openstack-ansible18:05
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_gnocchi master: Install only the required storage driver extras  https://review.openstack.org/54289818:06
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_gnocchi master: Switch to Keystone extra  https://review.openstack.org/54290018:06
mnaserrebased with depends-on logan- ^18:07
logan-cool, hopefully that does the trick18:07
*** vnogin has quit IRC18:09
cloudnullo/ logan-18:09
cloudnullhows it ?18:09
odyssey4melogan- mnaser made a suggesiton in https://review.openstack.org/#/c/542898/218:10
logan-hey cloudnull. always a good day when its friday :)18:12
logan-how about you18:12
cloudnullsame.18:12
cloudnullfriday fun day :)18:12
mnaserhey thats a good call odyssey4me18:13
mnaservoila18:14
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_gnocchi master: Install only the required storage driver extras  https://review.openstack.org/54289818:14
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_gnocchi master: Switch to Keystone extra  https://review.openstack.org/54290018:14
mnaserhttps://github.com/openstack/openstack-ansible-os_keystone/blob/master/tasks/keystone_pre_install.yml#L16-L23 -- could someone explain whats the point of delegate_to + with_items as ansible_play_hosts ?18:15
*** olivierbourdon38 has joined #openstack-ansible18:15
mnasersounds like delegate_to, with_items and the when can be dropped to accomplish the same exact thing?18:15
logan-it is there to allow serialization of the playbook18:16
logan-https://github.com/openstack/openstack-ansible-os_keystone/commit/86c42e0697049757641656dac60cb9a2e00159f518:16
mnaserah ok18:16
mnaserfeels so dirty :<18:16
logan-yeah18:16
*** SerenaFeng has joined #openstack-ansible18:20
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249118:25
*** acormier has quit IRC18:28
*** acormier has joined #openstack-ansible18:29
*** acormier_ has joined #openstack-ansible18:32
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_aodh stable/pike: Zuul: Remove project name  https://review.openstack.org/54297118:32
*** acormier has quit IRC18:33
*** acormier_ has quit IRC18:36
mnaseralso whats the reasoning behind the log dir link/dir checks18:38
openstackgerritMerged openstack/openstack-ansible-os_nova master: Avoid explicit requirement for groups to exist  https://review.openstack.org/53777418:51
*** chyka has joined #openstack-ansible18:52
*** stuartgr has left #openstack-ansible18:53
*** chyka has quit IRC18:56
openstackgerritMerged openstack/openstack-ansible-tests stable/pike: test-set-nodepool-vars: Add nodepool mirror discovery for openSUSE  https://review.openstack.org/54245618:57
odyssey4melogan- mnaser unfortunately the integrated build won't actually verify the whole stack build there - you'd need a build that deploys gnocci (or forces all wheel to build)18:59
logan-ah true19:00
odyssey4meif you have a gap to test an an AIO logan- please go ahead, otherwise I'll fire something up tomorrow19:00
odyssey4metime for me to !computer19:00
odyssey4memnaser the logic for those relates to when the role is built on a host without containers - there's some weird symlink things we do which puts the folder in /openstack/log instead of /var/log, then we symlink... not sure why - I think it was to put all logs in the same place19:01
mnaserodyssey4me: i see, it's a bit odd, hopefully we can clean that up at some point because it adds to the things that tie up roles to OSA19:02
odyssey4meI would really like us to completely re-think logging/syslog next cycle, but unfortunately I don't have the spare cycles for it.19:02
mnaser++19:02
odyssey4meWe have systemd-journal which could do far better, and we can be rid of a ton of legacy cruft if we figure out a goal we're all keen on.19:02
openstackgerritMerged openstack/openstack-ansible-os_designate stable/pike: Zuul: Remove project name  https://review.openstack.org/54290919:03
ashakodyssey4me: removing the container with the broken instance seems to have fixed it as you suggested19:03
mnaserodyssey4me: and im pretty sure with nspawn, systemd-journal logs can be accessed from the host.. i think19:03
odyssey4memnaser perhaps on interest, I did this patch yesterday which I think is a little cleaner: https://github.com/openstack/openstack-ansible-os_tempest/commit/4b9f1010fe884bfc17f9782b15343187700d27bf19:03
mnasercloudnull: might know more :)19:03
odyssey4memnaser yes they can, which is much more awesome19:03
mnaseri'm hacking around this weekend around adding an 'keystone_installation_type' option, which you can guess does what19:04
odyssey4mebut for our lxc users it'd be nice to work out something equivalent - perhaps have all containers on a host journal to the host - then the host ships to the log host19:04
mnaser"how do i make this change without breaking the world and making both source and binary installations work"19:05
mnaserso far i'm trying to refactor the stages which need modification such as pre install and install to seperate files which are included based on install method, with common things remaining in the main file19:06
mnaserto minimize impact19:06
logan-im going to try the repo build on this pike aio i have running thru the gate odyssey4me, so no worries19:06
odyssey4memnaser you might want to take some notes from pabelanger's work in https://github.com/openstack/ansible-role-nodepool which caters for packaged and source deploys, and there's some scaffolding to allow the init stuff to be deployed (machine containers and hosts) or not (eg: app containers like docker)... there are some pretty neat patterns in there19:07
mnaserodyssey4me: nice ill check that out right now19:07
odyssey4memnaser also see https://github.com/openstack/openstack-ansible-specs/blob/master/specs/queens/deployment-stages.rst which I'd be very happy to partner up on, that work needs contributors as it's a big job on its own and I'll be trying to do it along with https://github.com/openstack/openstack-ansible-specs/blob/master/specs/queens/python-build-install-simplification.rst which dovetails nicely into it19:09
odyssey4methe staged deployments proposal lends itself very nicely to also having different installation methods19:09
odyssey4meawesome, thank logan-19:10
logan-is there a ptg etherpad up?19:10
odyssey4mealright, time for me to head out for the night - thanks all!19:10
odyssey4melogan- not as far as I've seen, but it about that time that evrardjp informs us :)19:10
logan-have a good weekend odyssey4me19:11
*** lbragstad has quit IRC19:18
*** SerenaFeng has quit IRC19:19
ashakshould I be able to upload images direct from my browser via the web interface with a fairly standard default setup?19:22
*** vnogin has joined #openstack-ansible19:24
ashakHrm, the actual request the browser makes isn't to horizon at all, it's to port 9292.19:25
ashakoh directly to glance19:25
idlemindugh finally, dispatcher.d setup to activate my veth via up) and pre-down) for br-vlan19:26
idlemind#networkmanager19:26
ashakmy issues looks like a CORS problem: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'https://172.29.236.4' is therefore not allowed access.19:29
idlemindashak i wrote a bug for that19:29
idlemindlet me find it19:29
idlemindhttps://bugs.launchpad.net/openstack-ansible/+bug/174768419:30
openstackLaunchpad bug 1747684 in openstack-ansible "Default Values Do Not Allow Image Uploads to Glance from Horizon" [Undecided,New]19:30
ashakooh, ta19:30
idlemindyou can modify your user_variables to get past 'er19:31
idlemindbut if you could post on the bug that it impacted you too maybe that'll help the pro's take a look at it19:31
ashakI'll have a read and will do that if I can confirm it's the same problem (which is sounds like it is)19:32
idleminddo we build the ifcfg-lxbr0 or is that part of someother process on redhat/centos?19:35
*** lbragstad has joined #openstack-ansible19:37
openstackgerritMarc GariĆ©py (mgariepy) proposed openstack/openstack-ansible master: [DOC] rectify galera recovery for systemd  https://review.openstack.org/54299419:37
*** idlemind has quit IRC19:37
*** idlemind has joined #openstack-ansible19:38
openstackgerritMerged openstack/openstack-ansible-galera_server stable/ocata: Fix Galera socket for RedHat  https://review.openstack.org/54289419:40
vadim_mgariepy, socket patch works, quotes in template can be omitted, though.19:42
*** kstev has quit IRC19:42
*** vnogin has quit IRC19:43
*** kstev has joined #openstack-ansible19:46
mgariepyvadim_, cool.19:51
mgariepywell it's merged now :)19:51
idlemindk, i think i'm going to stop letting kickstart attempting to do any network configuration on my hosts beyond putting a single ip on the server; it's nearly useless past that19:53
mgariepyidlemind, try with preseed instead of kickstart :P\19:53
mgariepythen you can do the same with disk partitioning haha19:54
idlemindmgariepy thx actually disk partitioning works like a champ beyond naming things a little against what my ocd desires19:55
idlemindi'll have to look into the preseed stuff19:55
idlemindi already have a post-install ansible script that does other things like adds cockpit, disables selinux (at least until we get 'er sorted in openstack) amongst other things19:56
mgariepyok19:57
idlemindi'll look at the preseed for partitioning to see if it'll give me more control over naming for sure19:57
mgariepypreseed + partitionning is not fun19:58
ashak+1 to that :)19:58
mgariepyidlemind, http://paste.openstack.org/show/666960/19:59
mgariepyall that for gpt + efi + raid1 ;p20:00
mgariepyhaha20:00
idlemindand nothing about names, so must use the same self-generated ones you get w/kickstart but i can hit the docs and see what i can dig up20:01
idlemindhttp://paste.openstack.org/show/666961/20:02
idlemindworks reliably for a bios boot w/a goofy amount of different size volumes (usb boot volume + 2 x 2tb, 2 x 4tb)20:02
logan-kickstart is so much nicer to provision with20:03
logan-disk wise anyway20:03
idlemindi'll take a look at preseed too20:03
mgariepyzfs is so much better dans mdraid + lvm also.20:03
idlemindi'd also like to figure out a way other than a dedicated pxe boot server to send custom ks's to my different hosts20:04
jrosseri wish zfs root worked properly20:04
idlemindlonger term i'd like to go ceph bluestore directly so my perf will get limited by that i'd suppose20:04
idlemindfor now just ghetto lvm disk20:04
jrosserthen all this nonsense would go away20:04
mgariepydebian-installer got me sad this morning, wait for someone to press a key on the kb before it continues20:05
mgariepyon all the server i had to install..20:05
mgariepyzfs root would be awesome20:06
jrosserwe have it working partially20:06
jrosserall the real activity like /var on a zfs mirror20:07
jrosserand the bare minimum needed to boot remains not on zfs20:07
mgariepyone of my colleague do netboot and the / on zfs.20:07
mgariepybut it's somewhat hackish ;) haha20:07
jrossergetting that all to come up working our a pxeboot was challenging20:08
jrosser*out of20:08
mgariepyfor my ctrl plane i do: / raid1 and then my container on zfs.20:08
mgariepybetter than nothing i guess.20:08
idlemindguess i never understood the love affair with zfs, is it all perf driven?20:10
jrosseri guess if anything it's the opposite of that20:11
jrosserfeatures + data integrity + not being RAID20:11
jrosserand of course when combined with lxc, copy-on-write20:12
idlemindor combined w/cinder you'd get copy on write like ceph provides which would be a huge boon to image creation time but i don't think zfs has a driver does it?20:13
idleminddoes zfs require ecc memory?20:13
mgariepyidlemind, everything needs ecc don't you think ?20:13
mgariepyit's always better.20:14
idlemindnot saying it isn't but for my lab i'd prefer it's not required20:14
idlemindand almost no commodity gear you find will have it20:14
openstackgerritMerged openstack/openstack-ansible-os_aodh stable/pike: Zuul: Remove project name  https://review.openstack.org/54297120:14
*** acormier has joined #openstack-ansible20:21
*** kstev has quit IRC20:25
*** olivierbourdon38 has quit IRC20:27
*** yolanda has quit IRC20:28
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_gnocchi stable/newton: Zuul: Remove project name  https://review.openstack.org/54301520:33
*** hamza21 has quit IRC20:33
*** dave-mccowan has quit IRC20:39
*** acormier has quit IRC20:45
*** acormier has joined #openstack-ansible20:47
*** acormier has quit IRC20:47
*** vnogin has joined #openstack-ansible20:48
openstackgerritMerged openstack/openstack-ansible stable/pike: Zuul: Remove project name  https://review.openstack.org/54288820:48
*** acormier has joined #openstack-ansible20:48
*** acormier has quit IRC20:50
*** acormier has joined #openstack-ansible20:50
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_ceilometer stable/ocata: Zuul: Remove project name  https://review.openstack.org/54302220:53
adcpdkidlemind thank you so much for such a nice troubleshooting yesterday. I finally got my networking setup and working as a flat network! Such a nice feeling when there are people out there to help and go out with you and fight with monsters you were scared of))) It was my first experience of using IRC channels in my life and I won't forget that. I will try to do my best to be around to help the newbies like me ;-)21:01
*** adcpdk has quit IRC21:06
cloudnull^ that's awesome!21:12
cloudnullcommunity for the win!21:12
cloudnullhttps://github.com/ansible/ansible/pull/35913 -- logan- have a look at that21:13
cloudnullshould greatly speed up variable render time21:13
cloudnullhowever! hopefully it doesn't break everything21:14
openstackgerritMohammed Naser proposed openstack/openstack-ansible-tests master: DNM: Remove database contract for Keystone  https://review.openstack.org/54302621:15
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_keystone master: DNM: Deploy Keystone from binary packages  https://review.openstack.org/54302721:16
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_keystone master: DNM: Deploy Keystone from binary packages  https://review.openstack.org/54302721:16
idlemindadcpdk thx i'm going to blush21:20
idlemindi too have found this community to be probably the best in the openstack ecosystem in terms of collaboration and one of the better open source projects as a whole21:20
mnaserdmsimard: looks like osa uses openstack-pike repos instead of master, should we use master or master-head?21:20
openstackgerritMajor Hayden proposed openstack/openstack-ansible stable/ocata: Remove jinja from failed_when  https://review.openstack.org/54139221:21
*** adcpdk has joined #openstack-ansible21:21
openstackgerritMajor Hayden proposed openstack/openstack-ansible master: Use package_state=present for CentOS 7 testing  https://review.openstack.org/54101721:21
dmsimardmnaser: where ?21:21
mnaserdmsimard: https://github.com/openstack/openstack-ansible-openstack_hosts/blob/master/vars/redhat-7.yml#L101-L11321:21
*** vnogin has quit IRC21:21
dmsimardmnaser: those are stable repos, not trunk21:22
*** chyka has joined #openstack-ansible21:22
dmsimardmnaser: purpose is not the same since OSA only uses RDO for deps21:22
mnaserdmsimard: correct, but i see 2 master repos in rdo, wondering which one is better21:22
mnaserdmsimard: aware, i'm going to be using master packages for my binary deployment fun (see reviews above)21:22
dmsimardmnaser: For trunk you want to use https://trunk.rdoproject.org/centos7-master/delorean-deps.repo21:23
*** vnogin has joined #openstack-ansible21:23
*** armaan has quit IRC21:23
dmsimardmnaser: for actual openstack packages, you want to use https://trunk.rdoproject.org/centos7-master/current-passed-ci/delorean.repo21:23
*** armaan has joined #openstack-ansible21:24
mnaserdmsimard: ok perfect!21:24
dmsimardmnaser: however note that current-passed-ci is a symlink that can change so you typically want to resolve the hashed repository and set up the hashed repository (i.e, https://trunk.rdoproject.org/centos7/17/a5/17a57d99b10f5872520e1a395214f80c9cfd1a65_c30c070f )21:24
mnaserdmsimard: for dev purposes though, could we use unhashed so it always tests on latest (at least now in the early iterations)21:25
dmsimardmnaser: unhashed means your job could start and have half the installed packages installed from a version and the other half on another version if it changed midway21:25
dmsimardnot sure if you see what I mean21:25
mnaserdmsimard: oh i see what you mean21:25
mnaserthat makes sense now21:25
dmsimardyou have two ways of retrieving the hash21:26
dmsimardyou can parse that delorean.repo file21:26
dmsimardand you also have a yaml file: https://trunk.rdoproject.org/centos7-master/current-passed-ci/commit.yaml (just realized we need to set up a text mimetype for that, use curl :))21:26
mnaseri think ill get_url it for now21:27
dmsimardyou can probably do a get_url and then use from_yaml or something21:27
*** adcpdk has quit IRC21:27
dmsimardthe formula for the trunk repository hash is ... I foret21:27
dmsimardforget21:27
mnaserwasnt it <openstack_commit_id>_<first 8 of delorean current commit> or something21:28
mnaser:p21:28
dmsimard$commithash_$distrohash[:-8] ?21:28
dmsimardyeah21:28
dmsimardI always forget which one comes first and how many characters it is ..21:28
evrardjplogan-: odyssey4me: the PTG's etherpad was discussed at last community meeting21:29
evrardjphttps://etherpad.openstack.org/p/osa-rocky-ptg21:29
evrardjpit's also on the list of etherpads21:30
evrardjpI took the liberty to add us to a photo slot21:30
idlemindugh, need to find me some reading on ansible'ing some cloud admin tasks (create projects / users)21:30
evrardjpidlemind: check in our ops repo21:30
evrardjp:)21:30
idlemindhopefully in an idempotent way so can run, run and run again as needed.21:30
idlemindcool cool will do21:30
spotzhey logan- is etcd a requirement for Pike in general or just recommended. I saw you did our play:)21:32
*** armaan has quit IRC21:33
openstackgerritJames E. Blair proposed openstack/openstack-ansible-haproxy_server stable/newton: Zuul: Remove project name  https://review.openstack.org/54302821:33
*** adcpdk has joined #openstack-ansible21:41
*** chyka has quit IRC21:43
*** chyka has joined #openstack-ansible21:43
*** sar has quit IRC21:44
*** vnogin has quit IRC21:45
*** tux_ has quit IRC21:45
spotzevrardjp: You sticking around or heading out?21:45
*** chyka has quit IRC21:48
openstackgerritMohammed Naser proposed openstack/openstack-ansible-os_keystone master: DNM: Deploy Keystone from binary packages  https://review.openstack.org/54302721:50
*** adcpdk has quit IRC21:59
*** adcpdk has joined #openstack-ansible21:59
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible master: Add nspawn container driver  https://review.openstack.org/47701722:03
*** chigang_ has quit IRC22:06
*** chigang_ has joined #openstack-ansible22:06
*** sar has joined #openstack-ansible22:08
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_swift stable/pike: Zuul: Remove project name  https://review.openstack.org/54303722:13
*** ildikov has quit IRC22:13
*** ildikov has joined #openstack-ansible22:14
idlemindhey, does the rsyslog container have a web-ui for viewing the logs (could one be added to it by chance)?22:15
idlemindssh + attach + less = not as fun22:16
*** boxrick has quit IRC22:19
*** boxrick has joined #openstack-ansible22:20
*** pbandark has quit IRC22:23
evrardjpspotz: sorry, I was out, and I will stop working today. Let's postpone to Mondays :)22:27
spotzevrardjp: we'll get time eventually!22:27
evrardjp:)22:28
openstackgerritJames E. Blair proposed openstack/openstack-ansible-os_nova stable/pike: Zuul: Remove project name  https://review.openstack.org/54304122:33
*** dave-mccowan has joined #openstack-ansible22:40
*** acormier has quit IRC22:41
*** armaan has joined #openstack-ansible22:43
*** trident has joined #openstack-ansible22:52
*** acormier has joined #openstack-ansible22:52
*** dariko1 has quit IRC22:55
*** acormier has quit IRC23:03
*** AndyWojo has quit IRC23:04
*** AndyWojo has joined #openstack-ansible23:04
*** armaan has quit IRC23:06
*** rodolof has quit IRC23:11
*** geekinutah has quit IRC23:16
*** geekinutah has joined #openstack-ansible23:16
*** vnogin has joined #openstack-ansible23:31
*** chyka has joined #openstack-ansible23:34
openstackgerritJames E. Blair proposed openstack/openstack-ansible-lxc_hosts stable/ocata: Zuul: Remove project name  https://review.openstack.org/54305223:34
*** esberglu has quit IRC23:37
*** chyka has quit IRC23:37
*** chyka has joined #openstack-ansible23:38
*** chyka has quit IRC23:42
*** eumel8 has quit IRC23:45
*** vnogin has quit IRC23:46
*** lbragstad has quit IRC23:49
openstackgerritJames E. Blair proposed openstack/ansible-hardening stable/pike: Zuul: Remove project name  https://review.openstack.org/54305323:54

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!