Wednesday, 2018-01-17

*** SmearedBeard has joined #openstack-ansible00:17
*** SmearedBeard has quit IRC00:22
*** cmart has joined #openstack-ansible00:27
*** chyka has quit IRC00:32
*** cjloader has joined #openstack-ansible00:42
*** smatzek has joined #openstack-ansible00:43
*** dave-mccowan has joined #openstack-ansible00:49
*** wspthr has quit IRC00:49
*** wspthr has joined #openstack-ansible00:50
*** cjloader has quit IRC01:09
*** cmart has quit IRC01:09
*** markvoelker has quit IRC01:25
*** woodard_ has joined #openstack-ansible01:30
*** woodard has quit IRC01:30
*** dave-mccowan has quit IRC01:42
*** dave-mccowan has joined #openstack-ansible01:42
*** chyka has joined #openstack-ansible01:54
*** chyka has quit IRC01:59
*** dave-mccowan has quit IRC02:19
*** cjloader has joined #openstack-ansible02:26
*** cjloader has quit IRC02:27
*** cjloader_ has joined #openstack-ansible02:27
*** esberglu has quit IRC02:44
*** dave-mccowan has joined #openstack-ansible02:52
*** ANKITA has joined #openstack-ansible02:58
*** vnogin has joined #openstack-ansible03:11
*** vnogin has quit IRC03:16
*** indistylo has joined #openstack-ansible03:20
*** smatzek has quit IRC03:20
*** smatzek has joined #openstack-ansible03:20
ANKITAhttps://bugs.launchpad.net/openstack-ansible/+bug/172979203:20
openstackLaunchpad bug 1729792 in openstack-ansible "openstack ansible error when execute setup-openstack.yml" [Low,Confirmed]03:20
*** smatzek has quit IRC03:22
*** smatzek has joined #openstack-ansible03:22
*** cjloader_ has quit IRC03:23
*** markvoelker has joined #openstack-ansible03:26
*** smatzek has quit IRC03:27
*** aruns has joined #openstack-ansible03:31
ANKITAis it fixed https://bugs.launchpad.net/openstack-ansible/+bug/172979203:31
openstackLaunchpad bug 1729792 in openstack-ansible "openstack ansible error when execute setup-openstack.yml" [Low,Confirmed]03:31
ANKITA?03:31
*** esberglu has joined #openstack-ansible03:31
*** indistylo has quit IRC03:33
*** aruns has quit IRC03:35
*** ykarel has joined #openstack-ansible03:45
*** markvoelker has quit IRC04:01
*** jbadiapa_ has quit IRC04:25
*** dave-mccowan has quit IRC04:32
*** bhujay has joined #openstack-ansible04:33
*** cjloader has joined #openstack-ansible04:54
*** chhagarw has joined #openstack-ansible04:56
*** chhavi has joined #openstack-ansible04:56
*** cjloader has quit IRC04:57
*** markvoelker has joined #openstack-ansible04:57
*** poopcat1 has quit IRC05:06
*** hybridpollo has quit IRC05:10
*** markvoelker has quit IRC05:16
*** chyka has joined #openstack-ansible05:21
*** chyka has quit IRC05:26
cloudnullspotz: maybe? that issue was something ivve brought up the other day.05:28
openstackgerritYandy Sun proposed openstack/openstack-ansible-os_cinder master: modify cinder spelling errors  https://review.openstack.org/53463706:15
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249106:16
*** pmannidi has quit IRC06:16
*** jbadiapa has joined #openstack-ansible06:18
*** markvoelker has joined #openstack-ansible06:28
*** pmannidi has joined #openstack-ansible06:33
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-os_neutron master: Fix LBaaSv2 config and package requirements  https://review.openstack.org/53380606:36
*** pmannidi has quit IRC06:38
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible master: Force ansible to ignore plugins when bootstrapping  https://review.openstack.org/53437706:47
*** mcarden has quit IRC06:48
*** pmannidi has joined #openstack-ansible06:50
*** markvoelker has quit IRC07:02
*** m|y|k has joined #openstack-ansible07:03
*** m|y|k has quit IRC07:04
*** xar- has quit IRC07:04
openstackgerritMerged openstack/openstack-ansible-os_tacker master: Provide the correct directory to fetch the keys  https://review.openstack.org/53436107:13
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-specs master: Blueprint for Blazar integration with OSA  https://review.openstack.org/52856707:14
*** threestrands_ has quit IRC07:20
*** armaan has joined #openstack-ansible07:22
*** gokhan_ has quit IRC07:29
*** mbuil has joined #openstack-ansible07:29
*** ANKITA has quit IRC07:31
*** ykarel has quit IRC07:36
*** armaan has quit IRC07:38
*** ykarel has joined #openstack-ansible07:38
*** ykarel has quit IRC07:43
*** ykarel has joined #openstack-ansible07:43
*** gokhan_ has joined #openstack-ansible07:45
*** hamza21 has joined #openstack-ansible07:48
*** pcaruana has joined #openstack-ansible07:48
*** pcaruana has quit IRC07:55
*** pcaruana has joined #openstack-ansible07:55
*** peri has joined #openstack-ansible07:56
*** quotaq has quit IRC07:57
cloudnulllooks like there were some changes in OSC -- http://logs.openstack.org/98/522598/13/check/openstack-ansible-upgrade-ubuntu-xenial/b461c96/job-output.txt.gz#_2018-01-16_19_11_14_506455 -- evrardjp odyssey4me we should add that to the bug list as it's looks to be breaking neutron upgrade testing.07:58
*** markvoelker has joined #openstack-ansible07:58
evrardjp[m]oh great, more....07:59
evrardjp[m]that's a nice "good morning" to me!08:00
evrardjp[m]having a look with the rest.08:00
evrardjp[m]and good morning everyone !08:01
prometheanfiremoin08:03
*** cristicalin has joined #openstack-ansible08:04
neithhello all08:05
*** jwitko_ has quit IRC08:12
*** xar- has joined #openstack-ansible08:13
*** shardy has joined #openstack-ansible08:14
*** cristicalin has quit IRC08:14
*** cristicalin has joined #openstack-ansible08:14
*** gkadam has joined #openstack-ansible08:21
*** cristicalin has quit IRC08:22
*** cristicalin has joined #openstack-ansible08:22
*** shardy is now known as shardy_afk08:25
jafehagood morning08:27
openstackgerritzhongshengping proposed openstack/openstack-ansible-os_trove master: Remove Trove's support for Heat  https://review.openstack.org/53467708:28
*** markvoelker has quit IRC08:32
*** mbuil has quit IRC08:35
*** ykarel has quit IRC08:36
*** cristicalin has quit IRC08:41
*** pmannidi has quit IRC08:41
*** tnogisto has quit IRC08:45
*** tnogisto has joined #openstack-ansible08:47
*** mbuil has joined #openstack-ansible08:48
*** shardy_afk is now known as shardy08:52
*** ykarel has joined #openstack-ansible09:04
*** SmearedBeard has joined #openstack-ansible09:08
evrardjpcloudnull: permission to update your patch?09:12
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Force ansible to ignore plugins when bootstrapping  https://review.openstack.org/53437709:16
*** chhagarw has quit IRC09:16
*** chhavi has quit IRC09:16
*** hw_wutianwei has quit IRC09:23
*** hw_wutianwei has joined #openstack-ansible09:23
*** pbandark has joined #openstack-ansible09:23
*** SmearedBeard has quit IRC09:28
*** markvoelker has joined #openstack-ansible09:29
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: [WIP] Integrate OVS AIO Scenario  https://review.openstack.org/53431809:32
*** SmearedBeard has joined #openstack-ansible09:33
bhujayHello cloudnull:  hwoarang: Needed some guidance regarding https://review.openstack.org/#/c/526655/ , I tested it worked well without adding the notify service restart after the ceilometer-upgrade task. Do you suggest me to add it still09:35
hwoarangmaybe something else notified the service09:43
*** alefra has joined #openstack-ansible09:43
hwoarangbut you still need it so you can be explicit09:43
*** ThomasSch has joined #openstack-ansible09:49
ThomasSchhi09:49
bhujayok I will add that09:51
bhujayhwoarang: ok i will add that09:52
alefrahi , i've a problem with neutron fwaas, if i enable firewall_v2 on neutron server i've this error Error loading class by alias: NoMatches: No 'neutron.service_plugins' driver found, looking for 'firewall_v2'09:53
alefrathe same thing appen also with neutron plugin firewall09:55
*** markvoelker has quit IRC10:02
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit  https://review.openstack.org/52249110:07
bhujayhwoarang: added notify and tested working ok , here is the log http://paste.openstack.org/show/646442/ . I did not get the indent part , ansible will fail if there is an indent error isn't it ?10:11
hwoarangbhujay: this is not correct ansible code. that's what the tests are telling you10:11
hwoarangah wait10:12
hwoarangsorry was looking at the wrong thing10:12
hwoarangstill the tests give -1 so you may want to look at it :)10:13
bhujayI tried the indent but it failed and without indent worked well http://paste.openstack.org/show/646449/  , let me see  if I can figure out what else cloud be wrong10:16
hwoarangindent is fine10:17
Taseerodyssey4me: can you please help me with this => http://logs.openstack.org/91/522491/38/check/openstack-ansible-functional-ubuntu-xenial/dbda033/job-output.txt.gz#_2018-01-17_06_51_19_12149510:20
*** vnogin has joined #openstack-ansible10:29
openstackgerritbhujay kumar proposed openstack/openstack-ansible-os_ceilometer master: Adds ceilometer-upgrade task  https://review.openstack.org/53471710:39
*** stuartgr has joined #openstack-ansible10:41
*** SmearedBeard has quit IRC10:42
*** SmearedBeard has joined #openstack-ansible10:43
hwoarangTaseer: it seems you are missing some config option to point congress to the correct mysql10:58
hwoarangsee how others roles do it like os_keystone10:58
hwoarangwhere do you define the database in congress?10:59
*** markvoelker has joined #openstack-ansible11:00
Taseerhwoarang: https://github.com/openstack/openstack-ansible-os_congress/blob/master/tasks/congress_db_setup.yml11:12
hwoarangthis is not enough11:13
hwoarangthis prepares the db11:13
hwoarangbut you need to tell it where to connect in the first place11:13
Taseerokay..11:13
hwoaranglike i said, os_keystone may help11:13
hwoarangor the other os_* roles11:13
hwoarangthey all contain some db string configuration11:13
*** hachi has joined #openstack-ansible11:17
*** hamza21 has quit IRC11:20
bhujayhwoarang: i messed up the whole thing by submitting my change without correct change id now it has created a new change ID  , is there any way correct it ?11:30
hwoarangbhujay: i dont think so just abandoned one of them11:31
hwoarang*abandon11:31
*** tnogisto has quit IRC11:32
*** markvoelker has quit IRC11:32
*** tnogisto has joined #openstack-ansible11:36
*** Taseer has quit IRC11:41
*** vnogin has quit IRC11:41
openstackgerritbhujay kumar proposed openstack/openstack-ansible-os_ceilometer master: Adds ceilometer-upgrade task  https://review.openstack.org/52665511:48
*** vnogin has joined #openstack-ansible12:07
*** shardy is now known as shardy_lunch12:12
*** smatzek has joined #openstack-ansible12:13
*** mbuil has quit IRC12:23
openstackgerritMerged openstack/openstack-ansible-os_keystone master: Correct keystone-doctor tag name  https://review.openstack.org/53427812:29
*** markvoelker has joined #openstack-ansible12:29
openstackgerritMerged openstack/openstack-ansible-haproxy_server master: Make haproxy stats refresh interval configurable  https://review.openstack.org/53260712:30
*** gameon has quit IRC12:31
*** chyka has joined #openstack-ansible12:34
*** mbuil has joined #openstack-ansible12:37
*** chyka has quit IRC12:39
*** admin0 has quit IRC12:43
*** dave-mccowan has joined #openstack-ansible12:45
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone stable/pike: Correct keystone-doctor tag name  https://review.openstack.org/53474612:45
*** mbuil has quit IRC12:52
*** ykarel has quit IRC12:57
*** SmearedBeard has quit IRC12:59
*** markvoelker has quit IRC13:03
*** pmannidi has joined #openstack-ansible13:04
*** mbuil has joined #openstack-ansible13:04
*** sauloaislan has joined #openstack-ansible13:05
sauloaislanMorning13:05
*** Taseer has joined #openstack-ansible13:06
CobHeadMorning13:07
sauloaislanI have one problem after this merge https://github.com/openstack/openstack-ansible/commit/330dcd376c2709d268501a9783365ada0402329213:08
*** woodard_ has quit IRC13:11
odyssey4mesauloaislan which is? cc cloudnull13:11
*** woodard has joined #openstack-ansible13:11
sauloaislanI need edit /var/lib/lxc/neutron_agents_container/eth12.ini after merge this file does not exist13:12
*** shardy_lunch is now known as shardy13:16
*** markvoelker has joined #openstack-ansible13:21
odyssey4mesauloaislan it sounds like we need some sort of compatibility or migration implementation there, which cloudnull will need to look into as that's part of a body of work he's working on13:23
openstackgerritMajor Hayden proposed openstack/ansible-hardening master: Use import/include_tasks  https://review.openstack.org/53432713:35
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/pike: Fix sources-branch-updater and add docs  https://review.openstack.org/53170813:36
openstackgerritFlávio Ramalho proposed openstack/openstack-ansible-os_monasca-agent master: Add pip_install as role dependency  https://review.openstack.org/53475813:37
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/pike: Fix sources-branch-updater and add docs  https://review.openstack.org/53170813:40
openstackgerritMerged openstack/openstack-ansible-os_cinder master: modify cinder spelling errors  https://review.openstack.org/53463713:41
openstackgerritMerged openstack/openstack-ansible-lxc_hosts master: tests: test.yml: Use the 'slurp' module to get the SSH key  https://review.openstack.org/53370213:45
*** pmannidi has quit IRC13:46
sauloaislanodyssey4me: Ok, I'll wait for him.13:47
sauloaislanodyssey4me: thk13:47
*** SmearedBeard has joined #openstack-ansible13:52
*** esberglu has quit IRC13:54
*** cmart has joined #openstack-ansible13:58
openstackgerritPeriyasamy Palanisamy proposed openstack/openstack-ansible master: Make Opendaylight as the BGP speaker using Quagga  https://review.openstack.org/52390713:59
*** rromans has quit IRC14:01
*** armaan has joined #openstack-ansible14:03
openstackgerritMerged openstack/openstack-ansible-os_tacker stable/pike: Update paste, policy and rootwrap configurations 2017-12-19  https://review.openstack.org/53363314:07
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-tests master: Vagrantfile: Use rync to copy working directory to VM  https://review.openstack.org/53476414:10
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible master: Force ansible to ignore plugins when bootstrapping  https://review.openstack.org/53437714:11
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_monasca-agent stable/pike: Initial OSA zuul v3 role jobs  https://review.openstack.org/53476614:11
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_monasca-agent stable/pike: Fix default role missing pip packages  https://review.openstack.org/53476714:12
*** esberglu has joined #openstack-ansible14:12
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-tests master: Vagrantfile: Use rsync to copy working directory to VM  https://review.openstack.org/53476414:12
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible master: Force ansible to ignore plugins when bootstrapping  https://review.openstack.org/53437714:14
cloudnullsauloaislan: what's going on ?14:15
sauloaislanII have a scenario where I have to use networks and flat and vlan networks. In the flat scenario I go in the file /var/lib/lxc/neutron_agents_container/eth12.ini and change br-vlan to br-mgmt14:19
sauloaislancloudnull:14:20
cloudnullthat should all work as expected.14:20
cloudnullpost that pr though the default would be to run the neutron agents on the host.14:21
cloudnullinstead of in a container14:21
cloudnullI have a similar setup at home.14:21
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_monasca-agent stable/pike: Update vars and test tooling for Pike  https://review.openstack.org/53476914:23
*** SmearedBeard has quit IRC14:23
*** smatzek has quit IRC14:24
sauloaislancloudnull: But when I need to use br-vlan on eth12 where am I going to change it?14:24
*** smatzek has joined #openstack-ansible14:25
*** mcarden has joined #openstack-ansible14:25
cloudnullit should no longer be necessary for you to modify a container config on that node. assuming you have veth pair on your br-vlan/mgmt interface for eth12 everything should work.14:26
cloudnullsauloaislan: I assume you have something like this https://github.com/openstack/openstack-ansible/blob/master/etc/network/interfaces.d/aio_interfaces.cfg#L44-L5914:27
*** smatzek has quit IRC14:29
cloudnullevrardjp: https://review.openstack.org/#/c/534377/ is past the point where it bootstraps in the gate, so it looks like it's working now, mind giving it a poke in your upgrade tests to validate?14:30
evrardjpalready testing it :)14:31
cloudnullsweet!14:32
cloudnullif it still doesn't work, i give up and we can go with a more heavy handed approach :)14:32
evrardjpI think we could still have one last thing on the same principles. But yeah, if it doesn't work, let's move on with the deletion. We shouldn't spend too much times on those issues, and iron out all the upgrades bugs asap14:33
sauloaislancloudnull: Ok I will try, thk14:34
cloudnullevrardjp: the gate is well into lxc_host setup now so from a greenfield pov its chugging a long.14:35
evrardjpyeah, don't look at centos.14:35
evrardjpcloudnull: on top of that, we recently have an increase of failures in tempest, since we have more tests now.14:36
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_monasca-agent stable/pike: Initial OSA zuul v3 role jobs  https://review.openstack.org/53476614:37
*** vnogin has quit IRC14:38
openstackgerritMerged openstack/openstack-ansible-os_monasca-agent stable/pike: Update vars and test tooling for Pike  https://review.openstack.org/53476914:38
cloudnullthat's not necessarily a bad thing, more failures due to more tests could be ok if it's helping imrprove our coverage14:38
*** vnogin has joined #openstack-ansible14:40
evrardjpcloudnull: http://paste.openstack.org/show/646516/14:41
evrardjpso with latest code, it seems better: (we do the bootstrap correctly)14:42
evrardjpbut, as you can see in the paste, we don't get inventory later.14:42
openstackgerritMerged openstack/openstack-ansible-os_monasca-agent stable/pike: Fix default role missing pip packages  https://review.openstack.org/53476714:42
evrardjplet me check what's the content of the .rc14:43
cloudnulltrying that now on a fresh vm14:43
*** jwitko has joined #openstack-ansible14:44
evrardjpit looks ok14:44
evrardjpso it's something happening during the upgrade I'd say.14:44
evrardjpI'll check the run_upgrade14:44
cloudnullyea, bootstrap ocata/pike to master is working fine now.14:46
odyssey4meheh, the rc file accepts any preset env var as precedence14:46
cloudnullnow to figure our the inventory part14:46
odyssey4meso you'll have to unset after the bootstrap14:46
cloudnullyea we source the boostrap-ansible.sh in gate-check-commit.sh so those options were getting inherited14:47
*** ianychoi has quit IRC14:48
openstackgerritMerged openstack/openstack-ansible-haproxy_server stable/pike: Add ability to manage non-inventory hosts  https://review.openstack.org/53361814:51
*** SmearedBeard has joined #openstack-ansible14:51
cloudnullsauloaislan: did you get it to work ?14:51
*** ianychoi has joined #openstack-ansible14:52
sauloaislancloudnull: I'll take this test because I'm in another, but then I can let you know if it worked, if you want?14:55
cloudnullthat'd be great14:57
cloudnullIf you're still having issues with it I'd love to know so we can figure them out together and get an appropriate fix for everyone.14:58
openstackgerritAndy McCrae proposed openstack/openstack-ansible-haproxy_server stable/ocata: Add ability to manage non-inventory hosts  https://review.openstack.org/53480015:02
openstackgerritMerged openstack/openstack-ansible-repo_build master: Add pcre-lib for uwsgi  https://review.openstack.org/53438715:03
evrardjpcloudnull: what do you mean?15:03
evrardjplet me check, one sec15:04
cloudnullI run gatecheck commit from pike, and cancel it as soon as it gets done with the bootstrap portion15:04
cloudnullthen do it again w/ master and it seems to be ok15:04
evrardjpcloudnull: oh yeah your patch seem fine15:05
cloudnullso now we just need to figure out what's up with the env var to make that happy15:08
*** sc68cal has quit IRC15:09
*** smatzek has joined #openstack-ansible15:10
*** cjloader has joined #openstack-ansible15:10
evrardjpcloudnull: I think that was part of my previous patch that took the bazooka approach15:13
evrardjplet me check15:13
evrardjpYes, I unset the ANSIBLE_INVENTORY before the bootstrap-ansible15:14
evrardjpthis way the bootstrap is clean, and the next call in the script should take the defaults15:15
evrardjpsee https://review.openstack.org/#/c/531123/2/scripts/run-upgrade.sh15:15
openstackgerritMajor Hayden proposed openstack/ansible-hardening master: Use import/include_tasks  https://review.openstack.org/53432715:15
evrardjpcloudnull: do we merge our patches together?15:16
evrardjpcloudnull: alternatively we could, export ANSIBLE_INVENTORY do the bootstrap, then unset.15:20
evrardjpin your patch15:20
cloudnulleither way is good by me15:21
cloudnullwe could make the two dependent15:21
cloudnullor abandon one and merge the bits in a single patch15:21
cloudnullwhatever is easiest15:21
evrardjplet's merge yours first, and do another bug with the second patch. We'll discuss the approach on the second patch, now that your patch is fine.15:24
openstackgerritMerged openstack/openstack-ansible-os_monasca-agent stable/pike: Initial OSA zuul v3 role jobs  https://review.openstack.org/53476615:25
*** bhujay has quit IRC15:28
*** hachi has quit IRC15:29
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Cleanup environment before bootstrap-ansible in upgrades  https://review.openstack.org/53112315:32
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Cleanup environment before bootstrap-ansible in upgrades  https://review.openstack.org/53112315:34
evrardjpcloudnull: rebased on top of yours: https://review.openstack.org/#/c/531123/15:34
evrardjpeverytime hachi leaves the channel, I am sad.15:34
cloudnullhaha15:35
cloudnullevrardjp: 1 comment in the 531123 otherwise lgtm15:35
evrardjphttps://en.wikipedia.org/wiki/Hachik%C5%8D15:35
*** SmearedBeard has quit IRC15:38
openstackgerritAndy McCrae proposed openstack/openstack-ansible-haproxy_server stable/newton: Add ability to manage non-inventory hosts  https://review.openstack.org/53481315:38
*** SmearedBeard has joined #openstack-ansible15:42
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Cleanup environment before bootstrap-ansible in upgrades  https://review.openstack.org/53112315:46
cloudnull++ lgtm15:47
*** sc68cal has joined #openstack-ansible15:49
evrardjptesting the upgrades with both patches now15:49
*** vnogin has quit IRC15:52
*** chas has joined #openstack-ansible15:54
*** chas has quit IRC15:54
*** vnogin has joined #openstack-ansible15:56
*** vnogin has quit IRC15:57
evrardjpok so we are at the next problem, initial problem listed there https://bugs.launchpad.net/openstack-ansible/+bug/174146215:58
openstackLaunchpad bug 1741462 in openstack-ansible "P-Q upgrade fails due to change in inventory" [Critical,In progress] - Assigned to Jean-Philippe Evrard (jean-philippe-evrard)15:58
cloudnullyup15:58
*** vnogin has joined #openstack-ansible15:58
evrardjphttp://paste.openstack.org/show/646525/15:59
evrardjplet me check if the key exists in the inventory15:59
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-galera_server master: RedHat: Ensure that conflicting MariaDB packages are removed  https://review.openstack.org/53481916:01
*** bhujay has joined #openstack-ansible16:05
evrardjpcloudnull: did you get to look at this inventory issue? I didn't like the inventory before, but now I am even more confused.16:06
cloudnullyup, mind if I amend your pr ?16:11
cloudnullor I can tack on to it16:11
ivvecloudnull: i was mentioned earlier about an issue that spotz also might have encountered, don't have the full log, what was it about if you recall?16:22
ivvelbaas?16:22
spotzivve I was talking about backgrouond on a patch I was reviewing but yeah I think you were part of the discussion I was referring to:)16:23
*** shardy has quit IRC16:23
ivvealright, i still have some issues but they are with neutron. i thought they were gone but now back on one of the routers16:24
ivveall are in standby mode16:24
ivveand neutron.log in one of the controllers @ agents container is nuts16:25
ivvei haven't had time to check yet.. was late yesterday as well16:25
*** shardy has joined #openstack-ansible16:25
cxc99is openstack-ansible compatible with NetworkManager? for some reason it didn't disable NM or the instructions don't say to first16:26
cloudnullcxc99: we don't generally touch the host networking. we have specific requirements in terms of setting up a set of bridges however how you get there is generally up to you.16:29
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible master: Cleanup environment before bootstrap-ansible in upgrades  https://review.openstack.org/53112316:31
cloudnullevrardjp: ^16:31
*** bhujay has quit IRC16:31
ivvehttps://hastebin.com/ocedujelow.sql16:31
ivvebut seems to have disappeared again.. after being down for a few hours16:31
ivveperhaps keepalived gave it a rest (if thats the problem)16:32
cloudnullivve: are you using l3 ha ?16:33
ivveaye16:33
ivveone router all went standby/backup16:33
openstackgerritMerged openstack/openstack-ansible-haproxy_server stable/ocata: Add ability to manage non-inventory hosts  https://review.openstack.org/53480016:34
ivveis there any good way to reset keepalived there, if its the issue?16:34
cloudnullkill keepalived and restart the neutron l3 agent16:35
ivveon that controller?16:35
cloudnullwithin the neutron-agent container (network node in master)16:35
cloudnullwe had similar experiences with l3ha in prod.16:35
cloudnullI've not tested it for a while now.16:35
ivvething is, i usually restart the node as it will down everything and then start it up in the correct order16:35
ivvebut that did not help16:36
ivvehowever it seems having the node down (for some magical amount of time) seem to have done just that16:36
ivvestuff like that gives me tics :D16:36
ivvei was going on with restarting both containers / services etc16:37
ivvethen went on with full controller reboot16:37
ivvenothing16:37
ivveso i halted it16:37
openstackgerritMerged openstack/openstack-ansible-haproxy_server stable/newton: Add ability to manage non-inventory hosts  https://review.openstack.org/53481316:37
cloudnull++ I hate crap like that.16:38
ivveha16:39
ivvei stand corrected16:39
ivveits still not working16:39
ivvewhich is better tbh16:39
ivveso it says the port is on16:40
ivvebut the ha ports are all down16:40
openstackgerritMerged openstack/openstack-ansible stable/newton: Avoid retrieving IP from a missing bridge  https://review.openstack.org/53224516:42
ivvecloudnull: got any magic for that? kill the keepalived + l3 agent restart then?16:42
cloudnullI'd just do something `ansible -m shell -a 'killall keepalived; systemctl restart neutron-l3-agent' neutron_agents --forks 116:43
cloudnull^ totally untested.16:43
cloudnullmaybe add a sleep ?16:44
cloudnull`ansible -m shell -a 'killall keepalived; systemctl restart neutron-l3-agent; sleep 5' neutron_agents --forks 116:44
cloudnullwhich would give the agent a couple seconds to spin up before the next one is killed and restarted16:45
ivveaye16:45
ivvesounds like a plan16:45
cloudnullidk if that works, but it's work a try16:45
cloudnull**worth16:45
*** pcaruana has quit IRC16:46
ivveso strange16:47
*** peri has quit IRC16:50
openstackgerritMajor Hayden proposed openstack/ansible-hardening master: Use import/include_tasks  https://review.openstack.org/53432716:55
*** armaan has quit IRC16:56
evrardjpcloudnull: you seem to have overriden my past changes.16:57
cloudnulloh did i rebase on an old review16:58
evrardjpyes16:58
cloudnullok. one sec16:58
ivveha network tenants also borked16:59
ivvejees16:59
evrardjpivve: I can check your keepalived issue with you tomorrow. If there is something that needs to be done in the role, we'll do it.16:59
evrardjpif you could file a bug with the context, that would be helpful17:00
ivveaye, ill try to find some solid error first17:00
evrardjpbut it doesn't sound like an ansible-keepalived issue, more a neutron one.17:00
ivveyup17:00
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible master: Cleanup environment before bootstrap-ansible in upgrades  https://review.openstack.org/53112317:01
cloudnullevrardjp: ^17:01
evrardjpyup I saw. looks better. Will test tonight.17:02
evrardjpIt would be great to merge it before EOD this way we can earn one day for periodic tests17:02
evrardjpafk again17:03
cloudnull++17:04
*** chyka has joined #openstack-ansible17:08
ivvecan the link local ha network tenants be deleted?17:09
ivvedon't feel like trying, but that would screw shit up pretty badly i suppose17:09
ivveif someone would by mistake or otherwise17:09
*** dcdamien has joined #openstack-ansible17:18
dcdamien> MySQL data is automatically backed up. To recover the database, use the database backups and rebuild the Galera cluster.17:18
dcdamienthis is from documentation. Can anybody explain what mechanism controls it?17:20
dcdamienI have checked playbooks and galera-server role but I don't see any related vars or tasks17:21
*** tobberydberg has quit IRC17:23
*** tobberydberg has joined #openstack-ansible17:24
*** vnogin has quit IRC17:24
dcdamienThis is from: https://docs.openstack.org/openstack-ansible/pike/admin/maintenance-tasks/backups.html <- it refers to `galera-cluster-maintenance` but there is nothing explained too17:25
*** shardy is now known as shardy_afk17:32
*** gkadam has quit IRC17:37
openstackgerritOpenStack Proposal Bot proposed openstack/openstack-ansible-os_monasca-agent stable/pike: Updated from global requirements  https://review.openstack.org/53485017:37
openstackgerritFrank Zhang proposed openstack/openstack-ansible-openstack_hosts master: Remove period sign before domain name  https://review.openstack.org/53485217:43
*** esberglu has quit IRC17:44
cloudnullivve: I want to say those networks can be deleted and should be recreated with an agent restart17:47
cloudnulldcdamien: lookin17:47
cloudnulldcdamien: here is the maintenance docs https://docs.openstack.org/openstack-ansible/pike/admin/maintenance-tasks/galera.html17:49
cloudnullas for the backup, I'm not sure that portion of the doc is true.17:49
cloudnullI think at one point we had holland running in the galera container17:49
cloudnullbut someone else might have some perspective on that ?17:50
odyssey4meI think we had it running on the host, because the data's in a bind mount from container to host17:50
cloudnullah that may be .17:51
*** alefra has quit IRC18:01
*** SmearedBeard has quit IRC18:02
*** sbra has joined #openstack-ansible18:04
*** mbuil has quit IRC18:04
*** shardy_afk has quit IRC18:10
openstackgerritMerged openstack/openstack-ansible-os_monasca-agent stable/pike: Updated from global requirements  https://review.openstack.org/53485018:14
*** ianychoi has quit IRC18:14
*** ianychoi has joined #openstack-ansible18:15
*** vnogin has joined #openstack-ansible18:16
*** vnogin has quit IRC18:21
*** germs has joined #openstack-ansible18:21
*** pbandark has quit IRC18:22
*** stuartgr has quit IRC18:39
*** openstackstatus has quit IRC18:40
*** openstackstatus has joined #openstack-ansible18:40
*** ChanServ sets mode: +v openstackstatus18:40
*** idlemind has quit IRC18:44
mhaydeni'm stumped on this tempest stuff in master18:45
mhaydenhttps://bugs.launchpad.net/openstack-ansible/+bug/174359918:46
openstackLaunchpad bug 1743599 in openstack-ansible "os_tempest fails to create a workspace" [Undecided,In progress] - Assigned to Major Hayden (rackerhacker)18:46
mhaydenit's almost like the venv has issues18:46
mhaydenthe stuff from 'bin' isn't in the $PATH and then tempest can't find subunit (although it is installed in the venv)18:46
*** idlemind has joined #openstack-ansible18:49
*** poopcat has joined #openstack-ansible19:01
*** poopcat1 has joined #openstack-ansible19:02
*** poopcat has quit IRC19:04
*** ivve has quit IRC19:05
*** hamza21 has joined #openstack-ansible19:14
*** ivve has joined #openstack-ansible19:19
*** esberglu has joined #openstack-ansible19:19
*** armaan has joined #openstack-ansible19:21
*** openstacking_123 has joined #openstack-ansible19:26
mgariepycloudnull, ping()19:38
cloudnullyo!19:39
mgariepydid you had any chance to play with the Epyc server ?19:39
cloudnullno,19:40
cloudnullI've done ryzen and threadripper :)19:40
openstackgerritMajor Hayden proposed openstack/openstack-ansible-os_nova master: Fix SELinux file contexts for nova's ssh keys  https://review.openstack.org/53489119:46
*** vnogin has joined #openstack-ansible19:49
*** cshen_ has joined #openstack-ansible19:50
*** vnogin has quit IRC19:54
*** cmart has quit IRC19:59
*** hybridpollo has joined #openstack-ansible20:07
*** gkadam has joined #openstack-ansible20:10
evrardjpcloudnull: are you there?20:17
evrardjpFYI, with your patch and my patch that you updated, we have this: http://paste.openstack.org/show/646542/20:17
evrardjpso i expect there is an inventory issue20:18
evrardjpthe fix make it work with ansible, but there is still something wrong going on20:18
cloudnullevrardjp: I'm still here.20:19
cloudnulllet me look into that.20:20
evrardjpthanks!20:21
evrardjpwant me to drop you my inventory somewhere?20:21
evrardjpit could probably help you20:21
*** cmart has joined #openstack-ansible20:24
openstackgerritOpenStack Proposal Bot proposed openstack/ansible-hardening master: Updated from global requirements  https://review.openstack.org/53490120:27
cloudnullevrardjp: sure.20:27
cloudnullemail it to me?20:27
evrardjpyeah I will do it as soon as I can20:28
evrardjpprobably on a pastebin would be enough20:28
cloudnullok.20:28
cloudnull++20:28
*** hw_wutianwei has quit IRC20:31
*** hw_wutianwei has joined #openstack-ansible20:32
*** cshen_ has quit IRC20:34
-openstackstatus- NOTICE: Zuul will be offline for a few minutes; existing changes will be re-enqueued; approvals during the downtime will need to be re-added.20:38
evrardjpcloudnull: https://paste.ubuntu.com/26406550/20:38
evrardjpok I am off. Will spend a better full day on stuff tomorrow! towards reliable upgrades and gates!20:44
*** hamza21 has quit IRC20:47
*** hamza21 has joined #openstack-ansible20:47
*** SmearedBeard has joined #openstack-ansible21:01
*** wspthr has quit IRC21:02
*** wspthr has joined #openstack-ansible21:05
*** hamza21 has quit IRC21:12
openstackgerritMajor Hayden proposed openstack/openstack-ansible-os_nova master: Fix SELinux file contexts for nova's ssh keys  https://review.openstack.org/53489121:17
*** smatzek has quit IRC21:24
*** wspthr has quit IRC21:35
*** wspthr has joined #openstack-ansible21:37
*** threestrands_ has joined #openstack-ansible21:44
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible master: Cleanup environment before bootstrap-ansible in upgrades  https://review.openstack.org/53112321:44
cloudnullevrardjp: ^ I think that makes it go21:44
*** threestrands_ has quit IRC21:45
*** threestrands_ has joined #openstack-ansible21:45
*** Smeared_Beard has joined #openstack-ansible21:54
*** SmearedBeard has quit IRC21:56
*** cmart has quit IRC21:59
*** sbra has quit IRC22:08
SamYaplego speed racer go22:13
*** jwitko_ has joined #openstack-ansible22:20
*** Smeared_Beard has quit IRC22:24
*** jwitko has quit IRC22:24
*** vnogin has joined #openstack-ansible22:25
*** jwitko_ has quit IRC22:27
*** gkadam has quit IRC22:29
*** vnogin has quit IRC22:30
*** jwitko has joined #openstack-ansible22:53
*** Guest52 has joined #openstack-ansible22:55
*** cjloader_ has joined #openstack-ansible23:01
*** cjloader has quit IRC23:01
*** dcdamien has quit IRC23:04
*** dcdamien has joined #openstack-ansible23:04
*** cjloader_ has quit IRC23:06
*** persia has quit IRC23:18
*** persia has joined #openstack-ansible23:18
*** dcdamien has quit IRC23:30
*** dcdamien has joined #openstack-ansible23:31
*** smatzek has joined #openstack-ansible23:35
*** smatzek has quit IRC23:39
openstackgerritMerged openstack/ansible-hardening master: Updated from global requirements  https://review.openstack.org/53490123:48
*** cjloader has joined #openstack-ansible23:52
*** pmannidi has joined #openstack-ansible23:52
*** persia has quit IRC23:53
*** openstacking_123 has quit IRC23:55
*** cjloader has quit IRC23:57

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!