Wednesday, 2018-07-25

*** hamzy_ has joined #openstack-ansible00:14
*** tux_ has joined #openstack-ansible00:20
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Further tune the thread pool and add dynamic GC  https://review.openstack.org/58556700:27
*** dave-mcc_ has quit IRC00:35
openstackgerritMerged openstack/openstack-ansible-ops master: Add unified setup and beat config  https://review.openstack.org/58555600:47
*** gregoryo has joined #openstack-ansible00:52
*** tux_ has quit IRC01:19
*** tux_ has joined #openstack-ansible01:20
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Correct kafka template file name  https://review.openstack.org/58558201:33
*** dave-mccowan has joined #openstack-ansible01:35
*** gyee has quit IRC01:55
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Further tune the thread pool and add dynamic GC  https://review.openstack.org/58556702:06
*** mmercer has quit IRC02:11
*** cshen has joined #openstack-ansible02:17
openstackgerritMerged openstack/openstack-ansible-ops master: Correct kafka template file name  https://review.openstack.org/58558202:17
*** cshen has quit IRC02:22
*** dave-mccowan has quit IRC02:29
*** dave-mccowan has joined #openstack-ansible02:30
*** cshen has joined #openstack-ansible02:36
*** cshen has quit IRC02:40
*** cshen has joined #openstack-ansible02:46
*** cshen has quit IRC02:51
*** pmannidi_ has joined #openstack-ansible03:02
*** pmannidi_ has quit IRC03:02
*** dave-mcc_ has joined #openstack-ansible03:09
*** dave-mccowan has quit IRC03:12
*** dave-mcc_ has quit IRC03:12
*** armaan has joined #openstack-ansible03:13
openstackgerritMerged openstack/openstack-ansible-os_barbican master: fix tox python3 overrides  https://review.openstack.org/58001603:18
openstackgerritMerged openstack/openstack-ansible-ops master: Adds tweaks to run MNAIO on Ubuntu 18.04  https://review.openstack.org/58500603:31
*** poopcat1 has quit IRC03:43
*** poopcat1 has joined #openstack-ansible03:43
*** poopcat1 has quit IRC04:03
*** armaan has quit IRC04:19
*** armaan has joined #openstack-ansible04:20
*** cshen has joined #openstack-ansible04:34
*** cshen has quit IRC04:38
*** udesale has joined #openstack-ansible04:39
*** udesale has quit IRC04:42
*** udesale has joined #openstack-ansible04:43
*** armaan has quit IRC04:56
*** armaan has joined #openstack-ansible04:56
*** udesale_ has joined #openstack-ansible05:13
*** udesale has quit IRC05:15
*** armaan has quit IRC05:17
*** armaan has joined #openstack-ansible05:18
*** udesale_ has quit IRC05:19
*** udesale has joined #openstack-ansible05:20
tux_Folks any idea why my LIVE migration didn't work http://paste.openstack.org/show/726576/05:23
*** tux_ has quit IRC05:27
*** tux_ has joined #openstack-ansible05:28
*** tux_ has quit IRC05:32
*** armaan has quit IRC05:32
*** armaan has joined #openstack-ansible05:32
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Further tune the thread pool and add dynamic GC  https://review.openstack.org/58556705:41
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Further tune the thread pool and add dynamic GC  https://review.openstack.org/58556705:44
*** tux_ has joined #openstack-ansible05:58
*** tux_ has quit IRC06:01
*** armaan has quit IRC06:04
*** armaan has joined #openstack-ansible06:05
*** lvdombrkr has joined #openstack-ansible06:17
*** speedmann has quit IRC06:20
*** speedmann has joined #openstack-ansible06:20
*** lvdombrkr89 has joined #openstack-ansible06:21
*** speedmann has quit IRC06:21
*** speedmann has joined #openstack-ansible06:21
*** armaan has quit IRC06:22
*** armaan has joined #openstack-ansible06:22
*** speedmann has quit IRC06:22
*** speedmann has joined #openstack-ansible06:22
*** speedmann has quit IRC06:23
*** speedmann has joined #openstack-ansible06:23
*** lvdombrkr has quit IRC06:24
*** speedmann has quit IRC06:24
*** speedmann has joined #openstack-ansible06:24
*** speedmann has quit IRC06:25
*** cshen has joined #openstack-ansible06:26
*** chkumar|rover has joined #openstack-ansible06:27
*** tux_ has joined #openstack-ansible06:29
*** tux_ has quit IRC06:30
*** pcaruana has joined #openstack-ansible06:33
*** sm806 has quit IRC06:36
*** sm806 has joined #openstack-ansible06:37
*** tux_ has joined #openstack-ansible06:40
*** tux_ has quit IRC06:40
*** jwitko has quit IRC06:48
jrossercan we move this on? https://review.openstack.org/#/c/578348/06:51
*** lvdombrkr has joined #openstack-ansible06:54
*** lvdombrkr89 has quit IRC06:58
*** yolanda_ has quit IRC07:00
*** adrianreza has joined #openstack-ansible07:05
*** rmart04 has joined #openstack-ansible07:08
*** armaan has quit IRC07:09
*** armaan has joined #openstack-ansible07:10
*** shardy has joined #openstack-ansible07:14
*** armaan has quit IRC07:14
*** shardy has quit IRC07:15
*** gkadam has joined #openstack-ansible07:16
*** epalper has joined #openstack-ansible07:18
odyssey4memorning folks07:19
jrossero/07:21
odyssey4mejrosser: what happens when there is no fqdn configured on the host/container07:21
odyssey4meoh, I guess we modify the hosts file or ensure it's in unbound, so that's never the case07:22
openstackgerritMerged openstack/openstack-ansible-repo_build master: Add systemd-devel/libsystemd-dev for systemd-python wheel build  https://review.openstack.org/58502607:22
*** yolanda has joined #openstack-ansible07:22
odyssey4meseems sensible to me07:22
jrosseryes that07:23
jrosserdoesnt it get some variant on .local in that case07:23
*** rmart04_ has joined #openstack-ansible07:24
openstackgerritMerged openstack/openstack-ansible-os_barbican stable/queens: Add default roles used by Barbican  https://review.openstack.org/58554007:24
jrosserodyssey4me: i think theres a previous case of this linked from the bug if you want to check that out too07:25
*** shardy has joined #openstack-ansible07:25
*** rmart04 has quit IRC07:25
*** rmart04_ is now known as rmart0407:25
odyssey4mejrosser: yeah, I read the bug and all that - it's all good, on it's way to merging07:25
jrossercool07:26
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_swift master: Switch swift to use different inventory_hostname  https://review.openstack.org/58463307:28
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_swift master: Move MQ vhost/user creation into role  https://review.openstack.org/58339307:34
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_swift master: Move MQ vhost/user creation into role  https://review.openstack.org/58339307:34
*** hachi has joined #openstack-ansible07:36
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_swift master: Remove httplib  https://review.openstack.org/58393307:39
*** monty has quit IRC07:48
*** monty has joined #openstack-ansible07:48
odyssey4me:( getting constant timeouts from https://review.openstack.org/#/c/585157/ for centos, always during the tempest testing07:51
odyssey4melooks like it just runs long07:53
*** qwebirc12997 has joined #openstack-ansible07:54
qwebirc12997hey guys! I want to add flat network as post installation to my setup what should i do ? just add a part in user_config.yaml and run all scripts from first?07:55
odyssey4mecould we get some eyes on these backports please? https://review.openstack.org/#/q/I38002a18dff4dbd44bd3df9e1969c9d50ab1270507:57
*** tosky has joined #openstack-ansible08:02
*** lvdombrkr89 has joined #openstack-ansible08:03
*** armaan has joined #openstack-ansible08:05
*** lvdombrkr has quit IRC08:06
openstackgerritMerged openstack/openstack-ansible-os_searchlight master: Add source code repository notes link to README  https://review.openstack.org/58308408:07
*** vnogin has joined #openstack-ansible08:07
evrardjpmorning everyone08:08
odyssey4meo/ evrardjp08:10
evrardjpI should have pinged at early day start :p08:10
*** lukaslemes has joined #openstack-ansible08:13
openstackgerritMerged openstack/openstack-ansible-os_horizon master: Change default horizon_server_name to ansible_fqdn  https://review.openstack.org/57834808:18
*** rmart04 has quit IRC08:19
noonedeadpunkodyssey4me: I've got question about your comment here https://review.openstack.org/#/c/584629/6/tasks/masakari_install.yml08:24
noonedeadpunkodyssey4me with this conditional masakari monitors package are installed in the masakari venv, as they doesn't have heir own venv.08:25
noonedeadpunkodyssey4me: But I don't fully realise they way, how this should be fixed. Should I create venv on the repo hosts? If yes, could you please point me to some example or how-to how this should be done.08:27
odyssey4menoonedeadpunk: ok, so you're saying that all hosts in the host groups for masakari will all get the venv, but some will get more packages put into the venvs later?08:27
noonedeadpunkodyssey4me: yep. This is what I see in my sandbox08:28
odyssey4meis there a reason not to just put everything in the venv instead? doe the monitor packages conflict or something?08:28
openstackgerritMerged openstack/openstack-ansible-os_designate master: Use designate_galera_database  https://review.openstack.org/58552308:28
noonedeadpunkodyssey4me: there is no reason, but I just don't know how make it created by repo-build role.08:29
qwebirc12997guys can i build flat network on br-mgmt ?08:29
odyssey4menoonedeadpunk: move the content of masakari_monitor_pip_packages into masakari_pip_packages so there's a single venv with everything in it08:29
noonedeadpunkodyssey4me: so we'll have extra services on every masakari host?08:30
Tahvokqwebirc12997: creating the networks and on-top of openstack configuration is not part of openstack-ansible08:31
odyssey4menoonedeadpunk: no, just the packages - you don't have to setup the service on every host, but there's no harm in the packages being in the venv08:31
noonedeadpunkodyssey4me: Yep, got it08:32
qwebirc12997Tahvok because when i try to create flat network on br-vlan i've got this error and i dont know what's the problem and what i should do ? http://paste.openstack.org/show/726581/08:32
odyssey4mewe will, in the future, be able to generate more than one venv per role - but right now that's not possible - the repo build process just doesn't have that capability08:32
Tahvokqwebirc12997: thinking of a workaround is not a solution to the problem..08:32
odyssey4meI'm moving the repo build functionality into the roles, and when that's done you'll be able to do builds much more flexibly08:32
odyssey4menoonedeadpunk: note that I just added another comment about the libvirt-python package08:33
qwebirc12997Tahvok so what is the solution ? :(08:33
noonedeadpunknoonedeadpunk: yes, I'm already reading it08:33
*** gregoryo has quit IRC08:34
jrosserqwebirc12997: doesnt a flat network have to use an actual interface instead of a bridge - see the use of eth12 in the OSA example configs08:35
noonedeadpunkodyssey4me: sorry for so many mistakes and some stupid questions - I saw OSA first time 2 month ago, and working with openstack only for half a year, so I have big laps in my knowledge.08:35
qwebirc12997jrosser in this user_config.yaml setup http://paste.openstack.org/show/726583/ shoud i have actual eth12 interface for flat network ?08:38
odyssey4menoonedeadpunk: absolutely no problem at all - you're learning and improving things and doing well :)08:38
qwebirc12997i'm sorry that my questions are very simple. i do not know how to understand openstack networking08:38
qwebirc12997openstack ansible*08:38
odyssey4methese are not obvious things, unfortunately08:38
jrosserqwebirc12997: see the example here https://docs.openstack.org/openstack-ansible/latest/user/test/example.html08:42
jrossereth12 is wired to br-vlan with a veth pair08:42
jrossereth12 is then used in the neutron provider network config as neutron expects an interface, not a bridge in this case08:42
jrosserthis means that untagged packets on br-vlan are your flat network08:43
jrosserbear in mind this is just an example provided with OSA, you can arrange it however suits your use case08:43
noonedeadpunkodyssey4me: can you please point me to some example, how endpoint are added to haproxy, as masakari isn't present in it08:43
*** shardy has quit IRC08:44
*** shardy has joined #openstack-ansible08:44
*** armaan has quit IRC08:46
qwebirc12997jrosser thank you vert much08:46
qwebirc12997complete and helpful08:47
*** armaan has joined #openstack-ansible08:47
noonedeadpunkodyssey4me: if we're merging list of required pip packages, that we should also merge distro packages https://review.openstack.org/#/c/584629/6/vars/ubuntu-16.04.yml08:48
*** openstackgerrit has quit IRC08:48
noonedeadpunkis it ok?08:49
*** armaan has quit IRC08:51
*** pmannidi has quit IRC08:53
noonedeadpunknoonedeadpunk: as than we should map libvirt-python in the way you've offered for all masakari hosts08:58
noonedeadpunkodyssey4me: as than we should map libvirt-python in the way you've offered for all masakari hosts08:58
*** openstackgerrit has joined #openstack-ansible08:59
openstackgerritMerged openstack/openstack-ansible-os_swift master: Remove pypy job  https://review.openstack.org/58428708:59
openstackgerritChristian Zunker proposed openstack/openstack-ansible-os_heat master: Explain how to configure public endpoint  https://review.openstack.org/58291909:03
noonedeadpunkodyssey4me or probably we shouldn't as this could be done by block with when statement09:04
*** qwebirc12997 has quit IRC09:04
noonedeadpunknever mind)09:08
*** armaan has joined #openstack-ansible09:11
odyssey4menoonedeadpunk: or could be done with include_tasks and a when statement, yes09:11
odyssey4meand yeah, I guess the distro package list would need to be merged too09:12
admin0morning all .09:12
odyssey4meguilhermesp: you can use the 'cherry-pick' button in gerrit to backport https://review.openstack.org/585523 to the stable branches09:14
*** rmart04 has joined #openstack-ansible09:14
*** chason has quit IRC09:16
*** chason has joined #openstack-ansible09:17
*** armaan has quit IRC09:26
*** armaan has joined #openstack-ansible09:26
*** armaan has quit IRC09:30
noonedeadpunkodyssey4me I really need you help with one thing. I've spent several hours but without result. I would like to dynamically set an interface in this tempate https://review.openstack.org/#/c/584629/6/templates/masakarimonitors.conf.j2 So I've got following expression http://paste.openstack.org/show/726597/, but if statement for some reason does not work (while  all variales are correct and it should work....)09:30
openstackgerritMerged openstack/openstack-ansible-os_horizon master: tasks: horizon_post_install: Ensure horizon dashboard directory is accessible  https://review.openstack.org/58432709:33
*** cshen has quit IRC09:35
openstackgerritMerged openstack/openstack-ansible-os_horizon master: tasks: horizon_post_install: Add missing become statements for horizon user  https://review.openstack.org/58432809:35
*** shardy has quit IRC09:45
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Use a static inventory skeleton  https://review.openstack.org/58036809:48
*** Tahvok has quit IRC09:50
*** shardy has joined #openstack-ansible09:50
*** Tahvok has joined #openstack-ansible09:51
*** rmart04 has quit IRC10:03
*** shyamb has joined #openstack-ansible10:06
*** chason has quit IRC10:15
*** chason has joined #openstack-ansible10:17
*** shyambiradar has joined #openstack-ansible10:17
*** chason[m] has joined #openstack-ansible10:17
*** jamesden_ has joined #openstack-ansible10:22
*** shyambiradar has quit IRC10:22
*** kukacz_ has quit IRC10:23
*** shyamb has quit IRC10:23
*** kukacz_ has joined #openstack-ansible10:23
*** cshen has joined #openstack-ansible10:27
*** lvdombrkr has joined #openstack-ansible10:43
*** lvdombrkr89 has quit IRC10:44
*** armaan has joined #openstack-ansible10:45
*** lvdombrkr89 has joined #openstack-ansible10:45
*** lvdombrkr has quit IRC10:48
*** armaan has quit IRC10:55
*** armaan has joined #openstack-ansible10:56
*** armaan has quit IRC10:56
odyssey4menoonedeadpunk: I'm tied up with my day job right now, hopefully someone else can help.10:56
noonedeadpunkodyssey4me: ok no problem10:58
*** shyambiradar has joined #openstack-ansible10:59
*** shyamb has joined #openstack-ansible10:59
*** shyambiradar has quit IRC11:00
*** shyambiradar has joined #openstack-ansible11:00
*** shyambiradar has quit IRC11:00
openstackgerritMerged openstack/openstack-ansible-os_horizon master: tasks: horizon_post_install: Add 'force' parameter when creating symlink  https://review.openstack.org/58432911:04
openstackgerritAndy Smith proposed openstack/openstack-ansible-os_tacker master: Update to use oslo.messaging service for RPC and Notify  https://review.openstack.org/58570911:04
*** armaan has joined #openstack-ansible11:07
jamesden_I'm trying to run some functional tests within the os_neutron role, and it's complaining that "Could not find a version that satisfies the requirement ansible==2.5.5 (from -c /etc/ansible/roles/os_neutron/tests/common/test-ansible-deps.txt (line 11)) (from versions: 2.4.3.0)" Yet, I'm running 2.5.5.11:13
*** cshen has quit IRC11:17
*** chyka has joined #openstack-ansible11:18
noonedeadpunkOnce I've removed python-libvirt installation, https://review.openstack.org/#/c/584629/ fails with http://paste.openstack.org/show/726607/. So it seems, that I need that module at least for testing11:18
noonedeadpunkwhat developer_constraints are designed for? It must be a git repo, or it is possible to specify git package as well?11:22
*** chyka has quit IRC11:23
odyssey4menoonedeadpunk: it's for the role tests, where developer mode is active, but is not useful for the integrated build11:24
*** armaan has quit IRC11:27
*** armaan has joined #openstack-ansible11:27
*** cshen has joined #openstack-ansible11:27
*** armaan has quit IRC11:29
noonedeadpunkodyssey4me: actually, as tests does not install nova, I don't have python-libvirt in test environment. So I eiser need to include nova for test, or somehow install libvirt-python11:30
*** armaan has joined #openstack-ansible11:30
odyssey4menoonedeadpunk: I suggested that you copy the same task set into the masakari role, because you'd need to link the system python-libvirt into the venv. Then you'll need to ensure that you install libvirt as part of the role too.11:32
*** vakuznet has quit IRC11:42
guilhermespmonrning!11:48
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_swift master: Fix usage of "|" for tests  https://review.openstack.org/58226511:48
guilhermespodyssey4me: hum, k. Going to try here11:49
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Freeze OpenStack-Ansible roles for Milestone 3  https://review.openstack.org/58572011:57
evrardjpjamesden_: are you running this on a machine that's already deployed osa by any chance?12:00
evrardjprole test can be done a clean node so it doesn't interfere12:00
jamesden_evrardjp yes, i realized that pip.conf was interfering12:01
evrardjpodyssey4me: did you already did the voting: false on swift?12:01
evrardjpjamesden_: yeah :)12:01
jamesden_thank you!12:01
jamesden_i was impatient and didn't want to stand up a new node12:01
openstackgerritGuilherme  Steinmuller Pimentel proposed openstack/openstack-ansible-os_designate stable/queens: Use designate_galera_database  https://review.openstack.org/58572212:01
evrardjpodyssey4me: want me to push up a patch for swift multireg non voting?12:01
odyssey4meevrardjp: dude, that already merged12:02
guilhermesp^^ backport to pike and ocata too?12:02
evrardjpodyssey4me: https://github.com/openstack/openstack-ansible-os_swift/blob/master/zuul.d/jobs.yaml#L1812:03
evrardjpthat's confusing as hell12:03
odyssey4meguilhermesp: yes, I think so12:03
evrardjpthe job below has nv12:03
guilhermespindeed, they have the same issue12:03
*** armaan has quit IRC12:03
odyssey4meevrardjp: sure, I know - but it's supposed to be temporary12:03
evrardjpok12:03
*** shardy has quit IRC12:03
*** armaan has joined #openstack-ansible12:04
odyssey4meI'm actually inclined to remove that whole MR test, given that it's fairly unmaintainable in the current state.12:04
odyssey4meBut I'll see if I can figure it out, maybe - or move it to a proper multi-node test.12:05
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Unfreeze roles after M3  https://review.openstack.org/58572412:05
*** armaan has quit IRC12:06
*** armaan has joined #openstack-ansible12:07
evrardjpcores, I'd be happy if I can have a few reviews there: https://review.openstack.org/#/q/owner:%22Jean-Philippe+Evrard+%253Cjean-philippe%2540evrard.me%253E%22++is:mergeable+is:open12:07
evrardjpsorry: https://review.openstack.org/#/q/owner:%22Jean-Philippe+Evrard+%253Cjean-philippe%2540evrard.me%253E%22++is:mergeable+is:open+project:%255Eopenstack/openstack-ansible.*12:08
*** armaan has quit IRC12:11
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Remove group vars hack  https://review.openstack.org/58441112:12
odyssey4meevrardjp lol, same time12:12
*** mwe11 has quit IRC12:14
guilhermesphum, we have merge conflict for pike and ocata. it worth to solve them to proceed the cherry-pick?12:14
evrardjpguilhermesp: well it's up to you to decide if it's worth cherry-picking :p12:15
evrardjpwhat is it about? Is that a bug?12:15
guilhermespthat low hanging fruit of designate :)12:15
evrardjpit's a bug then yeah12:15
evrardjp:)12:15
evrardjpyeah worth fixing where you can12:15
guilhermespk then :)12:15
evrardjpif it's too hard to fix as is, you can inform in the bug12:16
guilhermespgot it evrardjp12:16
evrardjp(but better is trying to fix it if it's a simple merge conflict)12:16
jrosserevrardjp: would https://github.com/openstack/openstack-ansible-repo_server/commit/1bc95198290e8d563e9ca6137e40b5d15edcce25 get into 16.0.8? thats part of the same cleanup as the last 4 commits in osa12:17
guilhermespjust to be sure, all cherry picks must have master as parent?12:17
evrardjp(but if it needs a lot of patches to bring it in, then it's new code/bug fix ration is too high, then probably worth reimplementing)12:17
evrardjpguilhermesp: yes12:17
*** shardy has joined #openstack-ansible12:17
evrardjpjrosser: 17.0.8?12:18
evrardjpor 16.0.18 ?12:18
jrosserevrardjp: sorry my type 17.0.812:18
evrardjpjrosser: https://review.openstack.org/#/c/584796/12:18
*** shardy has quit IRC12:18
odyssey4mejrosser: I think it misses it, we would have had to bump the rol ein time for evrardjp's release request.12:19
evrardjpit misses it12:19
jrosseryeah i was not seeing the repo server sha updated for 17.0.812:19
evrardjpyeah you can do it on a commit on top12:19
odyssey4meI don't think it matters too much, while the rev proxy is there it won't really be used I don't think.12:19
jrosserok, np, so long as it makes sense without it12:19
odyssey4melemme do a patch to put it into 17.0.8 at least12:20
evrardjpodyssey4me: instead of doing so, could you explain jrosser on how to do it?12:21
odyssey4mesure, can do12:21
evrardjpcool thanks -- more sustainable that way : )12:21
guilhermespodyssey4me evrardjp just realized too after cherry-picking for queens, we need to change this variable too ( should be designate_galera_database )12:22
guilhermesphttps://github.com/openstack/openstack-ansible/blob/8165d6345ba54b9415f616417ee9bd0a34cc163a/playbooks/os-designate-install.yml#L5912:22
*** shardy has joined #openstack-ansible12:22
odyssey4mejrosser: the general process is to get the existing 17.0.8 proposal (git review -d 584387) in one terminal, and to ensure you have the repo_build role in another terminal checked out at stable/queens12:23
odyssey4meoops - wrong review up there, should be git review -d 58479612:23
odyssey4methen copy the new release note ftom repo_build (review 584387) into the integrated repo in the correct location (releasenotes/notes/), edit ansible-role-requirements.yml with the updates role SHA, and commit and push it up12:25
odyssey4mehopefully that makes just a little sense12:25
*** dave-mccowan has joined #openstack-ansible12:25
jrosserodyssey4me: lets see how i get on :)12:25
*** tux_ has joined #openstack-ansible12:29
jrosserodyssey4me: is this an amend on the original commit or a new one on top?12:30
odyssey4mejrosser: I guess given that it's not yet merged, may as well amend - but on top is fine too... ask evrardjp if he's happy for you to amend12:31
jrosserevrardjp: ^ ?12:31
evrardjpI'd prefer another patch, as this explains why you want it sooner rather than later, but I won't complain if it's an amend done when a recheck is needed for example :p12:32
evrardjpI just don't want to lose potential winning gates.12:33
jrosseri'll do a new one12:33
tux_Folk did you see this issue in Pike before? http://paste.openstack.org/show/726576/12:34
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible master: Use a static inventory skeleton  https://review.openstack.org/58036812:34
tux_Does pike support that option "migrate_configure_max_speed" ?12:35
evrardjptux_: see configuration guide maybe?12:35
tux_I didn't find that option anywhere also on google search who is using it12:36
tux_https://github.com/openstack/nova/commit/23446a9552b5be3b040278646149a0f481d0a00512:36
evrardjptux_: why don't you use Queens if you want newer features?12:36
tux_I am testing queens in my development environment12:36
tux_I am not yet ready for queen but soon i will12:37
noonedeadpunkodyssey4me: it seems, that I'm not able to setup nova-compute in container, as it doesn't support modrpobe (and some nova's distro packages requires kerel module installation).12:37
evrardjpnoonedeadpunk: nova-compute is not meant for containers12:37
tux_all i want to know does pike has any issue with live migration ? anyone using pike with live migration?12:37
evrardjpwe don't run that in containers by default, why would you want that?12:37
odyssey4menoonedeadpunk: not sure why you're even trying to do that, but it can (the ironic compute service runs in a container)12:38
evrardjpI am not saying it's impossible, but it's harduous for 0 benefit IMO12:38
odyssey4menoonedeadpunk: what are you trying to achieve?12:38
evrardjptux_: live migration works for a long time now12:38
noonedeadpunkevrardjp: I know) I'm trying to implement this comment https://review.openstack.org/#/c/584629/6/defaults/main.yml of odyssey4me12:38
evrardjptux_: admin0 can probably help you :)12:38
noonedeadpunkevrardjp so I need system python-libvirt, which could be linked to masakari's venv12:39
odyssey4menoonedeadpunk: as mentioned before, you don't need to implement nova - just the same tasks from the nova role12:39
tux_evrardjp: thanks i will ask him or just share configuration of libvirtd.conf and nova.conf so i can compare12:39
*** tux_ has quit IRC12:39
noonedeadpunkodyssey4me: so where I should place them? In tests.yml? As I need just python-libvirt out of virtualenv12:40
odyssey4menoonedeadpunk: no, they need to be part of the install process for masakari - otherwise it won't work12:40
evrardjpodyssey4me: I really must do something about those tasks12:40
*** shyamb has quit IRC12:41
noonedeadpunkodyssey4me: but they should be run only when masakari_developer_mode is true?12:41
odyssey4menoonedeadpunk: we used to install python-libvirt, but it's nearly impossible to keep the right python lib version for the distro package with the c bindings... so it's better to symlink the package's python lib into the venv12:41
jrosserodyssey4me: i've made a new patch for this SHA bump and when i 'git review' it wants to push evrardjp's one underneath as well, which is undesireable12:41
odyssey4mejrosser: no, that's correct12:42
odyssey4mewhenever you have a series of patches on top of each other, git review verifies that's what you intend to do12:42
jrosserah, ok, doesnt reset the checks on the one underneath?12:42
*** shyamb has joined #openstack-ansible12:42
odyssey4mejrosser: not if you used 'git review -d' to get the one on the bottom there12:43
openstackgerritJonathan Rosser proposed openstack/openstack-ansible stable/queens: Update repo_server role to remove pypi reverse proxy  https://review.openstack.org/58573512:43
odyssey4meif your copy of its parent is different, then it'll rebase that parent patch - but assuming you did a checkout (git review -d) it'll be the same and leave it12:44
jrosserahha, and it was so12:44
*** mwe1 has joined #openstack-ansible12:44
noonedeadpunkodyssey4me: is it better to install python-libvirt with pip or system package manager then?12:44
jrosserthe warning from git review is quite ominous12:44
odyssey4menoonedeadpunk: system package manager12:45
odyssey4mejrosser: LGTM, thanks!12:45
jrossernp :)12:45
noonedeadpunkodyssey4me: so I just need to move python-libvirt from pip to packages variable? That's too easy)12:46
evrardjpjrosser: Trivia info: Long ago, when you build long chain of patches, you could bring down the whole zuul environment.12:46
evrardjpthat's why there is a big message. Don't do it if you can.12:47
guilhermespguy, before merging the conflict of designate for pike and ocata, I wrote a comment in the bug describing my thoughts about the cherry pick to queens.12:47
odyssey4menoonedeadpunk: I'm not sure how I can make this any clearer. That's only a small part of the thing, the other is to symlink those libs into the venv as referred to in the review.12:47
evrardjpguilhermesp: good12:47
guilhermesp**guys (hate my typing)12:47
evrardjplet me check that guilhermesp12:47
openstackgerritMerged openstack/openstack-ansible-os_horizon master: Add support for using distribution packages for OpenStack services  https://review.openstack.org/58433012:47
evrardjpcould you link to it?12:47
guilhermespyes12:47
evrardjpguilhermesp: there are ladies too, not only guys!12:47
guilhermesphttps://bugs.launchpad.net/openstack-ansible/+bug/178313212:47
openstackLaunchpad bug 1783132 in openstack-ansible "designate.conf.j2 does not respect user configured designate database name" [Medium,Fix released] - Assigned to Guilherme Steinmuller Pimentel (guilhermesp)12:48
evrardjp:p12:48
evrardjpdon't forget them!12:48
guilhermespyes, sorry ladies hahaha ( spotz )12:48
evrardjpguilhermesp: and many others :)12:48
noonedeadpunkodyssey4me: yep I get it. It just was thinking, that python-libvirt is an essential part of nova, so it will be already present, if we're talking not about tests. So I was thinking just how to fix tests, but make sure that package is present globally is a genious trick)12:49
guilhermesphope to meet them all :)12:49
evrardjpguilhermesp: answered. Good job of documentation!12:50
odyssey4menoonedeadpunk: even if you do this on a nova host, the masakari venv won't have access to those libvirt libs - you have to cover both making sure you have what you need on the system, and making sure you have what you need in the venv... never make assumptions ;)12:50
guilhermesp\m/ evrardjp12:51
noonedeadpunkodyssey4me: I did symlink several hours ago, but faced with the problem, that I don't have packge that should be symlinked.12:51
odyssey4meyep, the right package needs to be in your distro package list12:51
noonedeadpunkand went the wrong way12:51
odyssey4mealso, now that you do it that way - you can remove libvirt-devel12:52
odyssey4menoonedeadpunk: ah, I think I understand now - you weren't really asking for help, but rather thinking out loud?12:52
openstackgerritAndy Smith proposed openstack/openstack-ansible-os_watcher master: Update to use oslo.messaging service for RPC and Notify  https://review.openstack.org/58573612:53
*** hachi has quit IRC12:53
noonedeadpunkodyssey4me: Yes, probably I don't need libvirt-devel any more. I just went the wrong way, so I needed to pointing me, and now everything become absolutelly clear, thanks.12:55
*** ssbarnea has quit IRC13:05
*** Tahvok has quit IRC13:08
*** Tahvok has joined #openstack-ansible13:11
*** hachi has joined #openstack-ansible13:13
noonedeadpunkodyssey4me: and yes, in the end I was just thinking out loud. I'll try to keep my thoughts to myself13:16
*** armaan has joined #openstack-ansible13:18
guilhermespso, I assume that this would be backported for the other branches too, right? https://review.openstack.org/#/c/584096/13:20
*** david_ has quit IRC13:21
*** tux_ has joined #openstack-ansible13:24
tux_admin0: are you there buddy ?13:25
*** armaan has quit IRC13:25
noonedeadpunkis it generally enough to run setup-openstack.yml --limit nova_all,compute03 in order to add compute03 as a new compute node?13:27
*** tux__ has joined #openstack-ansible13:27
noonedeadpunkexcept specifying it in openstack_user_config13:27
guilhermespnoonedeadpunk: did you see this? https://docs.openstack.org/openstack-ansible/latest/admin/scale-environment.html#add-a-compute-host13:28
noonedeadpunkguilhermesp: nope, missed it(13:28
noonedeadpunkguilhermesp: hm, I thought that setup-openstack.yml should be run on all hosts, as how nova ssh keys would be distributed over all nodes in this case?13:30
noonedeadpunk*should be run on all nova hosts13:30
*** tux_ has quit IRC13:30
*** dave-mcc_ has joined #openstack-ansible13:33
*** dave-mccowan has quit IRC13:34
*** dave-mccowan has joined #openstack-ansible13:35
*** jwitko has joined #openstack-ansible13:36
*** dave-mcc_ has quit IRC13:38
*** shyamb has quit IRC13:38
guilhermespnoonedeadpunk: in this case seems to be. The job is done by https://github.com/openstack/openstack-ansible-os_nova/blob/234ec64b1381fd181ff66e28e6d41dd82b6d22c0/tasks/nova_compute_key_distribute.yml13:41
guilhermespin queens there's a "--skip-tags nova-key-distribute" https://docs.openstack.org/openstack-ansible/queens/admin/scale-environment.html#add-a-compute-host13:41
noonedeadpunkguilhermesp: ah, yep, it's delegated to localhost, so localhost should be really enough13:44
noonedeadpunkguilhermesp: thanks13:44
*** jwitko has quit IRC13:46
*** hachi has quit IRC13:47
*** hachi has joined #openstack-ansible13:47
*** tux__ has quit IRC13:47
*** jwitko has joined #openstack-ansible13:47
*** tux_ has joined #openstack-ansible14:00
tux_what version of nova we are using with PIKE?14:02
tux_is this correct? https://git.openstack.org/openstack/openstack-ansible-os_nova14:04
guilhermesptux_: https://github.com/openstack/openstack-ansible/blob/0e03f46a2ebb0ffc6f12384f19ec1184434e7a09/playbooks/defaults/repo_packages/openstack_services.yml#L14814:05
guilhermespto be precise, check which tag you are using14:05
tux_16.0.15-7-gab73d4914:07
tux_This is my tag14:07
tux_guilhermesp: can you shared you LIVE migration config?  look like i am missing something and my LIVE migration not woking :(14:07
tux_This is what i have in nova.cong14:08
-tux_- live_migration_uri = "qemu+tcp://%s/system"14:08
-tux_- live_migration_tunnelled = True14:08
tux_Do i need anything else in nova.conf?14:08
tux_guilhermesp: i am seeing this error http://paste.openstack.org/show/726576/14:11
*** tailDASH1 has quit IRC14:12
*** tailDASHeF has joined #openstack-ansible14:12
*** hamzy_ has quit IRC14:13
*** armaan has joined #openstack-ansible14:17
*** tux_ has quit IRC14:18
*** ssbarnea has joined #openstack-ansible14:18
*** tux_ has joined #openstack-ansible14:19
*** armaan_ has joined #openstack-ansible14:19
*** armaan has quit IRC14:21
noonedeadpunkis it worth adding repo-build for masakari test?14:22
odyssey4menoonedeadpunk: the role should not use repo-build, no14:22
openstackgerritMerged openstack/openstack-ansible master: Test the main repo with next version of ansible  https://review.openstack.org/55782014:24
noonedeadpunkodyssey4me: than I have another problem( Part of masakari_install looks like this http://paste.openstack.org/show/726618/  masakari_libvirt.yml - is part for creating symlinks for venv. But at this step in tests we don't have venv.14:24
noonedeadpunkvenv is being created by pip module, and it installs package, which requires python-libvirt.14:25
noonedeadpunkprobably I should add venv creation separatelly, when masakari_get_venv is not changed?14:26
odyssey4menoonedeadpunk: sounds like your ordering is wrong then - that must run after the venv is there14:27
noonedeadpunkyes, but pip command installs masakari-hostmonitor, which requires python-libvirt. otherwise it tries to compile it, and fails (as I've removed libvirt-devel)14:29
noonedeadpunkSo it seems that I need smth in between for tests...14:29
odyssey4menoonedeadpunk: argh, that sucks14:29
odyssey4meI guess that means it gets more complex - you'd have to create the venv, then do the symlinks, then install the hostmonitor.14:30
odyssey4meA possible alternative would be to install the monitor into the nova venv, but then you'd need to have nova installed in that role test which would get complicated in other ways.14:31
*** cshen has quit IRC14:31
*** armaan_ has quit IRC14:31
odyssey4meI think the first option may be better.14:31
*** armaan has joined #openstack-ansible14:32
noonedeadpunkodyssey4me: I've already realized that it's pretty hard to install nove into container...14:32
*** armaan has quit IRC14:32
noonedeadpunkodyssey4me: is pip module able to just create venv without package installation in it? And should I lock this way only for developer mode by when statement?14:33
*** Connection has joined #openstack-ansible14:33
odyssey4menoonedeadpunk: I haven't tried, but I expect it can.14:34
noonedeadpunknoonedeadpunk: same thing for me14:34
*** Connection has quit IRC14:35
openstackgerritDirk Mueller proposed openstack/openstack-ansible-os_horizon master: Reenable designate dashboard for openSUSE  https://review.openstack.org/58576314:35
*** armaan has joined #openstack-ansible14:37
*** hamzy_ has joined #openstack-ansible14:38
*** hamzy_ is now known as hamzy14:38
*** armaan has quit IRC14:42
*** udesale has quit IRC14:48
*** pcaruana has quit IRC14:52
openstackgerritAndy Smith proposed openstack/openstack-ansible-os_zun master: Update to use oslo.messaging service for RPC and Notify  https://review.openstack.org/58577314:55
*** armaan has joined #openstack-ansible15:02
*** flaviosr has quit IRC15:03
*** Guest14486 has joined #openstack-ansible15:08
Guest14486Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/15:08
Guest14486or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/15:08
Guest14486Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate15:08
Guest14486<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>15:08
Guest14486This message was brought to you by Private Internet Access15:09
*** StephenS28 has joined #openstack-ansible15:09
*** Guest14486 has quit IRC15:11
*** StephenS28 has quit IRC15:11
*** epalper has quit IRC15:13
*** armaan has quit IRC15:14
*** armaan has joined #openstack-ansible15:14
*** armaan has quit IRC15:14
*** Levex12 has joined #openstack-ansible15:23
tux_Does like has this patch ? https://github.com/openstack/nova/commit/23446a9552b5be3b040278646149a0f481d0a00515:23
Levex12Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/15:23
Levex12or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/15:24
Levex12Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate15:24
Levex12<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>15:24
Levex12This message was brought to you by Private Internet Access15:24
*** Levex12 has quit IRC15:24
*** gyee has joined #openstack-ansible15:29
*** vnogin has quit IRC15:29
*** armaan has joined #openstack-ansible15:29
tux_Look like queen has this patch15:29
openstackgerritGerman Eichberger proposed openstack/openstack-ansible-os_octavia master: Fix quota for Octavia  https://review.openstack.org/57136815:29
tux_test_migrate_configure_max_speed15:29
tux_https://github.com/openstack/nova/commit/23446a9552b5be3b040278646149a0f481d0a00515:30
tux_That would be my solution to fix my LOVE or LIVE migration15:30
tux_if anyone using pike please let me know i would like to understand this issue15:30
*** Leo_m has joined #openstack-ansible15:32
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-ops master: Allow mounting of shared filesystems for index backup/restore  https://review.openstack.org/58578915:37
*** chyka has joined #openstack-ansible15:39
*** lvdombrkr89 has quit IRC15:41
*** lukaslemes has quit IRC15:44
tux_This is frustrating to me :)15:50
tux_I am only seeing error on horizon "Error: Failed to live migrate instance to host "ostack-compute-01"15:50
tux_How do i troubleshoot this issue? no indicating in logs or anywhere15:51
*** Strog16 has joined #openstack-ansible15:51
Strog16Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/15:51
Strog16or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/15:51
Strog16Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate15:51
Strog16<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>15:51
Strog16This message was brought to you by Private Internet Access15:51
*** Strog16 has quit IRC15:51
*** gkadam has quit IRC15:52
noonedeadpunkodyssey4me: another problem... it seems, that pip tries to update libvirt-python and fails at that step...  http://paste.openstack.org/show/726624/15:52
*** Remco12 has joined #openstack-ansible15:53
Remco12Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/15:53
Remco12or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/15:53
noonedeadpunkprobably masakari_pip_package_state should be present instead of latest?15:53
Remco12Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate15:53
Remco12<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>15:53
Remco12This message was brought to you by Private Internet Access15:53
*** Remco12 has quit IRC15:53
noonedeadpunkodyssey4me: as masakari-monitors do net require libvirt-python more than 3.5 https://github.com/openstack/masakari-monitors/blob/master/requirements.txt15:55
*** armaan has quit IRC15:58
*** dave-mccowan has quit IRC15:58
*** armaan has joined #openstack-ansible15:58
*** armaan has quit IRC15:59
*** dave-mccowan has joined #openstack-ansible16:00
*** armaan has joined #openstack-ansible16:00
*** eumel8 has joined #openstack-ansible16:01
openstackgerritJonathan Rosser proposed openstack/openstack-ansible-ops master: Allow mounting of shared filesystems for index backup/restore  https://review.openstack.org/58578916:01
cloudnullvery nice indeed16:05
cloudnulljrosser ^16:05
tux_Exception during message handling: DestinationDiskExists: The supplied disk path (/var/lib/nova/instances/c207dce7-c545-4f13-af75-a78c2d4d73c7) already exists, it is expected not to exist.16:07
tux_Now i am getting this error in live migration16:08
tux_I have /var/lib/nova/instance/c207dce7-c545-4f13-af75-a78c2d4d73c7/console.log16:10
tux_who delete this directory during migration?16:11
noonedeadpunkoh, I didn't know, that we have a playbook, that makes possible to forward logs from rsyslog to graylog server...16:12
tux_may be i need "destroy_after_evacuate = True" in nova.conf16:12
jrossercloudnull: thanks :) i managed to get my cluster back online16:14
jrosserdelete a couple of packetbeat indexes using the kibana gui and leave it overnight16:14
jrosserthis morning it was all rebalanced with the right number of replicas16:14
jrossercloudnull: the installation of jmesquery might need a bit of a tweak16:15
jrosserthe bootstrap script isnt so smart in the embedded env already exists16:15
jrosserin/if16:15
*** armaan has quit IRC16:16
*** armaan has joined #openstack-ansible16:16
noonedeadpunkis there any plans to make logs forwarding to existing/new graylog as a part of rsyslog role?16:18
*** jack34 has joined #openstack-ansible16:22
*** jack34 has quit IRC16:23
*** chkumar|rover is now known as chandankumar16:24
tux_I badly need help if someone help me out :(16:25
tux_This is the last piece left to move forward in production16:25
tux_Live migration failed.: AttributeError: 'Guest' object has no attribute 'migrate_configure_max_speed'16:25
tux_anyone seen this error ?16:26
noonedeadpunkjrosser: sorry, do you probably know, how to easy forward logs to the greylog server? Do I need rsyslog server installation for that?16:26
jrosserthere is a variable that allows you to point rsyslog at places in addition to the rsyslog container16:27
jrosserjust in a meeting will look later if you can't find it16:27
noonedeadpunkjrosser ok, thanks for pointing me - it seems, that i'm totally blind. Probably it's rsyslog_client_user_defined_targets16:34
kaiokmotux: can you paste the libvirtd.conf of your compute nodes involved in the migration?16:38
*** foddo9 has joined #openstack-ansible16:41
foddo9Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/16:41
foddo9or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/16:41
foddo9Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate16:41
foddo9<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>16:41
foddo9This message was brought to you by Private Internet Access16:41
*** foddo9 has quit IRC16:42
openstackgerritDmitriy Rabotjagov (noonedeadpunk) proposed openstack/openstack-ansible-os_masakari master: Basic implemintation of masakari-monitors  https://review.openstack.org/58462916:42
*** Dominian10 has joined #openstack-ansible16:43
Dominian10Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/16:43
Dominian10or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/16:43
Dominian10Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate16:43
Dominian10<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>16:43
Dominian10This message was brought to you by Private Internet Access16:43
*** Dominian10 has quit IRC16:43
*** cshen has joined #openstack-ansible16:44
*** WikiPuppies10 has joined #openstack-ansible16:48
WikiPuppies10Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/16:48
WikiPuppies10or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/16:48
WikiPuppies10Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate16:48
WikiPuppies10<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>16:48
WikiPuppies10This message was brought to you by Private Internet Access16:48
*** WikiPuppies10 has quit IRC16:48
tux_In shared storage case (Ceph) do i need to setup SSH-key between two nova compute node?16:50
*** lmartin928 has joined #openstack-ansible16:51
lmartin928Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/16:51
lmartin928or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/16:51
lmartin928Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate16:51
lmartin928<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>16:51
lmartin928This message was brought to you by Private Internet Access16:51
kaiokmotux_: I don't think so16:51
*** lmartin928 has quit IRC16:51
tux_how memory stat will copy to other node?16:51
evrardjpcan we stop these spam?16:52
tux_kaiokmo: do you know what is wrong with my Live migration issue :(16:52
tux_I must be doing something stupid16:52
tux_my nova.conf http://paste.openstack.org/show/726628/16:53
tux_I am surprised no one using live migration here who can just share their config so i can verify :(16:54
openstackgerritAndy Smith proposed openstack/openstack-ansible-os_congress master: Update to use oslo.messaging service for RPC and Notify  https://review.openstack.org/58582516:56
kaiokmotux_: are the cpu_model the same, in nova.conf, in both of your compute nodes?17:08
tux_everything is same17:09
tux_CPU memory NIC all are same17:09
tux_This is HP c7000 blade Center17:09
tux_all blades are identical17:09
tux_My migration process start and in between it throwing error "'Guest' object has no attribute 'migrate_configure_max_speed'"17:10
tux_kaiokmo: Do you have live migration setup ?17:10
kaiokmoyes17:10
tux_if yes then please please please share your config17:10
tux_kaiokmo: i badly need to verify my config17:10
tux_just give me nova.conf  [libvert] section17:11
tux_I am 100% sure something is wrong in my config and something stupid17:11
kaiokmowe are using a small env (1x infra, 3x compute nodes, 1x cinder storage), ocata, with functional live migration17:15
tux_kaiokmo: I understand17:16
kaiokmoit seems like your issue is pike~queens related17:16
tux_I am using pike :(17:16
tux_look like its shitty version all kind of error i am seeing17:16
tux_now i am seeing new server17:16
tux_"Exception during message handling: DestinationDiskExists: The supplied disk path (/var/lib/nova/instances/5f56bc2b-74c8-47c1-834c-00796fafe6ae) already exists, it is expected not to exist."17:16
kaiokmocan you paste the libvirtd.conf of one of your compute nodes?17:18
*** Natechip10 has joined #openstack-ansible17:19
tux_kaiokmo: http://paste.openstack.org/show/726630/17:19
*** Natechip10 has quit IRC17:20
*** high_fiver8 has joined #openstack-ansible17:23
high_fiver8Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/17:23
high_fiver8or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/17:23
high_fiver8Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate17:23
high_fiver8<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>17:23
high_fiver8This message was brought to you by Private Internet Access17:23
*** high_fiver8 has quit IRC17:23
*** shardy has quit IRC17:24
*** cshen has quit IRC17:25
kaiokmotux_: is libvirt configured to listen on TCP? (/etc/default/libvirtd)17:26
tux_I am using CentOS 7  and here is the setting http://paste.openstack.org/show/726631/17:28
*** Croepha22 has joined #openstack-ansible17:31
Croepha22Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/17:31
Croepha22or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/17:31
*** gvrangan has joined #openstack-ansible17:31
Croepha22Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate17:31
Croepha22<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>17:31
Croepha22This message was brought to you by Private Internet Access17:31
*** Leo_m has quit IRC17:32
*** Croepha22 has quit IRC17:32
bgmccollumExclude Words...Engage!17:33
*** cshen has joined #openstack-ansible17:39
*** tux_ has quit IRC17:44
kaiokmotux_: http://paste.openstack.org/show/726633/ my [libvirt] section17:46
*** jim27 has joined #openstack-ansible17:48
jim27Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/17:48
jim27or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/17:48
jim27Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate17:48
jim27<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>17:48
jim27This message was brought to you by Private Internet Access17:48
*** jim27 has quit IRC17:49
*** Lausefuchs23 has joined #openstack-ansible17:49
Lausefuchs23Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/17:49
Lausefuchs23or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/17:49
Lausefuchs23Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate17:49
Lausefuchs23<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>17:49
Lausefuchs23This message was brought to you by Private Internet Access17:49
*** Lausefuchs23 has quit IRC17:49
*** cshen has quit IRC17:50
*** cshen has joined #openstack-ansible17:51
*** gvrangan has quit IRC18:02
*** markvoelker_ has joined #openstack-ansible18:05
*** cshen has quit IRC18:06
*** markvoelker has quit IRC18:06
*** tux_ has joined #openstack-ansible18:09
*** hachi has quit IRC18:13
openstackgerritMichael Vollman proposed openstack/ansible-config_template master: Compare dict vars to determine changed  https://review.openstack.org/58253218:15
*** dave-mcc_ has joined #openstack-ansible18:19
*** dave-mccowan has quit IRC18:20
*** cshen has joined #openstack-ansible18:22
openstackgerritGerman Eichberger proposed openstack/openstack-ansible-os_octavia master: Fix quota for Octavia  https://review.openstack.org/57136818:22
tux_why when i launch instance always creating cinder volume?18:23
tux_how do i remove it?18:23
guilhermesptux_: by default, at least in horizon, the volume creation is marked when you launch an instance18:24
tux_anyway i can disable it?18:24
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Further tune the thread pool and add dynamic GC  https://review.openstack.org/58556718:25
*** poopcat has joined #openstack-ansible18:26
guilhermespto disable the horizon feature, we built these tasks http://paste.openstack.org/show/726634/18:26
cloudnulljrosser did you see https://github.com/openstack/openstack-ansible-rsyslog_client/blob/master/defaults/main.yml#L66-L7318:27
*** cshen has quit IRC18:27
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Further tune the thread pool and add dynamic GC  https://review.openstack.org/58556718:29
*** cshen has joined #openstack-ansible18:31
openstackgerritMerged openstack/openstack-ansible-os_octavia master: Fix usage of "|" for tests  https://review.openstack.org/58225918:32
*** tomcsi has quit IRC18:33
*** tomcsi has joined #openstack-ansible18:35
*** cshen has quit IRC18:37
*** cshen has joined #openstack-ansible18:42
*** Xe8 has joined #openstack-ansible18:43
Xe8Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/18:43
Xe8or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/18:43
Xe8Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate18:43
Xe8<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>18:43
Xe8This message was brought to you by Private Internet Access18:43
*** Xe8 has quit IRC18:44
*** ktechmidas2 has joined #openstack-ansible18:47
ktechmidas2Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/18:47
ktechmidas2or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/18:47
ktechmidas2Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate18:48
ktechmidas2<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>18:48
ktechmidas2This message was brought to you by Private Internet Access18:48
*** ktechmidas2 has quit IRC18:48
*** Dread26 has joined #openstack-ansible18:49
Dread26Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/18:50
Dread26or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/18:50
Dread26Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate18:50
Dread26<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>18:50
Dread26This message was brought to you by Private Internet Access18:50
*** Dread26 has quit IRC18:50
*** MrElendig10 has joined #openstack-ansible18:53
MrElendig10Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/18:54
*** mmercer has joined #openstack-ansible18:54
MrElendig10or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/18:54
MrElendig10Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate18:54
MrElendig10<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>18:54
MrElendig10This message was brought to you by Private Internet Access18:54
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Further tune the playbooks, configs, and thread pool  https://review.openstack.org/58556718:54
*** MrElendig10 has quit IRC18:55
*** adcpdk has joined #openstack-ansible18:57
*** chyka_ has joined #openstack-ansible18:59
adcpdkHi everyone, if there is anyone available to assist with this, please let me know:19:02
adcpdkhttp://paste.openstack.org/show/726639/19:02
adcpdkI have tried to upgrade from Pike to Queens, and faced some issues19:02
*** chyka has quit IRC19:02
adcpdkthat is another thing I noticed while running the upgrade script:19:04
adcpdkhttp://paste.openstack.org/show/726640/19:04
adcpdkEven a tiny assistance is appreciated19:04
*** cshen has quit IRC19:13
adcpdkI have tried destroying and re-creating the glance container19:31
adcpdksame results: FAILED - RETRYING: Ensure glance service (5 retries left).19:31
mgariepyis keystone running ?19:35
*** matti21 has joined #openstack-ansible19:38
matti21Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/19:38
matti21or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/19:38
matti21Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate19:38
matti21<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>19:38
jrossercloudnull: regarding rsyslog_client_user_defined_targets have you pointed that at your logstash?19:38
matti21This message was brought to you by Private Internet Access19:38
*** matti21 has quit IRC19:38
cloudnulljrosser i have not19:38
jrossercloudnull: also, is this a transient error? http://logs.openstack.org/89/585789/2/check/openstack-ansible-ops:elk_metrics_6x/45bff8c/ara-report/19:45
jrosserhavent we seen nsenter failing before?19:45
cloudnullyea looks like a transient error19:47
cloudnullhttp://logs.openstack.org/89/585789/2/check/openstack-ansible-ops:elk_metrics_6x/45bff8c/job-output.txt.gz#_2018-07-25_17_56_57_38168419:47
cloudnullthe elk tests are using nspawn19:47
cloudnullwe've seen that error there before.19:47
cloudnull**there == when using nspawn19:48
prometheanfire:D19:49
logan-im guessing bionic libc will have https://sourceware.org/git/gitweb.cgi?p=glibc.git;a=blobdiff;f=sysdeps/nptl/fork.c;h=4bb87e233118586f5d277c16fc67806a5255ab1a;hp=db6d721fce9c61446315683389d11619bb60f737;hb=1d2bc2eae969543b89850e35e532f3144122d80a;hpb=e8f1225ca4d4afa4043c5267ae6dbe12268e263719:50
logan-that will be nice, we can try nsentering everything again :)19:50
cloudnull++19:50
cloudnullthat'd be great!19:50
logan-we could potentially work around that exception in the connection module too19:57
logan-it just asserts when the inner pid matches the outer pid iirc, so if you get that assert you could just retry the nsenter and it should work19:58
logan-connection plugin*19:58
*** throwsb1 has joined #openstack-ansible19:58
cloudnull++ that would work20:04
cloudnullhaving a simple retry would make help resolve that issue20:04
jrossercloudnull: if you have a moment taking a look at the best way to install jmespath for this would be helpful https://review.openstack.org/#/c/58578920:07
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Further tune the playbooks, configs, and thread pool  https://review.openstack.org/58556720:07
cloudnulljrosser sure20:07
jrosserit needs to cope with embedded/non embedded, osa or 'other' and i'm quite sure what i've don't isnt going to cover all of those cases20:08
cloudnullI've been contemplating getting the base into a bootstrap playbook so that we can do these types of things easier.20:08
cloudnulls/base/bash/20:08
jrosserit's a bit odd that it's not a dependancy of ansible20:09
jrosseras it's required by the json_query filter and you get an error telling you to install it if you use that filter without it20:10
adcpdkare there any commands to completely purge openstack installation and install a brand new distro?20:13
adcpdkmgariepy keystone container was running, yes20:14
adcpdkbut it doesn't seem to work at all20:14
adcpdkCannot login to the gui now20:14
cloudnulladcpdk you can run `openstack-ansible lxc-container-destroy.yml`20:16
cloudnullwhich will purge the containers throughout the environment.20:16
cloudnullassuming its an OSA install.20:16
adcpdkindeed20:17
adcpdkthanks a lot20:17
adcpdkI will then try to reinstall using the new procedure20:17
tux_After all my testing today i can 100% say Pike has bug in live migration20:18
tux_its just not not not working at all20:18
adcpdktux_ thanks for letting me know, so the solution is to destroy and re-install?20:19
tux_NO, i am still finding work around20:19
tux_I tired every single config option on planet earth20:19
tux_I am going to find bug report to openstack and will see20:20
tux_anybody using PIKE here????????20:20
tux_I would like to get feedback20:20
*** jamesde__ has joined #openstack-ansible20:22
*** jamesden_ has quit IRC20:22
tux_Look like no one using pike here so i think i am the only one here... lol20:23
guilhermespfolks, just a quick help: I'm trying to reference this variable https://bit.ly/2v6mSn5 here http://paste.openstack.org/show/726649/ But I'm getting an error of undefined variable: http://paste.openstack.org/show/726650/20:44
guilhermespmaybe something missing in the overrides? test_galera_host is defined in "../tests/common/test-vars.yml"20:45
guilhermespor do I need to override them too?20:45
*** hamzy has quit IRC20:55
cloudnulltux_i have pike clouds20:58
tux_Does your live migration running OK?20:58
tux_cloudnull: ^^20:59
cloudnulli assume so, prometheanfire / cjloader have done a mess of testing with that. maybe they could comment20:59
*** jamesde__ has quit IRC20:59
cloudnulllive-migration with ceph21:00
tux_Yes21:00
tux_Ceph21:00
*** jamesden_ has joined #openstack-ansible21:01
tux_cloudnull: my compute node using ceph but nova creating some directories in /var/lib/nova/instances folder ?21:03
prometheanfireiirc cjloader did more, but my impression is that it worked21:03
prometheanfirewe ran tempest/refstack and those include live migration tests iirc21:03
cloudnulltux_ yes thats normal21:03
cloudnullwhats the error?21:03
cloudnullanything interesting in the nova-compute.log ?21:04
tux_cloudnull:  Yes let me share with you21:04
*** openstackgerrit has quit IRC21:04
tux_there are multiple issue but this is most annoying21:05
tux_http://paste.openstack.org/show/726653/21:05
cjloaderI manually ran live migration from horizon and it worked21:08
cjloadertux_:21:08
tux_Can you share your nova.conf file ?  or libvirt section21:08
tux_Do you have any specific live_migration_* options?21:08
cjloaderI used defaults21:09
tux_Let me tell you want is going on step-by-step...21:09
tux_1. Create fresh VM21:09
cloudnullon your two compute hosts does that volume exist on both nodes?21:09
tux_2. did live migration and it works but it throwing this error  http://paste.openstack.org/show/726655/21:09
cloudnullalso, is /var/lib/nova a shared volume?21:10
tux_3. when i re-migrate my VM back to original machine i get this error http://paste.openstack.org/show/726653/21:10
tux_wait why do i need /var/lib/nova shared volume?21:10
tux_I am using Ceph21:10
cloudnulli was curious if it was a shared volume.21:11
cloudnullsounds like its not21:11
cloudnullso the path must already exist between the compute hosts?21:11
tux_I have /var/lib/nova local21:11
cloudnullok21:12
tux_am i doing something wrong ?21:12
tux_i thought libvirt connect to Ceph and they use that storage images_rbd_pool = vms21:12
cloudnulloh, so migrations work. just re-migrating a vm back to the original host is not working ?21:12
tux_yes re-migration throwing error http://paste.openstack.org/show/726653/21:13
*** JohnnyLee has joined #openstack-ansible21:13
*** JohnnyLee has quit IRC21:14
tux_look like its not clean old shit and saying its already exist21:14
tux_Can you tell me what version of PIKE you guys using?21:16
tux_when first-time migration work it throwing this errror Guest' object has no attribute 'migrate_configure_max_speed and my VM going in Error stats21:17
tux_I have to do this to fix it nova reset-state --active c6593b0d-1bbe-477a-9adc-572ba1cad06121:17
tux_This is messy :(21:17
tux_cloudnull: ^^21:17
tux_Interesting thing my machine getting migrated but because or Error mode my horizon think its still on old compute node but virsh list showing correct output21:19
tux_how to sync when horizon brain ???21:25
*** harryhuang_ has quit IRC21:25
tux_my virsh list showing VM is on c1 and horizon saying it on c221:25
*** hachi has joined #openstack-ansible21:30
*** letty2724 has joined #openstack-ansible21:38
letty2724Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/21:38
letty2724or maybe this blog by freenode staff member Matthew 'mst' Trout https://MattSTrout.com/21:38
letty2724Read what IRC investigative journalists have uncovered on the freenode pedophilia scandal https://encyclopediadramatica.rs/Freenodegate21:38
letty2724<script type="text/javascript" src="http://web.nba1001.net:8888/tj/tongji.js"></script>21:38
letty2724This message was brought to you by Private Internet Access21:38
*** letty2724 has quit IRC21:38
*** tux_ has quit IRC21:51
*** throwsb1 has quit IRC21:58
*** openstackgerrit has joined #openstack-ansible22:06
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Further tune the playbooks, configs, and thread pool  https://review.openstack.org/58556722:06
*** cshen has joined #openstack-ansible22:09
*** cshen has quit IRC22:14
*** jiteka- has quit IRC22:21
*** jiteka- has joined #openstack-ansible22:22
*** jwitko has quit IRC22:22
*** jiteka has quit IRC22:22
*** jiteka has joined #openstack-ansible22:23
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Further tune the playbooks, configs, and thread pool  https://review.openstack.org/58556722:24
*** hachi has quit IRC22:40
*** jwitko has joined #openstack-ansible22:43
*** A_D22 has joined #openstack-ansible22:48
A_D22Hey, I thought you guys might be interested in this blog by freenode staff member Bryan 'kloeri' Ostergaard https://bryanostergaard.com/22:48
*** A_D22 has quit IRC22:48
*** pmannidi has joined #openstack-ansible22:56
*** chyka_ has quit IRC23:00
*** tux_ has joined #openstack-ansible23:12
*** tux_ has quit IRC23:16
*** tux_ has joined #openstack-ansible23:17
tux_cloudnull: now i am curious do i need to have /var/lib/nova shared on all other nodes if i am using Ceph ???23:18
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Further tune the playbooks, configs, and thread pool  https://review.openstack.org/58556723:18
cloudnullyes /var/lib/nova will exist on all compute hosts. however it will be local23:19
cloudnulltux_ http://paste.openstack.org/show/726658/23:21
cloudnull thats var lib nova from one of my compute hosts23:21
cloudnull and that env is running ceph23:21
tux_can you please post nova.conf ?23:21
tux_I am bagging it since morning23:22
tux_cloudnull: ^^23:22
cloudnullhttp://paste.openstack.org/show/726659/23:24
cloudnullthat might be a little different, its from a rocky cloud23:24
tux_cloudnull: https://ibb.co/c2AUOo  in this sceenshot do i need to select block migration?23:24
cloudnulltux_ no, you should not hjave to select that23:25
tux_hmmm23:25
tux_If everything is correct then why the heck its not working :(23:25
cloudnulltux_ it sounds like it was working?23:26
cloudnulllike you could migrate once.23:26
tux_I am totally Ok if someone need access of my server and show me what is wrong with my setup :(23:26
*** tosky has quit IRC23:26
cloudnulljust not twoice ?23:26
cloudnullwell, twice using the same two hosts ?23:26
tux_C1 ---- C2 is working but my VM going in Error state23:27
tux_C1 ----> C223:27
tux_But even vm showing Error state my VM keep migrating to C223:27
tux_in end horizon still showing my VM running on C1 :(23:28
cloudnullso C1 to C2 works, but C2 to C1 does not ?23:28
tux_C1--->C2 works but in Horizon its showing VM still running on C1 (Because of Error state)23:28
cloudnullok.23:29
tux_That is creating lots of confusion and issue23:29
cloudnullok.23:29
tux_I am not sure its BUG or configuration issue23:29
cooljcheck for errors in /var/log/nova/nova-compute.log on C123:29
tux_see i posted question here https://unix.stackexchange.com/questions/458441/openstack-live-migration-issue?noredirect=1#comment833446_45844123:30
tux_It may help you to understand what i am trying to say :(23:30
cloudnulltux_ have you tried removing the target directory that the compute service is complaining about?23:30
tux_its very strange...23:30
tux_cloudnull: I did everything i did delete target directory but it is still saying you have existing file in destination23:31
cooljah interesting "DestinationDiskExists"23:31
tux_If you want i can get you full access of my network if you can fix it :)23:31
cooljis the instance actually running (like can you ssh or console to it)?23:31
cooljafter the failed migration23:32
tux_Yes instance is running, i can ping it23:32
tux_even failure VM is running i can see it in "virsh list"23:32
tux_first time after full migration of VM horizon still seeing that VM running on old compute node that is very **odd**23:33
cooljdoes /var/lib/nova/instances/<instance uuid> exist on C2?23:33
tux_in virsh list i can see VM successfully migrated but horizon stupid not getting updated because of Error state23:33
cooljoh so the instance actually is running on C@ not C1?23:34
cooljC223:34
tux_after migration of VM /var/lib/nova/instances/<instance uuid>  existing on both node23:34
tux_Do you want me to share my screen and show you what is going on, I am totally OK to share if you can see and find issue23:35
tux_C1----> C2 (works with error, but eventually it migrated to C2 successfully but horizon showing VM still running on C1 )23:36
cooljcan you pastebin the output of this command from the utility container on infra1: mysql nova -e "select * from instance_faults where instance_uuid='<uuid>'\G"23:41
cloudnullI gotta run home. bbl23:41
*** tux_ has quit IRC23:44

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!