Monday, 2018-09-17

*** jonher has quit IRC00:05
*** openstackgerrit has quit IRC00:09
*** jonher has joined #openstack-ansible00:25
*** openstackgerrit has joined #openstack-ansible00:29
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Converg the logstash pipelines  https://review.openstack.org/60250100:29
*** jonher_ has joined #openstack-ansible00:43
*** jonher has quit IRC00:45
*** jonher_ is now known as jonher00:45
*** sawblade6 has joined #openstack-ansible00:59
*** sawblade6 has quit IRC01:04
*** sawblade6 has joined #openstack-ansible01:19
*** sawblade6 has quit IRC01:23
*** sawblade6 has joined #openstack-ansible01:31
*** sawblade6 has quit IRC01:35
*** sawblade6 has joined #openstack-ansible01:42
*** mschuppert has quit IRC01:44
*** sawblade6 has quit IRC01:50
*** jmccrory has quit IRC01:54
*** jmccrory has joined #openstack-ansible01:57
*** jonher has quit IRC02:00
*** jonher has joined #openstack-ansible02:02
*** lbragstad has joined #openstack-ansible02:08
*** SamYaple has quit IRC02:14
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Converg the logstash pipelines  https://review.openstack.org/60250102:15
*** sawblade6 has joined #openstack-ansible02:22
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Converg the logstash pipelines and enhance memory backed queues  https://review.openstack.org/60250102:25
*** sawblade6 has quit IRC02:33
*** lbragstad has quit IRC02:36
*** dave-mccowan has quit IRC02:37
*** lbragstad has joined #openstack-ansible02:41
*** dxiri has joined #openstack-ansible02:43
*** sawblade6 has joined #openstack-ansible03:01
*** sawblade6 has quit IRC03:06
*** sawblade6 has joined #openstack-ansible03:07
*** sawblade6 has quit IRC03:11
*** Anirudh has quit IRC03:16
*** sawblade6 has joined #openstack-ansible03:28
*** udesale has joined #openstack-ansible03:34
*** sawblade6 has quit IRC03:41
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Converg the logstash pipelines and enhance memory backed queues  https://review.openstack.org/60250103:42
*** oanson has quit IRC03:44
openstackgerritDuc Nguyen proposed openstack/openstack-ansible-os_keystone master: Replace port 35357 with 5000  https://review.openstack.org/58425603:53
*** sawblade6 has joined #openstack-ansible03:56
*** sawblade6 has quit IRC04:02
*** udesale has quit IRC04:03
openstackgerritDuc Nguyen proposed openstack/openstack-ansible-os_keystone master: Replace port 35357 with 5000  https://review.openstack.org/58425604:05
openstackgerritMerged openstack/openstack-ansible-os_swift master: Trivial: Fix the pep8 warning  https://review.openstack.org/57678104:21
*** udesale has joined #openstack-ansible04:24
*** udesale has quit IRC04:25
*** udesale has joined #openstack-ansible04:25
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Converg the logstash pipelines and enhance memory backed queues  https://review.openstack.org/60250104:28
openstackgerritKevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Converg the logstash pipelines and enhance memory backed queues  https://review.openstack.org/60250104:45
openstackgerritjacky06 proposed openstack/openstack-ansible-os_horizon master: [WIP]: add cloudkitty dashboard  https://review.openstack.org/60301305:03
*** shyamb has joined #openstack-ansible05:08
openstackgerritjacky06 proposed openstack/openstack-ansible-os_horizon master: Add cloudkitty dashboard  https://review.openstack.org/60301305:11
openstackgerritjacky06 proposed openstack/openstack-ansible-os_horizon master: Add cloudkitty dashboard  https://review.openstack.org/60301305:14
openstackgerritjacky06 proposed openstack/openstack-ansible-os_horizon master: Add cloudkitty dashboard  https://review.openstack.org/60301305:16
openstackgerritjacky06 proposed openstack/openstack-ansible-os_horizon master: Add cloudkitty dashboard  https://review.openstack.org/60301305:19
*** jbadiapa has joined #openstack-ansible05:25
*** shyamb has quit IRC05:27
openstackgerritOlivier Bourdon proposed openstack/openstack-ansible stable/queens: [DNM] Testing interfaces after deployment  https://review.openstack.org/60297605:29
*** dxiri has quit IRC05:33
openstackgerritMerged openstack/openstack-ansible-ops master: Converg the logstash pipelines and enhance memory backed queues  https://review.openstack.org/60250105:41
*** sawblade6 has joined #openstack-ansible05:54
*** sawblade6 has quit IRC05:58
*** shyamb has joined #openstack-ansible06:00
openstackgerritjacky06 proposed openstack/openstack-ansible-os_congress master: Fix the wrong pome-page  https://review.openstack.org/60301906:00
openstackgerritjacky06 proposed openstack/openstack-ansible-os_tacker master: Fix the wrong home-page  https://review.openstack.org/60302006:01
*** dxiri has joined #openstack-ansible06:30
*** luksky has joined #openstack-ansible06:34
*** udesale has quit IRC06:38
*** udesale has joined #openstack-ansible06:40
*** udesale has quit IRC06:45
*** sawblade6 has joined #openstack-ansible06:50
*** sawblade6 has quit IRC06:55
*** olivierb has joined #openstack-ansible07:01
*** luksky has quit IRC07:10
*** shardy has joined #openstack-ansible07:15
*** cshen has joined #openstack-ansible07:17
evrardjpgood morning07:29
evrardjpjrosser: or hwoarang could you have a look at https://review.openstack.org/#/c/602669/ ?07:30
*** priteau has joined #openstack-ansible07:30
*** dxiri has quit IRC07:33
*** sawblade6 has joined #openstack-ansible07:34
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_neutron stable/rocky: Update branch to point to Rocky  https://review.openstack.org/59980107:34
openstackgerritOlivier Bourdon proposed openstack/openstack-ansible stable/queens: [DNM] Testing interfaces after deployment  https://review.openstack.org/60297607:36
*** sawblade6 has quit IRC07:38
jrosserevrardjp: just looked at that patch - is qdrouterd-overrides.yml included where you expect it to be?07:39
jrosserJust on my07:40
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_swift stable/rocky: Update branch to point to Rocky  https://review.openstack.org/59980807:40
jrosserMobile so tricky to look beyond the diff just now07:40
evrardjpodyssey4me: can you contact me when https://review.openstack.org/#/c/602669/ is merged?07:44
evrardjpsoI can do the backport07:44
evrardjpjrosser: it is fine for me07:44
evrardjpjrosser: but let me double check :)07:44
evrardjpjrosser: I checked, it's good: https://github.com/openstack/ansible-role-qdrouterd/blob/master/tox.ini#L2607:47
*** luksky has joined #openstack-ansible07:49
jrosserevrardjp: is there some tox magic that automatically includes qdrouterd-overrides.yml? the only reference to it looks to be in the tox.ini [testenv:upgrade] which the patch removes07:52
evrardjpjrosser: yeah, the default fonctional test is loading ${ROLE_NAME}-overrides07:55
evrardjpthat's why I pinged you thelink with ROLE_NAME properly defined07:55
evrardjpjrosser: as you are with a mobile you can read the blog post I finished to write during the PTG to give me your opinion: https://evrard.me/custom-inventories-with-openstack-ansible : )07:56
evrardjpofc it's more a hint of what can be done than a real scenario -- but it should help people07:56
jrosserah i found it https://github.com/openstack/openstack-ansible-tests/blob/master/test-ansible-functional.sh#L3707:58
jrossermakes sense07:58
*** dxiri has joined #openstack-ansible08:00
*** shyamb has quit IRC08:02
hwoaranggood morning08:03
evrardjpgood morning hwoarang08:03
*** hamzaachi has joined #openstack-ansible08:06
*** gameon has joined #openstack-ansible08:10
*** shyamb has joined #openstack-ansible08:14
*** rpittau has joined #openstack-ansible08:23
*** udesale has joined #openstack-ansible08:25
*** DanyC has joined #openstack-ansible08:26
hwoarangdoes anyone understand the '"'_galera_client_repo' is undefined"' error in suse jobs08:26
hwoarangsomething must be wrong in the galera-client role but i can't find out what :/08:26
hwoarangsome variable precedence/interpolation thing08:26
*** udesale has quit IRC08:27
*** udesale has joined #openstack-ansible08:28
jrosserhwoarang: it's possibly needing this to go in https://review.openstack.org/#/c/602927/08:28
*** udesale has quit IRC08:29
*** udesale has joined #openstack-ansible08:29
jrossersee the commit message there referencing meta role inclusions, which is just how that galera_client stuff goes wrong08:29
*** udesale has quit IRC08:29
hwoaranghmm yeah i notice that the gather facts in galera-client actually include those from keystone08:29
*** udesale has joined #openstack-ansible08:30
hwoarangthis is very tricky now08:31
hwoarangthank you jrosser for taking care of it08:31
*** dxiri has quit IRC08:33
*** sawblade6 has joined #openstack-ansible08:37
*** jonher_ has joined #openstack-ansible08:40
*** tosky has joined #openstack-ansible08:41
*** jonher has quit IRC08:42
*** jonher_ is now known as jonher08:42
*** sawblade6 has quit IRC08:44
eumel8hello osa cores, please can you merge https://review.openstack.org/#/c/602953/ ? This would solve (hopefully) translation job for project documentation08:45
*** udesale has quit IRC08:47
odyssey4memorning folks08:50
odyssey4mehwoarang: yeah, the issue is that galera_client switched to using suse.yml, then all the other roles when doing include_vars for galera_client found their own role's suse-42.yml file instead of the galera_client's suse.yml file first (due to the way we do the include_vars)... so I had to switch up all the other roles to use suse.yml08:52
openstackgerritjacky06 proposed openstack/openstack-ansible-os_congress master: Fix the wrong home-page  https://review.openstack.org/60301908:52
odyssey4methe nova role is failing because there's another package already installed which conflicts08:52
odyssey4meevrardjp: yeah, I'll handle those role test cleanup backports some time today08:53
*** udesale has joined #openstack-ansible08:55
*** shyamb has quit IRC08:56
hwoarangodyssey4me: there is no way to make this first_find modules only look in the role itself?08:58
openstackgerritMerged openstack/openstack-ansible-os_gnocchi stable/rocky: Upgrade pip/setuptools/wheel in role test venv  https://review.openstack.org/60268408:58
*** sawblade6 has joined #openstack-ansible08:59
*** dxiri has joined #openstack-ansible09:00
*** udesale has quit IRC09:01
*** electrofelix has joined #openstack-ansible09:01
*** sawblade6 has quit IRC09:03
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-galera_client master: RedHat: Remove packages with conflict with MariaDB  https://review.openstack.org/60305609:03
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_nova master: Rename SUSE vars file to cater for any version  https://review.openstack.org/60292709:04
*** hamzaachi has quit IRC09:04
*** mschuppert has joined #openstack-ansible09:04
*** gokhani has joined #openstack-ansible09:09
openstackgerritMerged openstack/openstack-ansible-os_cinder master: Clean up role tests  https://review.openstack.org/60076809:11
*** dxiri has quit IRC09:18
openstackgerritMerged openstack/openstack-ansible-os_keystone master: Replace port 35357 with 5000  https://review.openstack.org/58425609:21
openstackgerritcaoyuan proposed openstack/openstack-ansible-os_cinder master: use include_tasks instead of include  https://review.openstack.org/58530709:23
*** priteau has quit IRC09:23
*** priteau has joined #openstack-ansible09:24
*** priteau has quit IRC09:30
*** shyamb has joined #openstack-ansible09:31
*** udesale has joined #openstack-ansible09:38
*** hamzaachi has joined #openstack-ansible09:42
*** priteau has joined #openstack-ansible09:44
openstackgerritTaseer Ahmed proposed openstack/openstack-ansible master: Integrate Blazar with OpenStack Ansible  https://review.openstack.org/54995609:59
*** oanson has joined #openstack-ansible10:05
*** hamzaachi has quit IRC10:06
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-galera_client master: RedHat: Remove packages with conflict with MariaDB  https://review.openstack.org/60305610:08
*** noonedeadpunk has joined #openstack-ansible10:08
openstackgerritManuel Buil proposed openstack/openstack-ansible-os_neutron master: Ceilometer is a requirement of networking-odl  https://review.openstack.org/60306810:09
*** arxcruz|ptg is now known as arxcruz10:11
openstackgerritOlivier Bourdon proposed openstack/openstack-ansible stable/queens: [DNM] Testing interfaces after deployment  https://review.openstack.org/60297610:12
openstackgerritcaoyuan proposed openstack/openstack-ansible-os_cinder master: use include_tasks instead of include  https://review.openstack.org/58530710:13
*** shyamb has quit IRC10:20
*** shyamb has joined #openstack-ansible10:23
*** sawblade6 has joined #openstack-ansible10:23
*** sawblade6 has quit IRC10:28
gokhanihi team , I have installed openstack pike with openstack-ansible. My environment is HA. I have a problem when associating floating ip to instance and adding gateway to router. ı am getting 'Stderr: bind: Cannot assign requested address' error.  Error logs are in http://paste.openstack.org/show/730102/.  My ml2.conf.ini is http://paste.openstack.org/show/730100/ and linuxbridgeagent.ini is http://paste.openstack.org/show/730101/.  I tried adding10:29
gokhaninet.ipv4.ip_nonlocal_bind=1  manually on router namespaces. it is worked and didn't give any errors. I am not sure this is the right thing to do. can anyone help me about this ? I wrote to also neutron team but haven't got any reply yet so I write there.10:29
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-galera_client master: RedHat: Remove packages with conflict with MariaDB  https://review.openstack.org/60305610:36
openstackgerritMerged openstack/openstack-ansible-nspawn_container_create master: Switch to using project-templates  https://review.openstack.org/57748810:39
*** rgogunskiy has joined #openstack-ansible10:42
*** sawblade6 has joined #openstack-ansible10:51
odyssey4mehwoarang: are you intending to port the suse-15 work back to rocky, or only Stein?10:51
hwoarangnoooo way10:51
hwoarangonly stein10:51
odyssey4meok good, then we don't need to port all those vars file name changes back10:51
hwoarangyes10:52
odyssey4mehowever, I am keen to port the removal of the galera_client role usage back as far as possible givent hat all our current branches don't need the client everywhere10:52
odyssey4meit should help reduce run time10:52
hwoarangsure10:52
hwoarangdo you also want to get https://review.openstack.org/#/c/601612/ in to we can resume the leap 15 jobs? it's easier for me than running a gazillion of vagrants10:53
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_magnum master: Remove mysql-python  https://review.openstack.org/60287610:53
odyssey4mealso, for anyone doing rechecks on the integrated build for master - until https://review.openstack.org/602927 merges, nothing will work10:55
openstackgerritMerged openstack/openstack-ansible-os_tacker stable/queens: Remove mysql-python  https://review.openstack.org/60295610:57
openstackgerritMerged openstack/openstack-ansible-os_tacker stable/rocky: Remove mysql-python  https://review.openstack.org/60295510:57
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_panko stable/rocky: Remove pip_install meta dependency  https://review.openstack.org/60307610:58
*** sawblade6 has quit IRC10:59
*** shyamb has quit IRC10:59
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_horizon stable/rocky: Remove mysql-python  https://review.openstack.org/60307711:02
*** shyamb has joined #openstack-ansible11:05
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-haproxy_server master: Disable HAProxy apparmor profile if present  https://review.openstack.org/60307811:06
odyssey4mebbiab, going afk for a while11:07
openstackgerritArx Cruz proposed openstack/openstack-ansible-os_tempest master: WIP - Enable stackviz support  https://review.openstack.org/60310011:28
evrardjpprometheanfire: are you there?11:35
openstackgerritMerged openstack/openstack-ansible-tests master: Revert "Drop opensuse-150 jobs"  https://review.openstack.org/60161211:42
openstackgerritMerged openstack/openstack-ansible-galera_client master: RedHat: Remove packages with conflict with MariaDB  https://review.openstack.org/60305611:49
jrosserreviews of this appreciated https://review.openstack.org/#/c/601534/11:51
evrardjpprometheanfire: I'd like to chat about networking-bgpvpn requirements, to compare opinions :p11:52
odyssey4mehwoarang: weird, we're still getting conflicts in the os_nova job even with the package removal: http://logs.openstack.org/27/602927/4/check/openstack-ansible-functional-distro_install-centos-7/c3eadab/logs/ara-report/result/29406d90-d772-4615-97c5-c54836dc7eb5/11:52
mhaydeni didn't break it (this time)11:52
evrardjpmhayden: haha11:52
odyssey4me:)11:52
*** ansmith has quit IRC11:53
*** dave-mccowan has joined #openstack-ansible11:54
jrosserodyssey4me: does this mean it tried to remove the wrong package, or it uninstalled it...... a bit unclear http://logs.openstack.org/27/602927/4/check/openstack-ansible-functional-distro_install-centos-7/c3eadab/logs/ara-report/result/5e9f1ec8-b167-49a1-92c3-46aa10475bce/11:55
odyssey4mejrosser: it looks like removing it with that name doesn't work :/11:55
odyssey4mewe got a little too excited with https://review.openstack.org/603056 I think11:56
jrosseryup11:56
*** fghaas has joined #openstack-ansible11:58
evrardjpI completely missed the bump to 10.211:59
evrardjpgood change11:59
evrardjphwoarang: I finally got an answer from mariadb ppl -- they contacted me to say it's not a priority but they will try to make a leap 15 package.12:00
evrardjpnot sure what that actually means.12:00
hwoarangevrardjp: they are is already a leap 15 package12:00
hwoarangfor >= 10.212:00
evrardjpI meant for 10.112:00
hwoarangah ok12:00
evrardjp10.2 I know you know :p12:01
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_trove stable/queens: Update trove tempest plugin path  https://review.openstack.org/60310912:01
hwoarangwell that's why i pushed for 10.2 :/12:01
evrardjphwoarang: I guessed :)12:01
hwoarangodyssey4me: :( i dont understand what yum wants me to do12:01
evrardjphwoarang: it's better to push for 10.2 as 10.1 is eol (or soon to be) anyway, right?12:01
*** sawblade6 has joined #openstack-ansible12:01
hwoarangi hope so12:01
evrardjpwe are in a new branch so I think we can afford it12:02
odyssey4mehwoarang: I guess we're going to have to fire up a centos node and try to figure out the right way toremove that package and any others12:02
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_trove stable/queens: Remove mysql-python  https://review.openstack.org/60295812:03
evrardjpodyssey4me: I guess that now that we have maintainers per distro, we can contact them? :D12:04
odyssey4meyeah, we could - let's wait and see when mnaser or guilhermesp come online, as they may know without needing to fire up a node12:04
*** sawblade6 has quit IRC12:05
hwoarangall these conflicts ring a bell for sure12:05
*** shyamb has quit IRC12:06
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-haproxy_server master: Disable HAProxy apparmor profile if present  https://review.openstack.org/60307812:08
*** olivierb has quit IRC12:09
*** olivierb has joined #openstack-ansible12:10
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_trove stable/pike: Use the correct branch for tempest tests  https://review.openstack.org/60311112:10
odyssey4meanother obscure error from centos http://logs.openstack.org/57/602957/1/check/openstack-ansible-functional-centos-7/e7f3e38/job-output.txt.gz#_2018-09-16_10_25_51_852150 for https://review.openstack.org/#/c/602957/12:12
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_trove stable/queens: Pin the tempest test plugin  https://review.openstack.org/60311212:15
noonedeadpunkhi everyone) hope you had great time at ptg)12:16
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_magnum master: Remove galera_client meta-dependency  https://review.openstack.org/60290312:18
ArchiFleKsHi, I'm using neutron-lbaasv2 and after an upgrade from pike to queens I get a `could not load neutron_lbaas.drivers.haproxy.namespace_driver.HaproxyNSDriver` I know, that lbaasv2 is deprecated and we are planning to migrate to ocatvia but we have old user with haproxy lbaas, and now I can seem to create new ones12:18
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_keystone master: SUSE: Add support for openSUSE Leap 15  https://review.openstack.org/60311412:19
noonedeadpunkI was toying with tempest, and I had really strage feeling about it's role in OSA. For instance why do we need to add tempest project, users, create private/public networks? As far as I see, tempest creates everything that he needs during tests execution, and drops them appropriate after ending. Everything he really needs - images and UUID of public network...12:20
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_sahara master: Remove galera_client meta-dependency  https://review.openstack.org/60291212:20
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_magnum master: Remove galera_client meta-dependency  https://review.openstack.org/60290312:21
odyssey4meArchiFleKs: apparently there's a migration tool, and it's be nice to get that implemented as part of the role12:22
noonedeadpunkActually I've stuck with role on public network creation, as of course I may create public VLAN with small network in it for tempest, but it doesn't seemed required by tempest itself. And existing public network can't be used as it was created under another project (and role tries to create it one more time, instead saying ok, as network exists)12:22
odyssey4meArchiFleKs: I have no idea what the caveats are, or how well it works though.12:22
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_keystone master: SUSE: Add support for openSUSE Leap 15  https://review.openstack.org/60311412:22
odyssey4megoing offline for a bit12:23
guilhermespmorning!12:23
jrosserodyssey4me: http://logs.openstack.org/57/602957/1/check/openstack-ansible-functional-centos-7/e7f3e38/job-output.txt.gz#_2018-09-16_10_26_35_04645812:23
ArchiFleKsodyssey4me: I think there is some sort of proxying in place in queens to redirect request to Octaviav2 API, in our case we did the pike to queens upgrade and were planning to deploy octavia in queens12:24
jrosserother centos peeps ^^ looks like CA bundle is not sufficiently up to date12:24
guilhermespso centos is doing weird things..hum lemme see it12:24
jrosserguilhermesp: ^ that link there looks like one for you12:24
evrardjpmorning guilhermesp12:24
evrardjpArchiFleKs: I don't know the current state -- I expect everyone to deploy octavia :p12:25
guilhermespevrardjp: \m/   jrosser so that a good thing to start a monday12:25
evrardjpguilhermesp: hahaha12:25
*** sawblade6 has joined #openstack-ansible12:25
evrardjpArchiFleKs: please talk to johnsom for the little toolkit he wrote for conversion12:26
*** sawblade6 has quit IRC12:30
openstackgerritMerged openstack/openstack-ansible stable/queens: Bump plugins SHA to include recent bugfixes  https://review.openstack.org/60299612:31
openstackgerritOlivier Bourdon proposed openstack/openstack-ansible stable/queens: [DNM] Testing interfaces after deployment  https://review.openstack.org/60297612:33
ArchiFleKsevrardjp: i'll do but for now I'm stuck and I lost all the old lbaas with haproxy and the namespaces are gone12:37
*** strattao has joined #openstack-ansible12:41
ArchiFleKsBut this might have also something to do with neutron-agent container being migrated from neutron agent container to bare metal host in queens12:42
odyssey4meArchiFleKs: hmm, well - hopefully you can work out how to do better and patch it so that the next person doesn't have a bad day - sorry about that!12:43
openstackgerritArx Cruz proposed openstack/openstack-ansible-os_tempest master: WIP - Enable stackviz support  https://review.openstack.org/60310012:46
prometheanfireevrardjp: ?12:48
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone stable/rocky: Replace port 35357 with 5000  https://review.openstack.org/60311812:49
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone stable/rocky: Ensure that SSHD is installed, enabled and running  https://review.openstack.org/60311912:51
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone stable/rocky: Set default ssl protocols according to used webserver  https://review.openstack.org/60312012:52
odyssey4memnaser: I think we can release the -w patches in https://review.openstack.org/#/q/I67e0d8f4ab52449a80adb5c479faefbc83617025 now12:53
*** hamzaachi has joined #openstack-ansible12:53
*** udesale has quit IRC12:54
evrardjpodyssey4me: want ppl to +w then?12:55
mgariepyodyssey4me, Replace port 35357 with 5000 we should create the cleanup playbook (haproxy endpoint) tp run on upgrade12:55
evrardjpok done12:55
evrardjpI see12:55
guilhermespodyssey4me: jrosser I was talking to mnaser so we agreed that you guys can drop os_trove centos jobs to non voting for now. We don't use trove so that's not a priority to fix right now but we will asap12:56
guilhermespis there another centos issue that we can help out?12:56
openstackgerritMerged openstack/openstack-ansible stable/rocky: Bump plugins SHA to include recent bugfixes  https://review.openstack.org/60299712:58
guilhermespanyways, what is fun is that in my local tests, the failing task is ok13:00
guilhermesphttps://www.irccloud.com/pastebin/TnvFuPni/13:00
openstackgerritManuel Buil proposed openstack/openstack-ansible-os_ceilometer master: Add the extra parameter in the tooz dependency  https://review.openstack.org/60312213:05
*** ansmith has joined #openstack-ansible13:07
openstackgerritWayne Warren proposed openstack/openstack-ansible-ops master: Make root partition size configurable  https://review.openstack.org/60285913:12
openstackgerritArx Cruz proposed openstack/openstack-ansible-os_tempest master: WIP - Enable stackviz support  https://review.openstack.org/60310013:12
prometheanfireevrardjp: https://review.openstack.org/#/c/603064/13:19
evrardjpprometheanfire: didn't I review this already?13:24
evrardjpprometheanfire: oh my bad13:24
evrardjpprometheanfire: thanks for the link13:24
prometheanfire:D13:25
openstackgerritWayne Warren proposed openstack/openstack-ansible-ops master: Change VM definition in deploy-vm mnaio playbook  https://review.openstack.org/60283213:25
openstackgerritWayne Warren proposed openstack/openstack-ansible-ops master: Use group vars to reduce redundancy in host vars  https://review.openstack.org/60283413:25
openstackgerritWayne Warren proposed openstack/openstack-ansible-ops master: Enable setting mnaio disk size by pxe server group  https://review.openstack.org/60283513:25
openstackgerritWayne Warren proposed openstack/openstack-ansible-ops master: Make root partition size configurable  https://review.openstack.org/60285913:25
openstackgerritMerged openstack/openstack-ansible-os_keystone stable/ocata: Clean up role tests  https://review.openstack.org/60273613:29
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-haproxy_server master: Disable HAProxy apparmor profile if present  https://review.openstack.org/60307813:31
*** goldenfri has joined #openstack-ansible13:39
noonedeadpunkmy question about tempest was to understand, if I may try to somehow patch it to be more usable for existing openstack deployments, made by OSA13:41
jrossernoonedeadpunk: what are you wanting to achieve?13:43
openstackgerritMerged openstack/openstack-ansible-os_keystone stable/pike: Clean up role tests  https://review.openstack.org/60273513:44
noonedeadpunkjrosser: my problem is, that I failed to deploy tempest, because of requirment of extra public network. So role tries to create new public network, while tempest may use existing shared network. As role is authorized as another project, it just fails to use existing one (as it tries to create another one under its project).13:45
noonedeadpunkMoreover I dodn't really understand the necessity of creatin extra user and extra tenant, and private network as tempest creates everything it needs with it own.13:46
noonedeadpunkwith futher appropriate removal13:47
jrossernoonedeadpunk: is it that on a fresh deploy, like in a CI test, there is no external network or router set up, so the tempest role puts those things in place?13:51
noonedeadpunkjrosser: no, it's not on the fresh deploy. I thought to use tempest for testing prod on regular basis, if api's are running and to get notified about failures13:54
jrosserok, in our case we override the tempest vars to point to our actual external networks, and it works ok13:55
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_glance master: SUSE: Add support for openSUSE Leap 15  https://review.openstack.org/60313313:55
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_glance master: SUSE: Add support for openSUSE Leap 15  https://review.openstack.org/60313313:56
noonedeadpunkbut the role will still fail at https://github.com/openstack/openstack-ansible-os_tempest/blob/master/tasks/tempest_resources.yml#L170-L186 or do you mean just to set tempest_service_available_neutron to false?13:57
jrossernoonedeadpunk: i have this in /etc/openstack_deploy/user_variables_tempest.yml http://paste.openstack.org/show/730186/13:57
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_keystone master: SUSE: Add support for openSUSE Leap 15  https://review.openstack.org/60311413:58
noonedeadpunkjrosser: ok, but then tempest_public_net_name should be created within admin project on default domain? As it takes auth from tempest_service_setup_host14:01
openstackgerritMarkos Chandras (hwoarang) proposed openstack/openstack-ansible-os_tempest master: SUSE: Add support for openSUSE Leap 15  https://review.openstack.org/60313914:01
noonedeadpunkAnd if public net is not owned by admin project, task will fail, as this network on this vlan is already exist14:02
*** KeithMnemonic has joined #openstack-ansible14:03
jrossernoonedeadpunk: as this is a "real" deploy rather than CI we have a post-deploy playbook of our own that creates the public networks, and adds all our admin users etc14:03
noonedeadpunkjrosser: am I right, that os_tempest is designed mostly for CI?14:04
noonedeadpunkalso I don't feel a lot of sense in creation of private network by the role, as tempest creates it's own private networks during the tests14:05
openstackgerritManuel Buil proposed openstack/openstack-ansible-os_ceilometer master: [DNM] - Testing  https://review.openstack.org/60314314:09
jrossernoonedeadpunk: i'm not sure - only thing is i see a lot of those things templated into tempest.conf. If you think they are wrong please raise a bug :)14:13
openstackgerritMerged openstack/openstack-ansible-os_keystone stable/queens: Clean up role tests  https://review.openstack.org/60273314:16
noonedeadpunkjrosser: got it14:17
openstackgerritjacky06 proposed openstack/openstack-ansible-os_horizon master: Add blazar dashboard into horizon  https://review.openstack.org/60314714:19
odyssey4menoonedeadpunk jrosser it'd be nice to make the os_tempest role more usable outside of CI14:22
odyssey4meat the very least to enable/disable some of the resources created and the project to use for testing, instead of assuming the admin project14:22
odyssey4memgariepy: does haproxy not clean itself up?14:23
jrosserwe wrote a role for refstack in the end14:23
odyssey4meif not, then yes absolutely14:23
odyssey4mejrosser noonedeadpunk I think historically it was only used for CI, but I think both ourselves and the TripleO folks would like it to be more general purpose, and happen to be used by CI14:24
mgariepyodyssey4me, adding stuff to old files yes, removing old file not used anymore.. not really.14:24
noonedeadpunkodyssey4me: yeah, I just was thinking about that. For example, in case if public network uuid is specified, we don't need to create public network. Probably I'll try to work on it...14:24
odyssey4meguilhermesp: the current issue we're having is with a pre-installed package conflicting with the mariadb packages14:25
odyssey4meunfortunately it seems that https://review.openstack.org/603056 doesn't solve the issue, which you'll see showing up in the results for https://review.openstack.org/60292714:26
openstackgerritWayne Warren proposed openstack/openstack-ansible-ops master: Change VM definition in deploy-vm mnaio playbook  https://review.openstack.org/60283214:26
openstackgerritWayne Warren proposed openstack/openstack-ansible-ops master: Use group vars to reduce redundancy in host vars  https://review.openstack.org/60283414:26
openstackgerritWayne Warren proposed openstack/openstack-ansible-ops master: Enable setting mnaio disk size by pxe server group  https://review.openstack.org/60283514:26
openstackgerritWayne Warren proposed openstack/openstack-ansible-ops master: Make root partition size configurable  https://review.openstack.org/60285914:26
odyssey4mefor the trove thing, I don't think it's particular to trove - but thanks, perhaps making it non-voting for now is a suitable workaround14:26
jrosserfwiw i need something fairly standalone for post-deploy tempest-alike tests, due to the way the networks are segmented14:27
jrosserhaving something trivially consumable in a jenkins job that just needs to know the endpoint and creds is ++ for me14:27
*** dxiri has joined #openstack-ansible14:28
mgariepyhmm maybe we could create a config-regen to cleanup everything in haproxy.14:29
*** spatel has joined #openstack-ansible14:29
spateljamesdenton: ^^14:29
spatelI have really serious issue going on with openstack networking :(14:30
spatelI am sure you must have something to advice me.14:30
spatelI have already sent you email related that so let me know when you available14:30
spatelcloudnull: ^^ i have question related SR-IOV deployment with OSA I am reading here https://docs.openstack.org/openstack-ansible-os_neutron/latest/configure-network-services.html14:31
openstackgerritjacky06 proposed openstack/openstack-ansible-os_horizon master: Add zun dashboard into horizon  https://review.openstack.org/60315214:31
spatelIn this snippet http://paste.openstack.org/show/730191/14:31
jamesdentonHey spatel - I'm tied up for a bit, but I did see your email this morning. Are you seeing the issues on all instances or one instance?14:31
spatelwhat is sriov_host_interfaces: "p1p1,p4p1"14:31
spatelall instance...14:32
spatelI have remove all my production traffic from openstack until i fix this issue...14:32
jamesdentonIs this a new issue? Or had it been there since you finished the deployment?14:32
spatelI have notice my instance not able to handle high PPS rate and seeing packet drop on TAP interface (TX drop)14:32
spatellast week we put new application on cloud and found this issue..14:33
jamesdentonthis is centos?14:33
spatelCentOS 7.514:33
jamesdentonand a Pike? Queens? Can't say i've seen anything like what you've described in recent deployments14:34
spatelqueens14:34
spatelThis is what i tired "hw_vif_multiqueue_enabled=true"14:34
spatelethtool -L <NIC> combined 814:34
spateland it reduce packet drop some level14:35
spatelalso i have notice my instance tx_queue / rx_queue size is 256 ( I am looking to increase that size also but its not supported in queens)14:36
jamesdentonWe typically don't have to adjust far from the defaults. I don't have anything to compare to at the moment14:36
spatellook like a lots to do to get high speed network support.. we are using VoIP application which has very high UDP pps rate14:36
jamesdentonYeah - high PPS may be better suited for DPDK or SR-IOV14:37
spatelNow have only option to go with SR-IOV14:37
jamesdentonhas this app been on openstack before?14:37
spatelI have support of SR-IOV in my Qlogic card14:37
spatelWe are use same app in aws with SRIOV and its running fine..14:38
spatelour AWS bill is very high so thought we should run on openstack in house :) and end up with packet loss14:38
*** goldenfri has quit IRC14:38
spateljamesdenton: i need small help to understand how SR-IOV fit in OSA ?14:38
spatelhttp://paste.openstack.org/show/730191/  in this snippet14:39
jamesdentonso sriov_host_interfaces are the interfaces that the net_name will map to, IIRC. It ends up "physnet1:p4p1, physnet1:p1p1", for example14:39
spatelwhat is this line ? sriov_host_interfaces: "p1p1,p4p1"14:39
jamesdentonThose are the interface names of the interfaces you want to use for SR-IOV14:39
jamesdentonthe PFs14:39
spatelI have eno1+eno2 (bond0) and attached with br-vlan14:39
jamesdentonyou can't use bonding with SR-IOV14:39
spateloh!! crap :(14:40
spatelthat would be more fun14:40
spatelyou are saying i have to remove bonding ?14:40
jamesdentonyes - you would need to break the bond for any interfaces used for SR-IOV14:41
jamesdentonor add a NIC14:41
spatelhow do people use SR-IOV with high availability ?  what if my switch is dead :(14:41
jamesdentonThey would probably say... "It's the cloud!"14:43
spatelI have question what if i tell SR-IOV my nic is eno1/eno2  and keep other bridge interface in place..14:43
spateldo you think it won't work?14:43
jamesdentonseriously, though, it's one of the pitfalls. But you could possibly add a second interface inside the VM using a separate VF and bond inside the VM. Never done it, though. DPDK is more appealing in this way, since you can build a bond on the hypervisor14:44
spatelwhat do you suggest should i use dpdk or sriov?14:44
jamesdentonWell, OSA does not yet have complete DPDK implementation. And it will require OVS.14:44
spatelall i want is stopping packet drops14:45
spateli don't have OVS14:45
openstackgerritjacky06 proposed openstack/openstack-ansible-os_horizon master: Add blazar dashboard into horizon  https://review.openstack.org/60315614:45
spatelso final option is SR-IOV or tune my tx/rx queue14:45
spatelDo you guys every test PPS rate on your lab machine?14:46
spatelI am surprise you are saying you are not seeing any issue...14:46
jamesdentonI don't know how much you're gonna squeeze out of tuning.14:46
jamesdentonWell, I have seen issues with certain applications, such as those you describe. So it's not well suited to a stock cloud14:46
jamesdentonMoving towards SR-IOV or DPDK requires careful consideration and planning.14:47
spatelhmm!! we know SRIOV going to work because we used in AWS14:47
openstackgerritjacky06 proposed openstack/openstack-ansible-os_horizon master: Add zun dashboard into horizon  https://review.openstack.org/60315214:48
jamesdentonSure, it may work just fine. There are underlying hardware considerations/architectures that Amazon abstracts from the user, and that you, as architect/operator, would need to build in14:48
spatelwhat other complication you are seeing to implement SR-IOV?14:48
spatelDo you think NUMA will help in this case?14:49
openstackgerritManuel Buil proposed openstack/openstack-ansible-os_ceilometer master: [DNM] - Testing  https://review.openstack.org/60314314:50
spateljamesdenton: do you know how to make this work in Queens OSA https://specs.openstack.org/openstack/nova-specs/specs/rocky/implemented/libvirt-virtio-set-queue-sizes.html14:50
jamesdentonUsing SR-IOV may impact migrations. Not sure if that's still an issue. DPDK will require very careful consideration w/r/t NUMA, as you will need to create zones/aggregates to ensure you don't cross the NUMA boundary or else performance will be impacted14:51
openstackgerritLogan V proposed openstack/openstack-ansible-haproxy_server master: Add 'absent' service state  https://review.openstack.org/60315814:51
spatelIf i am able to increase tx/rx queue size then that will help too14:51
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone stable/ocata: Updated from OpenStack Ansible Tests  https://review.openstack.org/60231014:51
jamesdentonThat seems to be in Rocky, not sure if it was backported14:51
cloudnullmornings14:52
spatelAt this point i don't care about Live migration so i am not considering it..14:52
spatelcloudnull: can help use with that :)14:52
cloudnull?14:52
spatelHow do i implement this feature in queens https://specs.openstack.org/openstack/nova-specs/specs/rocky/implemented/libvirt-virtio-set-queue-sizes.html14:52
mgariepythanks logan-14:52
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone stable/ocata: import zuul job settings from project-config  https://review.openstack.org/59475214:52
openstackgerritLogan V proposed openstack/openstack-ansible-haproxy_server master: Add 'absent' service state  https://review.openstack.org/60315814:52
spatelEven its some kind of patching i am ok to do that14:53
_d34dh0r53_mornings :)14:53
*** _d34dh0r53_ is now known as d34dh0r5314:54
jamesdentonAre you running the QEMU/libvirt versions it mentions? You can try updating the XML files manually, if so, as it mentions14:54
spatelI am running 3.9.x14:54
spatelI tired virsh edit but its not working.. after machine reboot that option disappear14:54
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone stable/pike: Define cache_timeout variable  https://review.openstack.org/60296414:54
spatelThat was my next question how to edit xml file in qemu ?14:54
openstackgerritLogan V proposed openstack/openstack-ansible-haproxy_server master: Add 'absent' service state  https://review.openstack.org/60315814:55
logan-mgariepy: no prob :)14:55
jamesdentonyou may need to reboot with virsh to test, not nova/openstack client14:55
cloudnull spatel, assuming you have the required version of libvirt and qemu you can add the new options to the nova config using a config_template override.14:56
spatelvirsh shutdown/start <UUID>   is that what you saying?14:56
openstackgerritjacky06 proposed openstack/openstack-ansible-pip_install master: Add the project source code repository in README  https://review.openstack.org/58488514:56
spatelcloudnull: i did this http://paste.openstack.org/show/730192/14:57
-spatel- rx_queue_size=102414:57
-spatel- tx_queue_size=102414:57
jamesdentonor virsh reboot14:57
spatelbut didn't work :(14:57
spateljamesdenton: let me try now and see..14:57
spatelcloudnull: ^^14:57
cloudnullhttp://paste.openstack.org/show/730193/14:57
cloudnullthat would get the two options into config14:57
cloudnullhowever you have to have the required version of libvirt and qemu14:58
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone stable/queens: Define cache_timeout variable  https://review.openstack.org/60296314:58
cloudnullwhich might not be available in CentOS?14:58
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone stable/queens: import zuul job settings from project-config  https://review.openstack.org/59510914:58
-spatel- [root@ostack-compute-65 ~]# libvirtd -V14:58
-spatel- libvirtd (libvirt) 3.9.014:58
spatelCentOS 7.514:58
openstackgerritArx Cruz proposed openstack/openstack-ansible-os_tempest master: WIP - Enable stackviz support  https://review.openstack.org/60310014:58
spatelqemu-kvm-ev-2.10.0-21.el7_5.4.1.x86_6414:59
spatelqemu 2.10 and libvirtd 3.914:59
odyssey4mecloudnull: rechecking the integrated build right now for master is pointless until we can get os_nova's patch to rename the suse vars file merged, and that's hanging up due to a package conflict on centos14:59
spatelrequired is (In QEMU 2.7.0 and libvirt 2.3.0)14:59
spatelso i should have support of those option15:00
cloudnullI think you have to have QEMU 2.10.0 and libvirt 3.7.015:00
cloudnullto have the virtio tunable15:00
spatelI have that15:00
cloudnulloh cool!15:00
spatelsorry HAVE TO HAVE?15:00
spateli have 3.9.x so that one not going to work?15:00
devxhello all15:01
cloudnullso then adding those options to the config should be all that is required15:01
cloudnullhave you provisioned a new vm since adding those options ?15:01
spatelI did by hand on one of compute node http://paste.openstack.org/show/730192/15:01
spateland build new instance on it but i am not seeing that two option in dumpxml15:02
cloudnullhttps://docs.openstack.org/releasenotes/nova/rocky.html15:02
cloudnull ah looks like this is in rocky15:02
spatelyes..15:02
spateli am on queens15:02
cloudnullso you wont have that feature quite yet15:03
* cloudnull looking if that option is in queens 15:03
spateli heard you can hack15:03
spatelsomeone told me in mailing list you can add in qemu.conf and that should work15:03
spateli have tried and didn't work :(15:03
cloudnullthat I'm not sure of15:04
spatellet me try to add in running instance and see if that work with jamesdenton: talking about15:04
cloudnullodyssey4me oh15:05
cloudnullis there a set of patches that need reviews?15:05
*** goldenfri has joined #openstack-ansible15:06
spateljamesdenton: i am trying this..15:06
-spatel- [root@ostack-compute-65 ~]# virsh edit instance-0000017715:06
-spatel- Domain instance-00000177 XML configuration edited.15:06
-spatel- [root@ostack-compute-65 ~]# virsh reboot instance-0000017715:06
-spatel- Domain instance-00000177 is being rebooted15:06
*** rgogunskiy has quit IRC15:06
spateljamesdenton: didn't work http://paste.openstack.org/show/730195/15:07
spatelstill ring buffer size it 25615:08
*** luksky has quit IRC15:08
jamesdentontry a shutdown and startup there. with virsh15:08
spatelas soon as i reboot instance my option no longer there "<driver name='vhost' queues='8'/>"  it removed rx_queue15:09
jamesdentonok. sorry dude15:09
spatelI think virsh edit won't work... something is missing15:09
*** goldenfri has quit IRC15:10
spatellet me start working on SR-IOV, i think that only can save me :(15:11
jamesdentonthat's probably the best bet15:11
spatelIf SR-IOV won't work then i have no use of my cloud :(15:11
spatelwe have 90% application is VoIP base..15:11
jamesdentonthat sort of use-case is why things like SR-IOV and DPDK exist.15:12
spateljamesdenton: if you get time.. can you try runing hping3 tool to generate high pps and see if you getting TX drops on tap interface15:12
spateli would be surprise if not15:12
jamesdentonsend me the exact commands, and i will test when i have time15:13
spatelor may be its related of OS (centos vs ubuntu)15:13
spatelLet me give you all my result which i collected...15:13
spatelhold on15:13
openstackgerritManuel Buil proposed openstack/openstack-ansible-os_ceilometer master: Bug fix: ceilometer_service_setup_host missing  https://review.openstack.org/60316515:16
spateljamesdenton: http://paste.openstack.org/show/730197/15:17
spateltry that out and see what you noticed15:18
*** cshen has quit IRC15:18
jamesdentonthanks15:18
spatelI would be surprise if you say there won't be packet drops :)15:18
jamesdentonwhat are the specs of these nodes?15:18
spatelI have created 15 vpcu core machine15:18
spateli have tired 8 vCPU core also but same result..15:19
spatelmemory 8GB15:19
spatelmake sure both vm on same LAN otherwise it will hurt your routing device..15:19
*** hamzaachi has quit IRC15:19
spatelI am using VLAN base provider networking15:19
jamesdentondo you know if your scaling_governor is set to performance? vs ondemand?15:20
spatelwhat is that?15:20
spatelnever heard before15:20
jamesdentonhttps://www.kernel.org/doc/html/v4.14/admin-guide/pm/cpufreq.html15:20
jamesdentonmany times machines default to ondemand cpu scaling, for energy efficiency. ondemand has a large impact on performance on 10G NICs especially. You will want to confirm you're set to performance. It may not come into play in this particular circumstance15:21
*** goldenfri has joined #openstack-ansible15:21
spotzodyssey4me: Looking at the image/VM I made last week at the PTG and CLI is failing with 'Failed to discover available identity versions' Have you used the script on Queens before?15:21
spatelmy HP DL360 G8 machine is configure for Dynamic power mode..15:22
spatelis that what you saying to set performance mode ?15:22
jamesdentonspatel you can verify by `cat /sys/devices/system/cpu/cpu0/coufreq/scaling_governor`15:22
jamesdentonsorry, `cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor`15:22
-spatel- [root@ostack-compute-65 ~]# cat /sys/devices/system/cpu/cpu0/coufreq/scaling_governor15:22
-spatel- cat: /sys/devices/system/cpu/cpu0/coufreq/scaling_governor: No such file or directory15:22
-spatel- [root@ostack-compute-65 ~]# cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor15:23
-spatel- conservative15:23
spatelthis is what i found15:23
spateljamesdenton: ^^15:23
jamesdentonk15:23
spatelis that OK?15:23
jamesdentonnot ideal15:24
spateltell me what i should change.. i am ok to do anything at this point15:24
jamesdentonspatel you can try, on the two computes in question: `echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor`15:26
jamesdentonand then rerun your tests15:26
openstackgerritManuel Buil proposed openstack/openstack-ansible-os_ceilometer master: Add the extra parameter in the tooz dependency  https://review.openstack.org/60316815:28
spatelOk let me do15:28
spatelalso i would like to get your side result too :)15:29
spatelI am going to try NUMA too to see if that increase perf15:29
jamesdentoni don't have anything to test with at the moment, i'm tied up in another environment with no computes15:29
spatelno worry.. when you get time..15:30
spatelno rush.. i have testing here anyway..15:30
spateli just wanted to see if someone else getting same behavior or its just me :(15:30
jamesdentonWell, it's very application-specific behavior IMO15:31
spatelmay be cloudnull get help me out he has nice lab :)15:31
jamesdentonwhat you're seeing is not unusual15:31
spatelyes15:31
*** skiedude has joined #openstack-ansible15:33
*** trident has quit IRC15:38
*** trident has joined #openstack-ansible15:39
*** Leo_m has joined #openstack-ansible15:40
*** KeithMnemonic has quit IRC15:52
mbuil_guys, I think os_ceilometer is currently broken: https://review.openstack.org/#/c/603143/. Hopefully, this patch fixes everything: https://review.openstack.org/#/c/603165/15:57
*** goldenfri has quit IRC16:03
spotzmbuil_: Review on the second, will hold off on the first until it goes through. Not sure who's about today or still travelling/recovering16:04
*** cshen has joined #openstack-ansible16:04
mbuil_spotz: thanks :), I also found this problem ==> https://review.openstack.org/#/c/603168/116:05
spateljamesdenton: i have to leave but i will keep you posted my result16:07
jamesdentonplease do, thanks16:07
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone stable/rocky: Define cache_timeout variable  https://review.openstack.org/60296116:07
spotzthat one is done too mbuil_16:07
*** spatel has quit IRC16:08
openstackgerritNicolas Bock proposed openstack/openstack-ansible-openstack_hosts master: Do not uninstall grub2  https://review.openstack.org/60248216:09
odyssey4meguilhermesp mnaser any news on figuring out how to resolve the centos conflicting package issue?16:10
openstackgerritManuel Buil proposed openstack/openstack-ansible-os_ceilometer stable/rocky: Bug fix: ceilometer_service_setup_host missing  https://review.openstack.org/60317516:11
odyssey4meOtherwise I'll fire up a test node now and see if I can figure it out, given it's blocking a ton of things.16:11
openstackgerritManuel Buil proposed openstack/openstack-ansible-os_ceilometer stable/rocky: Add the extra parameter in the tooz dependency  https://review.openstack.org/60317616:11
*** gyee has joined #openstack-ansible16:12
*** olivierb has quit IRC16:19
*** goldenfri has joined #openstack-ansible16:24
openstackgerritMerged openstack/openstack-ansible stable/pike: Bump plugins SHA to include recent bugfixes  https://review.openstack.org/60299516:28
openstackgerritMerged openstack/openstack-ansible master: Remove non-working dynamic build of AIO scenario table  https://review.openstack.org/60295316:28
openstackgerritLogan V proposed openstack/openstack-ansible-haproxy_server master: Add 'absent' service state  https://review.openstack.org/60315816:37
*** dxiri has quit IRC16:38
openstackgerritMerged openstack/openstack-ansible-os_keystone stable/ocata: Updated from OpenStack Ansible Tests  https://review.openstack.org/60231016:51
*** goldenfri has quit IRC16:55
openstackgerritMerged openstack/openstack-ansible-os_keystone stable/ocata: import zuul job settings from project-config  https://review.openstack.org/59475216:58
openstackgerritWayne Warren proposed openstack/openstack-ansible-ops master: Make openstack-service-setup compatible with older ansible  https://review.openstack.org/60286017:01
openstackgerritMerged openstack/openstack-ansible-os_keystone stable/queens: import zuul job settings from project-config  https://review.openstack.org/59510917:08
*** dxiri has joined #openstack-ansible17:12
*** dxiri has quit IRC17:13
*** dxiri has joined #openstack-ansible17:13
guilhermespodyssey4me: sorry for the delay. Can you provide me again the patch that is blocking stuff?17:18
odyssey4meguilhermesp: at this stage it's a lack of path - but I can link the error to you17:18
odyssey4mehttps://review.openstack.org/#/c/602927/ needs to merge, and it's failing here: http://logs.openstack.org/27/602927/4/check/openstack-ansible-functional-centos-7/622cf6a/logs/ara-report/result/96d301ab-e9e8-4b92-9512-2b436c3b4db7/17:19
odyssey4meit's when trying to install the galera client on the host, which already has a conflicting package installed17:19
guilhermespah odyssey4me my bad, I thought that the issue was about the trove patch linked earlier17:20
odyssey4meguilhermesp: oh that's another issue, which is actually in the rabbitmq role where it uses a yum plugin to do pinning... for some reason that's failing, but that's a secondary issue which we can look at later because it's not happening in master17:22
guilhermespso odyssey4me are on it already? I can take a look at this now if you want17:22
odyssey4meguilhermesp: I'm trying to replicate the issue now - not sure if it'll work given I'm not using an infra host17:25
odyssey4meI do have a conflicting package from anaconda on the host, so I'm hoping that is good enough to work it out17:26
guilhermespso I think worth it to have a second hand on this. Going to run locally17:26
odyssey4methanks guilhermesp - appreciate it17:28
odyssey4mean option we have is to merge https://review.openstack.org/#/c/602927/4 and https://review.openstack.org/#/c/602907/3 - although I would prefer to keep them separate to make porting back easier because only one needs porting back17:29
*** fghaas has quit IRC17:32
guilhermesplet me see if I can figure out another option too. I'm already running the tests locally17:34
*** luksky has joined #openstack-ansible17:35
d34dh0r53cloudnull: https://review.openstack.org/#/c/586612/ :O17:40
d34dh0r53When you see it...17:40
*** goldenfri has joined #openstack-ansible17:42
*** shardy has quit IRC17:46
odyssey4meguilhermesp: ok, so I've confirmed that yum is not case-sensitive for package names - which is a bit of a pain when it comes to trying to remove a package of the same name17:50
openstackgerritLogan V proposed openstack/openstack-ansible-haproxy_server master: Add 'absent' service state  https://review.openstack.org/60315817:50
guilhermespthat's not good odyssey4me17:52
odyssey4mewell, there is another issue17:54
guilhermespodyssey4me: so this was the commit the fired up the centos isseus? https://github.com/openstack/openstack-ansible-galera_client/commit/e74cdc291170f4aceb8effb30650d995475bcf5517:54
*** priteau has quit IRC17:55
odyssey4mefor some reason, I'm seeing a bunch of openstack packages installed - in this case I even see the 'glance' package installed - and this is a source build, so it shouldn't be there17:55
odyssey4meguilhermesp: no, that was trying to solve this issue17:55
odyssey4methe package 'mariadb-common-3' doesn't exist, so it does nothing17:56
odyssey4methe package name is 'mariadb-common' (lower case) and the version is 3:10.1.20-2.el717:56
odyssey4meso we could try and make it remove a specific version, but that would seem to be less generic and not really sustainable17:57
odyssey4meI'm kinda thinking that perhaps we list the installed packages, and grep them using case sensitivity, then uninstall those17:57
odyssey4mebut doing that would also remove deps - and in my case those included a bunch of things like 'glance' (which is ok, it shouldn't be there) and 'MySQL-python' which needs to be there :/17:58
odyssey4meit looks like I can reinstall mysql-python afterwards though, which is good17:59
openstackgerritMerged openstack/openstack-ansible-ops master: Deploy ELK in MNAIO  https://review.openstack.org/58661218:00
odyssey4meguilhermesp: hmm, I have a suspicion that yum is installing 'recommended' packages, not just the ones asked for18:09
*** spatel has joined #openstack-ansible18:09
*** openstackgerrit has quit IRC18:10
*** openstackgerrit has joined #openstack-ansible18:10
openstackgerritDavid Wilde proposed openstack/openstack-ansible-ops master: MNAIO ELK Updates  https://review.openstack.org/60320118:10
odyssey4meoh, apparently such a thing doesn't exist18:11
*** DanyC has quit IRC18:11
*** DanyC has joined #openstack-ansible18:12
guilhermesphuuum... I'm still trying to reproduce it locally, I'm facing some issues with run_tests.sh odyssey4me how did you run your env?18:13
*** aspiers has quit IRC18:13
odyssey4meguilhermesp: ./run_tests.sh functional18:14
odyssey4mealthough I did hit an issue when it got to neutron and created virtualenvs... it failed to import requests and broke, so I have to make them manually on the host18:14
guilhermesphere's what happens when I run run_tests.sh functional  http://paste.openstack.org/show/730200/18:16
*** DanyC has quit IRC18:16
guilhermespwithout functional parameter, I got a python3 interpreter issue. Odd coz I always use clean centos vm's for my tests18:16
odyssey4meweird18:20
odyssey4meok, I think I have a possible solution - will push up a patch now18:20
guilhermespI will spin my test env anyway, I'm curious about the issue18:21
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_client master: Find the list of packages to remove using rpm  https://review.openstack.org/60320518:31
guilhermespodyssey4me: seems that I was accidentally using an old centos vm that I was using to test other sutff. Running the tests in a clean env now...18:31
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova master: Rename SUSE vars file to cater for any version  https://review.openstack.org/60292718:31
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_client master: Find the list of packages to remove using rpm  https://review.openstack.org/60320518:35
FrankZhangodyssey4me: hey man, looks this commit's gating always failed at tempest keystone plugin install step, do we have any known problem at this point? Since I think this one unrelated to current commit. https://review.openstack.org/#/c/599641/ error log: http://paste.openstack.org/raw/730201/18:36
*** cshen has quit IRC18:36
odyssey4meFrankZhang: yeah, it's failing because of the tempest plugins - then I made it depend on the patch which puts the plugins there, then it failed because of other issues - I've not had time yet to circle back18:37
odyssey4meFrankZhang: you can see in PS4 that the barbican build failed before adding the plugins18:38
odyssey4menow the basbican build works, but it breaks everything else18:38
odyssey4me*barbican18:38
odyssey4meso the tempest patch in depends-on needs to be modified to either pin, or we need to do something else rather than that18:38
odyssey4meit may just be a case of implementing reasonable pins for all those plugins18:39
guilhermespodyssey4me: so I'm going to run your PR so if we would have any issue we have more eyes on it18:39
FrankZhangodyssey4me: a bit worry about the master and rocky merged commit, seems gating didn't complaint against this point, wonder why it works for master and rocky.18:42
odyssey4meFrankZhang: master and rocky already had the right os_tempest patch18:42
odyssey4mealso, they have recent enough dep packages for it to work18:42
odyssey4meis there a reason why you absolutely need this in queens, especially given it's just additional tests18:43
FrankZhangodyssey4me: not quite a special reason, since my team wish ship barbican in rocky and so on. So far it's good.18:44
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_client master: Find the list of packages to remove using rpm  https://review.openstack.org/60320518:44
*** kmlau777 has joined #openstack-ansible18:45
odyssey4meFrankZhang: if it's not essential for Queens, then I'd recommend just abandoning the queens patch - there's no reason to create more work if it's not required, and implementing it requires changes which could be destabilising which seems countr-productive given that queens is stable18:45
FrankZhangodyssey4me: you're right, talking to team briefly and probably abandon it soon18:46
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova master: Remove galera_client meta-dependency  https://review.openstack.org/60290718:47
odyssey4meguilhermesp: it looks to me like it works, hopefully the method isn't too ugly :p18:48
guilhermesphahahaha odyssey4me I'm following your changes and testing them too, seems to work indeed18:49
kmlau777Greetings! I'm really stuck in setup-openstack.yml and have no clue what causes the error....18:50
kmlau777Can somebody please shed some light on this error?18:51
kmlau777https://pastebin.com/Dkzidgqx18:51
kmlau777I'm trying to deploy ocata with 3 infra 1 compute, 1 storage and 1 loadbalancer18:52
openstackgerritArx Cruz proposed openstack/openstack-ansible-os_tempest master: WIP - Enable stackviz support  https://review.openstack.org/60310018:56
*** cshen has joined #openstack-ansible18:56
openstackgerritNicolas Bock proposed openstack/openstack-ansible master: Change into correct path  https://review.openstack.org/60234119:02
*** aspiers has joined #openstack-ansible19:05
odyssey4mekmlau777: do you have a host called lb1 in your inventory?19:07
odyssey4mekmlau777: also, what tag/release are you using?19:07
kmlau777Okay...I finally get it working...I add the loadbalancer hostname 'lb1' to the host table in the deployer node and the error is gone.19:07
*** fghaas has joined #openstack-ansible19:07
kmlau777odyssey4me: I use the latest ocata tag19:07
odyssey4mekmlau777: oh man, ocata is pretty much dead now - I'd suggest upgrading when you can as it's not likely to get any more fixes19:08
odyssey4mehwoarang nicolasbock is zypper case sensitive when looking at package names for install/remove ?19:09
kmlau777odyssey4me: I understand...but for odd requirement ocata was requested....19:09
odyssey4mekmlau777: well, hopefully it all works - there are still some fixes trickling in19:10
kmlau777odyssey4me: Anyhow thanks for the advice!19:10
openstackgerritNicolas Bock proposed openstack/openstack-ansible-openstack_hosts master: Do not uninstall grub2  https://review.openstack.org/60248219:10
odyssey4mehwoarang nicolasbock it seems it has an option to be19:11
kmlau777odyssey4me: regarding the error i got earlier, is there documentation somewhere i must have overlooked to ensure the deployer node should resolve to all the targets by names?19:11
odyssey4mekmlau777: no, that error is unusual19:12
kmlau777odyssey4me: argh. i see.19:12
odyssey4mekmlau777: I suspect that's due to a bug that crept in recently, but I think there have been recent fixes to the plugins repo to fix that19:13
kmlau777odyssey4me: hopefully it doesnt ripple into other error19:13
odyssey4meI think https://review.openstack.org/601825 resolved it.19:13
nicolasbockodyssey4me: It's case sensitive:19:13
nicolasbocksudo zypper remove Git-core19:13
nicolasbockNo provider of 'Git-core' found.19:14
odyssey4menicolasbock: ok, awesome - I just saw that the removal of conflicting packages for galera_server is only done after the packages are installed... so I'm going to patch it up to re-order, and to make the centos removals happen again19:14
nicolasbockNice!19:14
*** dxiri has quit IRC19:16
*** dxiri has joined #openstack-ansible19:16
*** fghaas has quit IRC19:20
prometheanfirenice19:23
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_server master: Revisit conflicting package removal  https://review.openstack.org/60321519:33
*** DanyC has joined #openstack-ansible19:37
strattaoodyssey4me or cloudnull I have been trying to look into details on adding the ec2-api to our openstack-ansible deployments. Can you or anyone else shed some light on why there isn’t an openstack-ansible-ec2-api role already? Is there really no demand for such a thing or is there some other big technical limitation?19:41
cjloaderhi guys19:41
guilhermespodyssey4me: https://review.openstack.org/#/c/602957/1 the rabbitmq failing test is passing.19:45
odyssey4meguilhermesp: so it's transient?19:47
guilhermespthat's what I'm suspecting odyssey4me19:47
odyssey4mestrattao: as far as I know, the ec2 api was deprecated long ago from nova - is there a new project to make one now?19:47
guilhermespboth patches are almost finishing and seems ok19:48
*** aspiers has quit IRC19:48
odyssey4meguilhermesp: oh man, transient issues suck19:49
guilhermespindeed... and suckes to identify the real cause =/19:49
guilhermespsucks*19:50
strattaoodyssey4me: my understanding was that they weren’t deprecating it per-se, but just putting it into it’s own separate repository. As far as I can tell, that lives here: https://github.com/openstack/ec2-api and still seems to have recent development and activity, unless I’m missing something...19:54
odyssey4mestrattao: ok, then no-one's put it in to make it work - neither in the nova role, nor in its own19:55
strattaoodyssey4me: ok, a little surprising that there wouldn’t be more demand for that! Either way, looks like I’ll be getting started on that work then. Hope you guys don’t mind if I come asking for some tips or pointers.19:57
odyssey4mestrattao: sure, happy to help!19:57
*** KeithMnemonic has joined #openstack-ansible20:01
odyssey4meguilhermesp: did https://review.openstack.org/#/c/603205/ work in your tests?20:01
odyssey4meif so, cores - it's be good to get some reviews20:01
*** aspiers has joined #openstack-ansible20:02
odyssey4mewell, then there's http://logs.openstack.org/05/603205/3/check/openstack-ansible-functional-centos-7/b59df76/logs/ara-report/result/5a783dea-e9e1-44ec-a257-e398eb560c54/ :/20:03
*** strattao has quit IRC20:03
*** hamzaachi has joined #openstack-ansible20:03
*** diablo_rojo has joined #openstack-ansible20:03
guilhermespyep odyssey4me it's working here20:04
guilhermespoh, I saw that removed more then you wanted, you mean postfix and redhat-lsb-core?20:06
*** fghaas has joined #openstack-ansible20:07
odyssey4meguilhermesp: yeah, I'm not so sure we should be removing those20:09
guilhermespyou want me to take a look and see if we can specify a little more what we want to remove?20:09
*** goldenfri has quit IRC20:10
guilhermespodyssey4me: also, all green https://review.openstack.org/#/c/602957/1 https://review.openstack.org/#/c/602965/120:11
odyssey4meguilhermesp: well, maybe 'mariadb*' should just be 'mariadb-common*' ?20:12
guilhermespI think so, did you change already?20:13
guilhermespI can search for the packages and its dependencies so we can specify something more accurate20:13
odyssey4methat's be good, if you have something up. - unfortunately I wiped my previous test system20:15
guilhermespI'm almost finishing to set up a new env20:16
guilhermespI can test it out20:16
logan-just discovered the 'intopic' gerrit search since the new gerrit puts the branch name in the topic when you cherry pick20:17
*** hamzaachi has quit IRC20:18
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_client master: Find the list of packages to remove using rpm  https://review.openstack.org/60320520:18
guilhermesp^ going to test it locally, I'd do that but you're faster odyssey4me hahaha20:19
odyssey4meguilhermesp: time for me to !computer, thanks for helping!20:19
odyssey4mehopefully https://review.openstack.org/602927 works now20:20
guilhermesphahahah thanks odyssey4me I can assume this if any issue appears20:20
*** fghaas has quit IRC20:23
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_server master: Revisit conflicting package removal  https://review.openstack.org/60321520:24
*** fghaas has joined #openstack-ansible20:25
*** lbragstad has quit IRC20:26
*** lbragstad has joined #openstack-ansible20:27
odyssey4mealright, chat with y'all tomorrow20:27
*** mmercer has joined #openstack-ansible20:28
cjloadercan someone point me to the repo where zuul jobs are created for osa?20:31
*** ansmith has quit IRC20:38
openstackgerritGerman Eichberger proposed openstack/openstack-ansible-os_octavia master: Break out config files for each octavia service  https://review.openstack.org/60322820:51
*** fghaas has quit IRC20:55
*** kmlau777 has quit IRC21:05
guilhermespso odyssey4me we are still getting conflicts =( http://logs.openstack.org/27/602927/5/check/openstack-ansible-functional-centos-7/f26bef8/logs/ara-report/result/36d6ce3b-aabf-45fe-b8a0-af24c5cb2e62/21:21
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_client master: Find the list of packages to remove using rpm  https://review.openstack.org/60320521:28
*** spatel has quit IRC21:29
openstackgerritDavid Wilde proposed openstack/openstack-ansible stable/rocky: Bump requirement SHA  https://review.openstack.org/60323321:29
*** diablo_rojo has left #openstack-ansible21:35
*** Leo_m has quit IRC21:43
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_server master: Revisit conflicting package removal  https://review.openstack.org/60321521:49
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_client master: Find the list of packages to remove using rpm  https://review.openstack.org/60320521:51
*** skiedude has quit IRC21:55
*** alongwayup has joined #openstack-ansible21:59
alongwayuphey, has anyone seen this before? running newton, and client did:21:59
alongwayupneutron port-update $port-uuid --allowed-address-pairs type=dict list=true ip_address=0.0.0.0/021:59
alongwayupthis added 0.0.0.0/1 and 128.0.0.0/1 to the ipset list of all internal ip's, effectively breaking the security groups and allowing all traffic in22:00
alongwayupseems exactly like https://bugs.launchpad.net/neutron/+bug/1461054 - but that's really old (this customer launched on liberty, and that was fixed in juno)22:01
openstackLaunchpad bug 1461054 in neutron kilo "[OSSA 2015-012] Adding 0.0.0.0/0 to allowed address pairs breaks l2 agent (CVE-2015-3221)" [Critical,Fix committed] - Assigned to Tristan Cacqueray (tristan-cacqueray)22:01
alongwayupso if you did an iptables on nova host, you would see the rule that allowed inter-vm talk:22:01
alongwayup 905K   55M RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set NIPv44046d62c-59c8-4fd0-a547- src22:01
alongwayupand the above 0.0.0.0/1 and 128.0.0.0/1 where added to that set22:02
*** luksky has quit IRC22:02
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_client master: Find the list of packages to remove using rpm  https://review.openstack.org/60320522:07
*** DanyC has quit IRC22:11
*** DanyC has joined #openstack-ansible22:12
prometheanfirecloudnull: was it piston that was gentoo based (talking to a metacloud guy now)?22:15
*** DanyC has quit IRC22:16
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-galera_client master: Find the list of packages to remove using rpm  https://review.openstack.org/60320522:23
*** cshen has quit IRC22:33
*** cshen has joined #openstack-ansible22:35
*** cshen has quit IRC22:39
*** cshen has joined #openstack-ansible22:44
*** cshen has quit IRC22:48
openstackgerritArx Cruz proposed openstack/openstack-ansible-os_tempest master: WIP - Enable stackviz support  https://review.openstack.org/60310022:56
*** dxiri has quit IRC23:12
*** mmercer has quit IRC23:19
*** tosky has quit IRC23:23

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!