| *** TxGirlGeek has quit IRC | 00:04 | |
| *** dim__ has quit IRC | 00:06 | |
| *** TxGirlGeek has joined #openstack-ansible | 00:08 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-tests master: Fix Gnocchi service URLs https://review.openstack.org/541946 | 00:11 |
|---|---|---|
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_aodh master: Update the Tempest plugin to point towards the new plugin https://review.openstack.org/541270 | 00:12 |
| *** pradiprwt has quit IRC | 00:12 | |
| *** markvoelker has joined #openstack-ansible | 00:17 | |
| openstackgerrit | Antony Messerli proposed openstack/openstack-ansible-ops master: Replace pxelinux with iPXE support in MNAIO https://review.openstack.org/539327 | 00:20 |
| *** weezS has quit IRC | 00:20 | |
| *** markvoelker has quit IRC | 00:24 | |
| *** rodolof has quit IRC | 00:27 | |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-pip_install stable/ocata: Zuul: Remove project name https://review.openstack.org/541953 | 00:40 |
| *** chyka has quit IRC | 00:42 | |
| *** Guy has quit IRC | 00:51 | |
| *** kstev1 has quit IRC | 00:51 | |
| *** brad[] has quit IRC | 01:15 | |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-os_molteniron stable/pike: Zuul: Remove project name https://review.openstack.org/541965 | 01:20 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_aodh master: Update the Tempest plugin to point towards the new plugin https://review.openstack.org/541270 | 01:25 |
| *** tux_ has joined #openstack-ansible | 01:30 | |
| *** weezS has joined #openstack-ansible | 01:33 | |
| *** TxGirlGeek has quit IRC | 01:36 | |
| *** Leo_m has quit IRC | 01:36 | |
| *** chyka has joined #openstack-ansible | 01:48 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_aodh master: Update the Tempest plugin to point towards the new plugin https://review.openstack.org/541270 | 01:53 |
| *** chyka has quit IRC | 01:53 | |
| *** acormier has quit IRC | 02:03 | |
| *** weezS has quit IRC | 02:13 | |
| *** rama_y has quit IRC | 02:26 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_aodh master: Update the Tempest plugin to point towards the new plugin https://review.openstack.org/541270 | 02:43 |
| *** acormier has joined #openstack-ansible | 02:44 | |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-apt_package_pinning stable/pike: Zuul: Remove project name https://review.openstack.org/541995 | 03:01 |
| *** acormier has quit IRC | 03:02 | |
| *** udesale has joined #openstack-ansible | 03:15 | |
| mnaser | mhayden: i think your theory about rdo+galera is valid. suse is working properly now for aodh but centos isn't | 03:26 |
| *** woodard_ has joined #openstack-ansible | 03:34 | |
| *** woodard has quit IRC | 03:34 | |
| *** dave-mccowan has quit IRC | 03:39 | |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-os_glance stable/pike: Zuul: Remove project name https://review.openstack.org/542005 | 03:41 |
| *** rama_y has joined #openstack-ansible | 04:07 | |
| *** esberglu has quit IRC | 04:10 | |
| *** bhujay has joined #openstack-ansible | 04:21 | |
| *** esberglu has joined #openstack-ansible | 04:26 | |
| *** weezS has joined #openstack-ansible | 04:27 | |
| *** esberglu has quit IRC | 04:30 | |
| *** dariko has joined #openstack-ansible | 04:35 | |
| *** weezS has quit IRC | 04:46 | |
| *** tux_ has quit IRC | 04:52 | |
| *** tux_ has joined #openstack-ansible | 04:53 | |
| *** tux_ has quit IRC | 04:58 | |
| *** tescra has joined #openstack-ansible | 04:59 | |
| tescra | hi | 04:59 |
| tescra | is this branch stable https://github.com/openstack/openstack-ansible/releases/tag/16.0.3? | 05:00 |
| *** acormier has joined #openstack-ansible | 05:03 | |
| *** acormier has quit IRC | 05:07 | |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible stable/ocata: Zuul: Remove project name https://review.openstack.org/542027 | 05:21 |
| *** errr_ has joined #openstack-ansible | 05:29 | |
| *** errr_ has joined #openstack-ansible | 05:29 | |
| Taseer | evrardjp: congress.conf is placed in /etc/congress. however, it tries to load it from /openstack/venvs/congress-untagged/etc/ | 05:32 |
| Taseer | how do I change it ? | 05:32 |
| *** pmannidi has quit IRC | 05:34 | |
| *** dariko has quit IRC | 05:34 | |
| *** dariko has joined #openstack-ansible | 05:35 | |
| *** gkadam has joined #openstack-ansible | 05:36 | |
| *** lbragstad has quit IRC | 05:37 | |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-os_ironic stable/pike: Zuul: Remove project name https://review.openstack.org/542029 | 05:42 |
| *** errr_ has quit IRC | 05:43 | |
| *** errr_ has joined #openstack-ansible | 05:43 | |
| *** errr_ has quit IRC | 05:44 | |
| *** dariko has quit IRC | 05:56 | |
| *** threestrands has quit IRC | 05:57 | |
| *** eumel8 has joined #openstack-ansible | 05:59 | |
| *** errr_ has joined #openstack-ansible | 05:59 | |
| *** errr has quit IRC | 06:01 | |
| *** errr_ is now known as errr | 06:02 | |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-openstack_hosts stable/ocata: Zuul: Remove project name https://review.openstack.org/542031 | 06:02 |
| *** chyka has joined #openstack-ansible | 06:22 | |
| *** chyka has quit IRC | 06:26 | |
| *** rama_y has quit IRC | 06:59 | |
| *** eumel8 has quit IRC | 07:01 | |
| *** eumel8 has joined #openstack-ansible | 07:01 | |
| hwoarang | good morning | 07:01 |
| *** sar has joined #openstack-ansible | 07:01 | |
| *** sar has quit IRC | 07:05 | |
| *** sar has joined #openstack-ansible | 07:05 | |
| hwoarang | pfff, what is up with galera ... | 07:12 |
| *** eumel8 has quit IRC | 07:28 | |
| *** ivveh has joined #openstack-ansible | 07:30 | |
| *** pcaruana has joined #openstack-ansible | 07:31 | |
| *** mardim has joined #openstack-ansible | 07:40 | |
| *** armaan has quit IRC | 07:49 | |
| *** armaan has joined #openstack-ansible | 07:49 | |
| evrardjp | hwoarang: hey | 07:58 |
| evrardjp | I fixed it yesterday evening | 07:58 |
| evrardjp | at least I thought I did | 07:58 |
| evrardjp | let me check | 07:58 |
| evrardjp | well | 07:58 |
| evrardjp | workarounded | 07:58 |
| evrardjp | https://review.openstack.org/#/c/541855/ | 07:58 |
| *** ivveh has quit IRC | 07:58 | |
| evrardjp | I don't know what's really wrong | 07:59 |
| evrardjp | but in the meantime, we've got a workaround | 07:59 |
| openstackgerrit | Maxime Guyot proposed openstack/openstack-ansible master: [WIP] Ceph RadosGW integration https://review.openstack.org/517856 | 08:02 |
| gokhan | good morning evrardjp, I am also testing your galera pacth on xenial. now it is on galera-install.yml | 08:02 |
| evrardjp | which branch? | 08:02 |
| evrardjp | I haven't patched the O/P yet | 08:03 |
| evrardjp | gokhan: if you're testing master without upgrades, that should be fine | 08:03 |
| gokhan | evrardjp, on pike . I thought it also should work on pike. | 08:04 |
| evrardjp | yeah but I got a merge conflict, you solved it? | 08:04 |
| evrardjp | probably nothing too big, but hey, just saying :) | 08:04 |
| *** mbuil has joined #openstack-ansible | 08:06 | |
| evrardjp | hwoarang: little question | 08:06 |
| evrardjp | what does this contain: https://github.com/openstack/openstack-ansible-galera_server/blob/stable/pike/vars/suse-42.yml#L17-L18 ? | 08:07 |
| gokhan | evrardjp, I manually applied your patch. and one question it tries to start mysql normally on infra1 and then failed but it is igniring this ? is this expected state ? | 08:07 |
| gokhan | *ignoring | 08:07 |
| evrardjp | gokhan: maybe, I don't know. | 08:07 |
| evrardjp | check the installed version | 08:07 |
| evrardjp | also if you have .31 already installed, my patch won't fix it | 08:08 |
| gokhan | evrardjp, I checked it installs .30 patch | 08:09 |
| evrardjp | ok good so that's maybe normal | 08:09 |
| gokhan | evrardjp, good news :) mysql startted on infra2 galera container | 08:11 |
| *** chyka has joined #openstack-ansible | 08:11 | |
| evrardjp | hwoarang: do you see a problem to backport https://github.com/openstack/openstack-ansible-galera_server/commit/a37bfcca09e4cfe92f4732ae10d30b9c82f586c2 ? | 08:13 |
| evrardjp | if I backport it, I can fix lower branches without moving to -nv any part (either the upgrade jobs, or the suse jobs) | 08:14 |
| *** olivierbourdon38 has joined #openstack-ansible | 08:14 | |
| *** chyka has quit IRC | 08:16 | |
| *** pkaralis has quit IRC | 08:16 | |
| *** mnaser has quit IRC | 08:17 | |
| hwoarang | evrardjp: sorry i had to run | 08:17 |
| gokhan | evrardjp, galera is workedsuccessfully on pike with your patch. | 08:18 |
| *** mnaser has joined #openstack-ansible | 08:18 | |
| hwoarang | evrardjp: so the repo contains the mariadb/galera packages but that's only for pike | 08:18 |
| hwoarang | in queens i removed that and i added the mariadb.org upstream repos | 08:18 |
| hwoarang | like centos | 08:18 |
| hwoarang | evrardjp: so if you want to backport that in pike then i am fine with it | 08:18 |
| hwoarang | evrardjp: this is not quite right for suse https://review.openstack.org/#/c/541855/3 | 08:20 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-galera_server stable/pike: Prevent latest mariadb version https://review.openstack.org/542083 | 08:23 |
| evrardjp | hwoarang: https://downloads.mariadb.com/MariaDB/mariadb-10.1.30/yum/ | 08:24 |
| hwoarang | hmm that's new... | 08:24 |
| evrardjp | well I had two questions basically | 08:24 |
| evrardjp | well that's not documented | 08:24 |
| evrardjp | that's a completely different repo | 08:24 |
| evrardjp | it has archives | 08:24 |
| evrardjp | so I thought it made sense to use it, for closer matching | 08:25 |
| evrardjp | I can use sles 12 | 08:25 |
| evrardjp | that was my first question | 08:25 |
| evrardjp | my second question is what does obs contain | 08:25 |
| *** peri has joined #openstack-ansible | 08:25 | |
| evrardjp | if it contains only .30 we don't need to backport that massive change here https://review.openstack.org/#/c/542083/ | 08:25 |
| *** eumel8 has joined #openstack-ansible | 08:25 | |
| hwoarang | obs contains 10.2 | 08:25 |
| evrardjp | ok | 08:25 |
| hwoarang | tomorrow maybe 10.3 | 08:25 |
| hwoarang | it's a moving target | 08:26 |
| evrardjp | ok so it's probably best to stick with my change in stable/ branches. | 08:26 |
| hwoarang | yes but the backport of the a37bfcca09e4cfe92f4732ae10d30b9c82f586c2 has to happen before your change | 08:26 |
| evrardjp | it's included | 08:26 |
| evrardjp | have a look at my link | 08:27 |
| evrardjp | It would be nice of you to review at least the pike one | 08:27 |
| evrardjp | that's until we fix it in non-emergency mode | 08:27 |
| hwoarang | hmm what link? | 08:27 |
| hwoarang | wait before i am mixing things | 08:28 |
| evrardjp | https://review.openstack.org/#/c/542083/ | 08:28 |
| hwoarang | *because | 08:28 |
| evrardjp | so master I just changed the url | 08:28 |
| hwoarang | ah yes | 08:28 |
| evrardjp | Pike I changed the url + backport of the using url | 08:28 |
| evrardjp | and we can merge this from O onwards. | 08:29 |
| evrardjp | this way upgrades are not broken | 08:29 |
| hwoarang | do you know what this downloads.mariadb repo is for? | 08:29 |
| evrardjp | what do you mean? | 08:29 |
| hwoarang | why downloads.mariadb instead of yum.mariadb repo? | 08:29 |
| evrardjp | https://mariadb.com/downloads/mariadb-tx | 08:29 |
| hwoarang | the domain has changed so i was just curious | 08:29 |
| evrardjp | the domain hasn't changed | 08:30 |
| evrardjp | that's the deal | 08:30 |
| evrardjp | well | 08:30 |
| evrardjp | you now have 2 places | 08:30 |
| evrardjp | they recommand to use yum. blabla to have always the latest with the package manager | 08:30 |
| evrardjp | but I saw the url on the page https://mariadb.com/downloads/mariadb-tx | 08:30 |
| evrardjp | just digged a little deeper | 08:30 |
| hwoarang | i see | 08:31 |
| evrardjp | (see also: https://downloads.mariadb.org/mariadb/repositories/#mirror=nucleus&distro=CentOS&distro_release=centos7-amd64--centos7&version=10.1 ) | 08:31 |
| evrardjp | as you can see, by default everything point out to yum. | 08:32 |
| evrardjp | yum.blabla | 08:32 |
| evrardjp | but it doesn't prevent the other url for being accessible and being a yum repo | 08:32 |
| hwoarang | right | 08:32 |
| evrardjp | and be frozen | 08:32 |
| hwoarang | ok lets roll with it | 08:32 |
| evrardjp | on top of that | 08:33 |
| evrardjp | ok | 08:33 |
| evrardjp | I'll fix the root cause asap | 08:33 |
| evrardjp | thanks for the help | 08:34 |
| hwoarang | i wonder if the breakage in 10.1.31 is known upstream | 08:34 |
| hwoarang | thank you for the backport | 08:34 |
| hwoarang | evrardjp: could you get https://review.openstack.org/#/c/538933/ in please? | 08:35 |
| hwoarang | and this is no longer necessary https://review.openstack.org/#/c/541417/ it seems. | 08:36 |
| hwoarang | somehow the original layout of the dir was restored | 08:36 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-galera_server stable/ocata: Prevent latest mariadb version https://review.openstack.org/542090 | 08:37 |
| evrardjp | hwoarang: it's maybe not an upstream bug, but our things. But because it will take time to properly investigate, I think it's better we release a fix asap | 08:38 |
| hwoarang | agreed | 08:38 |
| *** gillesMo has joined #openstack-ansible | 08:38 | |
| evrardjp | this way ppl are not wondering what's going on, we can just say: Use the latest version | 08:39 |
| evrardjp | hwoarang: done | 08:40 |
| hwoarang | awesome thanks! | 08:40 |
| *** tnogisto has quit IRC | 08:40 | |
| evrardjp | hwoarang: could you vote on https://review.openstack.org/#/c/542090/1 ? | 08:42 |
| hwoarang | ok | 08:43 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: Test keystone with a memcached pool https://review.openstack.org/540001 | 08:49 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_rally master: Reinitialize the virtualenv https://review.openstack.org/535822 | 08:50 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_rally master: Updated from global requirements https://review.openstack.org/535069 | 08:51 |
| *** vnogin has quit IRC | 08:51 | |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Ability to skip security hardening tags in leap https://review.openstack.org/541057 | 09:02 |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-ops stable/newton: Zuul: Remove project name https://review.openstack.org/542096 | 09:02 |
| *** hw_wutianwei has quit IRC | 09:07 | |
| *** hw_wutianwei has joined #openstack-ansible | 09:08 | |
| *** rodolof has joined #openstack-ansible | 09:14 | |
| *** SmearedBeard has joined #openstack-ansible | 09:22 | |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-os_gnocchi stable/pike: Zuul: Remove project name https://review.openstack.org/542103 | 09:23 |
| *** armaan has quit IRC | 09:23 | |
| *** armaan has joined #openstack-ansible | 09:24 | |
| *** ivveh has joined #openstack-ansible | 09:32 | |
| *** SmearedBeard has quit IRC | 09:41 | |
| odyssey4me | dim_ idlemind that problem is known and was fixed recently - see https://review.openstack.org/#/q/Ic39f884016a6ba0498110267027783001a087a1c | 09:46 |
| *** SmearedBeard has joined #openstack-ansible | 09:47 | |
| *** tnogisto has joined #openstack-ansible | 09:47 | |
| evrardjp | odyssey4me: what about the ocata thing, you have logs that can help now? | 09:48 |
| odyssey4me | evrardjp not sure - will look into it later today | 09:52 |
| openstackgerrit | Taseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit https://review.openstack.org/522491 | 09:52 |
| odyssey4me | evrardjp you'll likely need to also do patches for the galera_client role, otherwise you'll have two different repositories? | 09:52 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: Test keystone with a memcached pool https://review.openstack.org/540001 | 09:54 |
| evrardjp | odyssey4me: yup, I think it's a good idea to have client version and server version identical, while not mandatory | 09:59 |
| evrardjp | also I don't pin for centos and suse, which could lead to issues. | 09:59 |
| odyssey4me | well, the repo you're using is effectively a pin | 09:59 |
| odyssey4me | oh - you mean you don't remove the newer version if it's there? | 10:00 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-galera_client master: Prevent latest mariadb version https://review.openstack.org/542119 | 10:08 |
| evrardjp | odyssey4me: we don't | 10:08 |
| odyssey4me | evrardjp I need https://review.openstack.org/541457 for the ocata debugging | 10:08 |
| evrardjp | odyssey4me: is that a backport? | 10:09 |
| odyssey4me | evrardjp it's not possible to do a cherry-pick as there have been too many changes in the later branches | 10:09 |
| evrardjp | ok | 10:10 |
| odyssey4me | so, as stated in the commit message, I did a copy-paste and cleaned stuff out | 10:10 |
| evrardjp | so I have to review it | 10:10 |
| *** stuartgr has joined #openstack-ansible | 10:11 | |
| odyssey4me | I also need https://review.openstack.org/541443 ported back - perhaps if hwoarang reviews and is happy then I can port it back before merge (due to the mariadb nonsense)? | 10:11 |
| evrardjp | odyssey4me: this starts to be a tangled mess. | 10:14 |
| evrardjp | What if we make the upgrade jobs optional in mariadb, so we can merge things properly in galera_server, so we can fix the whole master for inclusion of things | 10:14 |
| evrardjp | then we can backport all what's not emergency | 10:15 |
| evrardjp | ofc the galera_client and galera_server would have to be merged in priority in all branches to prevent deployers from installing a wrong version | 10:15 |
| hwoarang | hmm | 10:15 |
| evrardjp | or we stop the emergency fixes, and we try to find out what's wrong in that new version | 10:16 |
| evrardjp | I'd say we should ship in the emergency workarounds as fast as possible, to let us the time | 10:17 |
| hwoarang | maybe it's easier to just disable the upgrade jobs for now | 10:17 |
| evrardjp | hwoarang: agreed. | 10:17 |
| evrardjp | Let me do it then in my patch real quick. | 10:17 |
| odyssey4me | yeah - disable the upgrade tests temporarily (it looks like we're upgrading 10.1->10.1 anyway?) to get the changes merged, update the integrated repo with those changes so that people can use them | 10:17 |
| odyssey4me | hwoarang are you able to look into or find out whether this is a known upstream bug | 10:18 |
| odyssey4me | while evrardjp implements the pinning to get us by | 10:18 |
| hwoarang | odyssey4me: i am in a meeting now so i will check a bit later. i also need to see the galera logs maybe there is sometihng obvious there | 10:19 |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Add os_gnocchi role to required projects https://review.openstack.org/541902 | 10:19 |
| *** hw_wutianwei has quit IRC | 10:22 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-galera_server stable/pike: Prevent latest mariadb version https://review.openstack.org/542083 | 10:22 |
| *** hw_wutianwei has joined #openstack-ansible | 10:22 | |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-rabbitmq_server stable/ocata: Zuul: Remove project name https://review.openstack.org/542120 | 10:23 |
| odyssey4me | evrardjp a bit of a nit, but perhaps the commit subject should be clearer about the version it's preventing given that it's one form of changelog? 'latest' doesn't really carry much meaning | 10:23 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-galera_server master: Prevent latest mariadb version https://review.openstack.org/541855 | 10:23 |
| evrardjp | odyssey4me: well it prevents ALL the latests in the future, because now it effectively freezes | 10:24 |
| evrardjp | and on top of that it also prevents a certain package. | 10:24 |
| odyssey4me | yes, but we're going to undo that once we've figured out the root cause and resolved that, right? | 10:24 |
| evrardjp | so I guess yes I could change | 10:24 |
| evrardjp | mmm I don't know, I guess we'll have to see | 10:25 |
| evrardjp | I'd undo for master branch | 10:25 |
| odyssey4me | or are you thinking of maintaining this pin forever? | 10:25 |
| evrardjp | until we find out the root cause, I'd say keep the pin | 10:26 |
| evrardjp | when root cause found, remove the pin, keep the freeze | 10:26 |
| evrardjp | maybe update the version | 10:26 |
| evrardjp | minor version* | 10:26 |
| odyssey4me | yet another thing to maintain - fun times | 10:26 |
| evrardjp | yeah, but it's properly frozen | 10:27 |
| odyssey4me | I think it's high time we revisit our pinning/freezing strategy to make it less of a burden | 10:27 |
| evrardjp | we already have as part of the cycle to bump upstream things | 10:27 |
| evrardjp | I think there is no point of this discussion while we are in emergency mode | 10:28 |
| odyssey4me | we actually use to freeze it like that - can't recall which series... we stopped doing it because freezing is still an option for deployers, and they can maintain their freeze | 10:28 |
| odyssey4me | anyway, your call - as you say, we need it now to get us over the hump | 10:28 |
| evrardjp | well you're right it doesn't make sense, we are freezing almost everything, and this one doesn't get freezed, so yes, what is the strategy | 10:29 |
| evrardjp | ? | 10:29 |
| evrardjp | To be decided at the ptg. | 10:29 |
| evrardjp | Document the strategy at that time | 10:29 |
| evrardjp | updated the etherpad with this as a proposed session | 10:31 |
| *** vnogin has joined #openstack-ansible | 10:32 | |
| evrardjp | ok I will focus on backporting the galera_client for now | 10:34 |
| evrardjp | we can decide later what we do depending on the results of the investigation | 10:34 |
| *** vnogin has quit IRC | 10:35 | |
| openstackgerrit | Merged openstack/openstack-ansible-galera_server stable/ocata: Prevent latest mariadb version https://review.openstack.org/542090 | 10:35 |
| hwoarang | odyssey4me: for example here is a 10.1.31 regression https://jira.mariadb.org/browse/MDEV-15230 | 10:37 |
| hwoarang | not sure if it's the same i am still deploying to get logs | 10:37 |
| *** hw_wutianwei has quit IRC | 10:42 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-galera_client stable/pike: Prevent latest mariadb version https://review.openstack.org/542125 | 10:43 |
| *** Smeared_Beard has joined #openstack-ansible | 10:45 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-galera_client stable/ocata: Prevent latest mariadb version https://review.openstack.org/542127 | 10:46 |
| *** SmearedBeard has quit IRC | 10:47 | |
| evrardjp | they are all under the same topic | 10:47 |
| *** vnogin has joined #openstack-ansible | 10:47 | |
| evrardjp | I'll then quickly bump the a-r-r when merged | 10:48 |
| evrardjp | hwoarang: are you working on finding the root cause? | 10:48 |
| evrardjp | I will tackle that too | 10:49 |
| hwoarang | evrardjp: not yet | 10:50 |
| *** hw_wutianwei has joined #openstack-ansible | 10:50 | |
| eumel8 | https://bugs.launchpad.net/openstack-ansible/+bug/1747629 | 10:55 |
| openstack | Launchpad bug 1747629 in openstack-ansible "A worker was found in dead state" [Undecided,New] | 10:55 |
| eumel8 | I tested some more ansible versions but had no much progress or ideas | 10:56 |
| openstackgerrit | Merged openstack/openstack-ansible-tests stable/ocata: Update role test log collection https://review.openstack.org/541457 | 11:02 |
| odyssey4me | eumel8 We've seen that when the deploy node runs out of memory. Ansible just bails out. | 11:07 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-galera_client stable/pike: Prevent latest mariadb version https://review.openstack.org/542125 | 11:09 |
| odyssey4me | evrardjp it may be best to add a reno to inform people that for OSA versions x and before, a variable override should be implemented for mariadb to work | 11:13 |
| odyssey4me | unfortunately we're introducing a new variable which will cause that override to break from the next release onwards - so perhaps we should rethink the addition of a new variable? | 11:15 |
| *** olivierbourdon38 has quit IRC | 11:17 | |
| openstackgerrit | Taseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit https://review.openstack.org/522491 | 11:17 |
| evrardjp | odyssey4me: not sure what you mean | 11:17 |
| odyssey4me | evrardjp well, in all released versions of OSA mariadb will now hang when creating the cluster, right? | 11:17 |
| evrardjp | hwoarang: I got issues on suse changes for Pike on galera_server. Seems issue to start mariadb | 11:17 |
| odyssey4me | so we should inform our user base via a reno how to work around that | 11:18 |
| *** Guy has joined #openstack-ansible | 11:18 | |
| evrardjp | http://logs.openstack.org/83/542083/2/check/openstack-ansible-functional-opensuse-423/01c098a/logs/openstack/container1/YaST2/config_diff_2018_02_08.log.txt.gz shows a difference with config file, maybe there are interesting things to read there | 11:18 |
| eumel8 | odyssey4me: I know. But it wouldn't go better with a newer ansible version | 11:18 |
| evrardjp | odyssey4me: yes | 11:19 |
| odyssey4me | eumel8 nope, we pin the ansible major.minor for each series because things break every release | 11:19 |
| evrardjp | Probably adding a known issue in the integrated repo, with the existing vars to override. | 11:19 |
| odyssey4me | eumel8 also, with each release the memory consumption is growing - not shrinking | 11:19 |
| hwoarang | evrardjp: ok will look at it | 11:19 |
| evrardjp | hwoarang: thanks | 11:19 |
| hwoarang | evrardjp: but for pike we also need this https://github.com/openstack/openstack-ansible-galera_server/commit/baa97f1df2035e2db969152f64b28b791e25d3d9 | 11:20 |
| hwoarang | i think that's the reason | 11:20 |
| evrardjp | odyssey4me: eumel8 the problem is for the translations deploy, because we deploy many components, and therefore the inventory is big. | 11:20 |
| odyssey4me | ah | 11:20 |
| evrardjp | we could probably put things into the same host, to converge more. | 11:20 |
| hwoarang | evrardjp: but the backport is not clean it needs some manual conflict resolution | 11:20 |
| odyssey4me | hwoarang if the backport isn't clean, perhaps there are some other patches missing too? | 11:21 |
| evrardjp | hwoarang: I knew you'd be the expert! | 11:21 |
| hwoarang | odyssey4me: yeah there are some fixes by cloudnull that improve systemd stuff | 11:21 |
| hwoarang | but that's very invasive | 11:21 |
| hwoarang | it's prob easier to resolve conflicts | 11:21 |
| odyssey4me | ah ok | 11:21 |
| hwoarang | it's the private device thing that breaks the service on suse | 11:21 |
| evrardjp | ... I see. | 11:22 |
| hwoarang | evrardjp: i can take over that pike patch | 11:22 |
| evrardjp | eumel8: I wanted to run all of this on metal, but I am blocked for now. | 11:22 |
| odyssey4me | evrardjp lemme pick up that all metal patch to resolve the final issues and new ones which have popped up | 11:23 |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-os_heat stable/newton: Zuul: Remove project name https://review.openstack.org/542139 | 11:23 |
| evrardjp | ok I will issue the release note for each branch and find out why galera fails with latest version. | 11:24 |
| eumel8 | evrardjp: related to translation deploy: something where I can help? We have still the plan to install translation checksite with OSA: https://review.openstack.org/#/c/487936/ | 11:27 |
| hwoarang | evrardjp: can i modify https://review.openstack.org/#/c/542083/ ? | 11:32 |
| *** hw_wutianwei has quit IRC | 11:33 | |
| *** hw_wutianwei has joined #openstack-ansible | 11:34 | |
| *** hw_wutianwei has quit IRC | 11:36 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Document known issue with galera 10.1.31 https://review.openstack.org/542144 | 11:42 |
| evrardjp | hwoarang: yes please do | 11:42 |
| evrardjp | eumel8: yes I will focus on this, it's important for us | 11:43 |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-rsyslog_client stable/newton: Zuul: Remove project name https://review.openstack.org/542145 | 11:43 |
| evrardjp | eumel8: I am testing on my machine on a regular basiss | 11:44 |
| evrardjp | I don't have the same issues, because I have more ram though | 11:44 |
| evrardjp | but I am making sure it's deploying fine. To be honest, I haven't run the test the last week or so. | 11:44 |
| evrardjp | last test was Thursday 25 January 2018 17:59:37 +0000. Working perfectly. | 11:45 |
| evrardjp | so yes I can strip down the machine for ram requirements | 11:45 |
| evrardjp | eumel8: or if there are components that don't need deploying anymore, that would be fine too, I can remove them from the deployment build | 11:45 |
| evrardjp | eumel8: do you know if the scope has changed? | 11:46 |
| *** taseer1 has joined #openstack-ansible | 11:46 | |
| evrardjp | (this way I know HOW far I can strip down things) | 11:47 |
| *** Taseer has quit IRC | 11:47 | |
| *** taseer2 has joined #openstack-ansible | 11:47 | |
| *** chyka has joined #openstack-ansible | 11:47 | |
| eumel8 | evrardjp: I tested also and saw sahara is consuming ram and cinder, especially cinder-backup runs with 100% cpu for a while. | 11:48 |
| *** hamzy_ is now known as hamzy | 11:50 | |
| eumel8 | strip down is a good choice, on the other hand we want to test all translatable stuff | 11:50 |
| *** hw_wutianwei has joined #openstack-ansible | 11:51 | |
| *** taseer1 has quit IRC | 11:51 | |
| *** chyka has quit IRC | 11:52 | |
| evrardjp | eumel8: that's true. But I was wondering if there were things not to translate anymore -- unfollowed projects or things like that. If it's not the case, we'll just make the deploy more efficient. | 11:54 |
| evrardjp | most of the time I see things running weird after octavia | 11:54 |
| evrardjp | happening in rsyslog | 11:54 |
| eumel8 | evrardjp>: sahara-dashboard we have a good translation progress. so it's would be worth to keep in | 11:54 |
| evrardjp | so I was wondering if that wasn't an octavia thing | 11:54 |
| eumel8 | will look... | 11:54 |
| evrardjp | I will try something this afternoon -- mixing both scenarios to put more things on metal, see how it goes. Trove wouldn't be one of them, but collocation can happen. | 11:56 |
| eumel8 | evrardjp: octavia-dashboard is mostly done. 100% translated in Japanese. German, Korean. | 11:57 |
| eumel8 | so no new content is this release. it would be a candidate | 11:57 |
| evrardjp | YOu need ppl for translating into french? | 11:58 |
| eumel8 | evrardjp: in french it's empty. feel free ;) | 12:01 |
| evrardjp | haha let's discuss that later. But yeah, why not. I know a few ppl that could possibly help. | 12:03 |
| eumel8 | evrardjp: new people are still required as long the AI programming is not ready yet :) | 12:05 |
| evrardjp | we're closer everyday. | 12:06 |
| evrardjp | :D | 12:06 |
| eumel8 | for normal text it's ready but in the OpenStack world are a lots of strings and words where the AI is going nuts. We have to work on it, but it's a nice use case for the new technology | 12:10 |
| *** dave-mccowan has joined #openstack-ansible | 12:11 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-galera_server stable/pike: Prevent latest mariadb version https://review.openstack.org/542083 | 12:15 |
| openstackgerrit | Taseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit https://review.openstack.org/522491 | 12:17 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-galera_server stable/pike: Prevent latest mariadb version https://review.openstack.org/542083 | 12:27 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Implement a scenario with no containers https://review.openstack.org/504224 | 12:28 |
| *** hw_wutianwei has quit IRC | 12:28 | |
| *** hw_wutianwei has joined #openstack-ansible | 12:28 | |
| odyssey4me | hwoarang if you have a minute to spare, would you mind taking a peek at https://review.openstack.org/541443 please? | 12:31 |
| odyssey4me | evrardjp mind if I tweak https://review.openstack.org/#/c/542144/1/releasenotes/notes/mariadb-10.1.31-package-issue-5889d1b7940489f1.yaml a bit? | 12:32 |
| *** SerenaFeng has joined #openstack-ansible | 12:34 | |
| odyssey4me | evrardjp also, mind if I propose a reno per branch as it will give better visiblity... this reno will only be published into queens | 12:34 |
| evrardjp | odyssey4me: I don't mind its edition, and I planned to double check per branch | 12:35 |
| evrardjp | I just wanted to wait a little for validation | 12:35 |
| odyssey4me | realistically speaking, this doesn't need to go into master - only the stable branches | 12:36 |
| odyssey4me | master is not yet released, and we'll have this fixed by release | 12:36 |
| evrardjp | odyssey4me: that's fairly true! | 12:36 |
| odyssey4me | so mind if I use this as a basis to submit reno's into each stable branch then? | 12:36 |
| *** foutatoro has joined #openstack-ansible | 12:37 | |
| odyssey4me | I'd rather you spent time figuring out the root cause and fixing it, than have you fiddle with that stuff. | 12:37 |
| *** vnogin has quit IRC | 12:39 | |
| *** vnogin has joined #openstack-ansible | 12:40 | |
| *** tux_ has joined #openstack-ansible | 12:40 | |
| hwoarang | odyssey4me: done | 12:42 |
| odyssey4me | thanks hwoarang | 12:43 |
| *** chhagarw has joined #openstack-ansible | 12:44 | |
| foutatoro | hi I run into this error http://paste.openstack.org/show/666490/ when deploying openstack (stable/ocata) | 12:46 |
| foutatoro | Somebody knows why does setup-infrastructure.yml not install rabbitmq ? | 12:46 |
| *** olivierbourdon38 has joined #openstack-ansible | 12:49 | |
| odyssey4me | foutatoro setup-infrastructure failed - you need to see why that is, without the infra in place you can't setup openstack yet | 12:52 |
| evrardjp | foutatoro: your repo failed | 12:53 |
| odyssey4me | foutatoro it looks like the repo build may have failed, so try this: openstack-ansible repo-install.yml -e repo_build_git_reclone=yes -e repo_build_wheel_rebuild=yes -e repo_build_venv_rebuild=yes | 12:53 |
| odyssey4me | once that succeeds, re-run setup0infrastructure | 12:53 |
| foutatoro | odyssey4me: Can I use the default user_variable or do I set "repo_build_pip_extra_indexes" and "uca_apt_repo_url" variable as in AIO deployment | 12:55 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Implement a scenario with no containers https://review.openstack.org/504224 | 12:57 |
| *** SerenaFeng has quit IRC | 12:57 | |
| odyssey4me | foutatoro you do not have to override any defaults, unless you're testing in a restricted environment where you cannot access public mirrors | 12:57 |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-os_swift stable/newton: Zuul: Remove project name https://review.openstack.org/542162 | 13:04 |
| hwoarang | odyssey4me: evrardjp: suse+pike is ok in https://review.openstack.org/542083 please get it in :) TIA | 13:05 |
| *** vnogin has quit IRC | 13:09 | |
| mhayden | happy thursday | 13:18 |
| mhayden | or friday eve | 13:18 |
| *** woodard_ has quit IRC | 13:20 | |
| *** woodard has joined #openstack-ansible | 13:20 | |
| mhayden | hwoarang: i gave the patch an extra +2 to show the gate that we're really serious | 13:21 |
| hwoarang | hehe | 13:21 |
| mhayden | i couldn't figure out the mariadb problem yesterday | 13:22 |
| mhayden | the receiver was all set up and ready to get data | 13:22 |
| mhayden | but the donor never figured out "hey, i'm the donor, i should send data" | 13:22 |
| mhayden | so the receiver got bored and took a walk | 13:23 |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-os_rally stable/newton: Zuul: Remove project name https://review.openstack.org/542178 | 13:24 |
| odyssey4me | I'm wondering whether there may be an issue with the xtradb version we're using, or whatever it is that facilitates the SST? | 13:26 |
| evrardjp | mhayden: it's sst | 13:29 |
| mhayden | ah okay | 13:29 |
| evrardjp | mhayden: using galera_wsrep_sst_method: rsync indeed fixes the issue, so I am digging deeper | 13:29 |
| mhayden | well at least that digging allowed me to find the issue with centos repo priorities for mariadb | 13:29 |
| mhayden | so that's a positive thing | 13:30 |
| evrardjp | as you can see https://mariadb.com/kb/en/library/mariadb-10131-changelog/ lots of changes in sst | 13:30 |
| evrardjp | yeah that's good, thanks for checking | 13:30 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-galera_server master: Set proper yum repo priority on CentOS 7 https://review.openstack.org/541924 | 13:32 |
| mhayden | evrardjp: ^^ centos fix | 13:32 |
| *** bhujay has quit IRC | 13:33 | |
| *** bhujay has joined #openstack-ansible | 13:38 | |
| mgariepy | morning | 13:39 |
| odyssey4me | mhayden if those yum repository changes aren't required any more - shouldn't we be removing the reno's ? | 13:41 |
| mhayden | possibly -- which renos? | 13:42 |
| *** hamza21 has joined #openstack-ansible | 13:42 | |
| odyssey4me | mhayden the ones we merged yesterday | 13:43 |
| odyssey4me | https://review.openstack.org/#/q/I5f1ecb71c6125c29d6487bef6c2be215c1b561c1 | 13:43 |
| odyssey4me | https://review.openstack.org/#/q/I213b66cddcad8328d7606385354a0a7194a5c569 | 13:44 |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-galera_client stable/pike: Zuul: Remove project name https://review.openstack.org/542183 | 13:44 |
| openstackgerrit | Merged openstack/openstack-ansible-galera_server stable/pike: Prevent latest mariadb version https://review.openstack.org/542083 | 13:45 |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible stable/pike: Use native packages to deploy keepalived https://review.openstack.org/542187 | 13:49 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-openstack_hosts master: Keep persistent systemd journals https://review.openstack.org/542188 | 13:52 |
| mhayden | odyssey4me: i'll go pull those out | 13:52 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-galera_client master: Remove release note for yum mirror change https://review.openstack.org/542189 | 13:53 |
| mhayden | the galera_server one didn't make it into anthing but ocata | 13:54 |
| *** bhujay has quit IRC | 13:54 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-galera_client stable/ocata: Remove release note for yum mirror change https://review.openstack.org/542190 | 13:54 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-galera_server stable/ocata: Remove release note for yum mirror change https://review.openstack.org/542191 | 13:55 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-galera_client stable/pike: Remove release note for yum mirror change https://review.openstack.org/542192 | 13:55 |
| mhayden | i think that covers it | 13:56 |
| odyssey4me | thanks mhayden | 13:57 |
| mhayden | de nada | 13:57 |
| *** vnogin has joined #openstack-ansible | 13:57 | |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible stable/ocata: Use native packages to deploy keepalived https://review.openstack.org/542193 | 13:57 |
| mgariepy | is it worth it to do the same for newton ? ^^ | 13:58 |
| odyssey4me | mgariepy given that UCA is configured on all hosts and containers, I don't really see why the keepalived role needs to do anything other than install from the available repositories | 14:00 |
| odyssey4me | so yeah, I think it may be best to implement that across all branches | 14:00 |
| openstackgerrit | Merged openstack/openstack-ansible-galera_server master: Prevent latest mariadb version https://review.openstack.org/541855 | 14:02 |
| *** vnogin has quit IRC | 14:02 | |
| *** rodolof has quit IRC | 14:03 | |
| *** rodolof has joined #openstack-ansible | 14:03 | |
| *** Mr_Smurf has quit IRC | 14:04 | |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-repo_server stable/ocata: Zuul: Remove project name https://review.openstack.org/542200 | 14:04 |
| *** hachi has joined #openstack-ansible | 14:06 | |
| mgariepy | newton needs uca for trusty. | 14:07 |
| mgariepy | isn't newton supposed to be eol ? | 14:07 |
| *** hamza21 has quit IRC | 14:08 | |
| openstackgerrit | Merged openstack/openstack-ansible-galera_client stable/pike: Prevent latest mariadb version https://review.openstack.org/542125 | 14:08 |
| openstackgerrit | Merged openstack/openstack-ansible-galera_client stable/ocata: Prevent latest mariadb version https://review.openstack.org/542127 | 14:08 |
| openstackgerrit | Merged openstack/openstack-ansible-galera_client stable/pike: Zuul: Remove project name https://review.openstack.org/542183 | 14:15 |
| evrardjp | odyssey4me: not for newton | 14:16 |
| evrardjp | woot | 14:16 |
| evrardjp | mgariepy: odyssey4me not for newton | 14:16 |
| evrardjp | I mean the keepalived thing | 14:16 |
| evrardjp | I think the version used is different, and on top of that trusty would definitely need uca | 14:17 |
| odyssey4me | mgariepy evrardjp what I meant is that UCA is already there, so the keepalived role doesn't need to add another repo | 14:17 |
| evrardjp | odyssey4me: I am not sure it's already there | 14:17 |
| evrardjp | because in Newton we don't set repos at early stages like we do in master: we were waiting for pip install (at best) | 14:17 |
| mgariepy | might or might not depending on how you deploy your stack. | 14:17 |
| odyssey4me | for queens - definitely, for earlier versions - maybe not | 14:17 |
| odyssey4me | alright, perhaps best to leave newton alone then | 14:18 |
| evrardjp | pip install doesn't run on haproxy nodes, so we're kinda in need for it | 14:18 |
| evrardjp | yeah newton should be left alone, the rest can be moved to native | 14:18 |
| mgariepy | i'll leave newton like it is now. | 14:18 |
| mgariepy | i did both pike and ocata tu use native. | 14:18 |
| odyssey4me | for ocata/pike I guess we need it to use UCA? or doesn't UCA have a newer package for it? | 14:18 |
| mgariepy | nop | 14:18 |
| odyssey4me | ah ok | 14:18 |
| mgariepy | keepalive is in uca for trusty | 14:18 |
| mgariepy | for xenial it's in archive.ubuntu.com | 14:19 |
| *** vnogin has joined #openstack-ansible | 14:21 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_rally stable/newton: Zuul: Remove project name https://review.openstack.org/542178 | 14:21 |
| openstackgerrit | Merged openstack/openstack-ansible-os_heat stable/newton: Zuul: Remove project name https://review.openstack.org/542139 | 14:22 |
| odyssey4me | mgariepy so perhaps newton should only do uca if the distro is trusty - not essential, but cleaner | 14:26 |
| openstackgerrit | Merged openstack/openstack-ansible-os_rally master: Reinitialize the virtualenv https://review.openstack.org/535822 | 14:26 |
| mgariepy | yeah but not sure what was the state of keepalived role for newton. | 14:27 |
| *** adcpdk has joined #openstack-ansible | 14:28 | |
| evrardjp | I can double check | 14:28 |
| *** esberglu has joined #openstack-ansible | 14:28 | |
| mgariepy | i'll leave newton like it is now, anyway it's not breaking anything, and it's not like anybody will do a new deploy on newton. | 14:28 |
| odyssey4me | fair enough | 14:29 |
| mgariepy | at least i hope so. | 14:29 |
| mgariepy | haha | 14:29 |
| evrardjp | different variable but it would work | 14:29 |
| evrardjp | yeah fair | 14:29 |
| mgariepy | and newton should probably be OEL.. | 14:29 |
| evrardjp | ok let me get back to the sst issue :) | 14:29 |
| evrardjp | mgariepy: yeah we can EOL now | 14:30 |
| evrardjp | I will issue last tag in the next days, then we can move to EOL | 14:30 |
| odyssey4me | evrardjp given the mariadb issues, we may still need some role SHA bumps before we issue the next tag | 14:31 |
| evrardjp | odyssey4me: yeah ofc | 14:33 |
| *** rama_y has joined #openstack-ansible | 14:33 | |
| evrardjp | the next release tag should be asked Friday 16th, so we have time | 14:33 |
| *** lbragstad has joined #openstack-ansible | 14:37 | |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Fix Gnocchi service URLs https://review.openstack.org/541946 | 14:40 |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-os_watcher stable/pike: Zuul: Remove project name https://review.openstack.org/542239 | 14:44 |
| *** hachi has quit IRC | 14:45 | |
| *** adcpdk has quit IRC | 14:51 | |
| *** jwitko_ has joined #openstack-ansible | 14:54 | |
| *** olivierbourdon38 has quit IRC | 14:55 | |
| *** SerenaFeng has joined #openstack-ansible | 15:02 | |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-lxc_hosts stable/pike: Zuul: Remove project name https://review.openstack.org/542248 | 15:04 |
| *** acormier has joined #openstack-ansible | 15:05 | |
| *** tux_ has quit IRC | 15:08 | |
| *** weezS has joined #openstack-ansible | 15:10 | |
| cloudnull | mornings | 15:12 |
| *** hamzy has quit IRC | 15:12 | |
| *** tux_ has joined #openstack-ansible | 15:12 | |
| *** sar has quit IRC | 15:15 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Add nspawn container driver https://review.openstack.org/477017 | 15:16 |
| *** tux_ has quit IRC | 15:17 | |
| *** chyka has joined #openstack-ansible | 15:20 | |
| openstackgerrit | Merged openstack/openstack-ansible-apt_package_pinning stable/pike: Zuul: Remove project name https://review.openstack.org/541995 | 15:23 |
| openstackgerrit | Merged openstack/openstack-ansible-pip_install stable/ocata: Zuul: Remove project name https://review.openstack.org/541953 | 15:23 |
| *** vnogin has quit IRC | 15:23 | |
| *** vnogin has joined #openstack-ansible | 15:25 | |
| idlemind | odyssey4me thx for the linkage on that novnc / spice console issue | 15:25 |
| idlemind | mhayden does the stig for communication session timeout also cause ssh to drop if no activity is happening? and i swear i remember in one of your preso's on youtube you mentioned a way to skip stig's by the name v-xxxx but i can't seem to find it | 15:27 |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_hosts stable/ocata: Zuul: Remove project name https://review.openstack.org/542031 | 15:28 |
| openstackgerrit | Merged openstack/openstack-ansible-os_molteniron stable/pike: Zuul: Remove project name https://review.openstack.org/541965 | 15:29 |
| *** chyka has quit IRC | 15:31 | |
| *** brad[] has joined #openstack-ansible | 15:34 | |
| openstackgerrit | Merged openstack/openstack-ansible-rsyslog_client stable/newton: Zuul: Remove project name https://review.openstack.org/542145 | 15:34 |
| *** SerenaFeng has quit IRC | 15:38 | |
| odyssey4me | idlemind find the tag for the task, then use --skip-tags <stig ref/tag number> | 15:39 |
| idlemind | so say in openstack-ansible setup-everything ... add --skip-tags <stig> | 15:39 |
| openstackgerrit | Merged openstack/openstack-ansible-ops stable/newton: Zuul: Remove project name https://review.openstack.org/542096 | 15:40 |
| idlemind | no way to skip the stig in my user_config or user_variables | 15:40 |
| odyssey4me | idlemind some of the items have config options - check the security role docs | 15:44 |
| idlemind | ya i've been digging around in them found a few w/bools to skip but few and far between | 15:44 |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-os_barbican stable/ocata: Zuul: Remove project name https://review.openstack.org/542269 | 15:44 |
| cloudnull | anything I can help with regard to our mariadb issues ? | 15:45 |
| odyssey4me | idlemind https://docs.openstack.org/ansible-hardening/latest/ | 15:45 |
| idlemind | thanks might be better on that page than just browsing the github repo lol | 15:46 |
| *** hamzy has joined #openstack-ansible | 15:46 | |
| odyssey4me | cloudnull I think that evrardjp has isolated it down to the extrabackup SST | 15:46 |
| cloudnull | just a change in the xtrabackup that's broken or something we need to update in our setup? | 15:47 |
| *** rama_y has quit IRC | 15:47 | |
| *** hamzy_ has joined #openstack-ansible | 15:50 | |
| *** Ritesh has joined #openstack-ansible | 15:51 | |
| *** hamzy has quit IRC | 15:52 | |
| mhayden | idlemind: holler if you're stuck -- i'm the usual maintainer of that role :P | 15:53 |
| idlemind | mhayden yar i'm looking at killing some more of the ssh stuff, in particular the getting kicked out every 10/15 mins. it's my lab so i'm not heartbroken if it's not 100% stig compliant. overall love the added security items | 15:54 |
| idlemind | i'm starting with security_sshd_client_alive_count_max back to 3 from 0 | 15:54 |
| mhayden | that helps | 15:54 |
| *** SerenaFeng has joined #openstack-ansible | 15:54 | |
| mhayden | also look for that other one that sets TMOUT in /etc/profile | 15:54 |
| mhayden | that one will cut off ssh and local shells | 15:55 |
| idlemind | k is that security_rhel7_session_timeout | 15:55 |
| mhayden | indeed | 15:55 |
| mhayden | and yes, both of those can be super annoying | 15:55 |
| idlemind | can i skip that one or does a value of 0 make it infinite? | 15:55 |
| *** kstev has joined #openstack-ansible | 15:55 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_glance stable/pike: Zuul: Remove project name https://review.openstack.org/542005 | 15:55 |
| mhayden | i think i set it to 0 | 15:55 |
| mhayden | yeah, i did | 15:55 |
| idlemind | k i'll start there thx | 15:55 |
| mhayden | another option is to start a tmux session when you connect | 15:55 |
| mhayden | tmux keeps it alive somehow | 15:55 |
| idlemind | tmux? | 15:55 |
| mhayden | yeah, tmux is a terminal muxer, like screen | 15:56 |
| idlemind | ah | 15:56 |
| idlemind | so it has hax | 15:56 |
| idlemind | i like it | 15:56 |
| *** tux_ has joined #openstack-ansible | 15:56 | |
| mhayden | ssh to a machine, type "tmux" and you're good to go | 15:56 |
| mhayden | CTRL-b and then d to detach | 15:56 |
| mhayden | "tmux a" to reattach | 15:56 |
| mhayden | (of course tmux does a ton more than that) | 15:56 |
| mhayden | but that should get you started | 15:56 |
| idlemind | cool i'll give that a look too | 15:56 |
| mhayden | in texas, we would say it's handier than hip pockets on a hog | 15:58 |
| idlemind | lol oh texas | 15:58 |
| idlemind | brb have to go eat more cheese | 15:58 |
| *** weezS has quit IRC | 15:59 | |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts stable/pike: Zuul: Remove project name https://review.openstack.org/542248 | 16:01 |
| evrardjp | mhayden: hahah | 16:01 |
| idlemind | btw added a second compute node to my little lab cloud, man dat was easy, 2 lines in my change controlled user_config and fire off a command to bring a kickstarted centos box into the fold so thx all | 16:01 |
| evrardjp | hip pockets on a hog | 16:02 |
| *** pcaruana has quit IRC | 16:02 | |
| mnaser | https://review.openstack.org/#/c/541924 failed for a small typo | 16:03 |
| mnaser | mhayden: mind having a look for a quick revision to wait for check to go through again? | 16:03 |
| evrardjp | great I just lost access to my machine doing a sst with xtrabackup. | 16:06 |
| evrardjp | my whole machine froze. | 16:06 |
| evrardjp | ?! | 16:06 |
| *** adcpdk has joined #openstack-ansible | 16:06 | |
| *** vnogin has quit IRC | 16:08 | |
| idlemind | ugh oh | 16:09 |
| idlemind | see just proof we should never backup | 16:09 |
| idlemind | overrated | 16:09 |
| idlemind | (jk) | 16:09 |
| adcpdk | Hi everyone! Hi idlemind! yesterday I was able to figure out with veth12 interface configuration, so now I can start the instances up. However I have a few problems still here and there. My biggest issue now is that even though I create flat public network and attach a router to it, I cannot ping the router from the external network, only form the internal. | 16:09 |
| *** SerenaFeng has quit IRC | 16:09 | |
| idlemind | adcpdk are you dim__ from yesterday? | 16:09 |
| *** udesale has quit IRC | 16:10 | |
| adcpdk | yes | 16:10 |
| adcpdk | dim is easier | 16:10 |
| idlemind | k i only support ipv6 external networks (jk) | 16:10 |
| idlemind | so your new flat network, i assume you have a layer 3 device (router) on the real-life physical side right? | 16:10 |
| adcpdk | well, what I call external is not actually external here, it's the same network I use for br-mgmt, I just want to be able to login to those vms using ssh (Security rules, like firewall is configured) | 16:11 |
| adcpdk | e.g. I use bridge br-mgmt binded with vlan30 enps0f0.30 | 16:12 |
| openstackgerrit | Merged openstack/openstack-ansible-os_ironic stable/pike: Zuul: Remove project name https://review.openstack.org/542029 | 16:12 |
| idlemind | adcpdk when you create your external network you say something for "net-name" if i recall yesterday we commented out the "flat" section or did you get that working with another physical or vlan interface | 16:13 |
| adcpdk | I have played with my variablemany times, no luck. Here is what I had: ```host_bind_override: "br-mgmt"```, ```host_bind_override: "enps0f0.30"```, ```host_bind_override: "eth12"``` | 16:14 |
| adcpdk | Yesterday it worked for internal network only | 16:14 |
| adcpdk | vms were able to communicate with the router, I found out then that personally I could not ping the router from the external network it must be attached to | 16:15 |
| adcpdk | ext-net--br-mgmt--R1--internal_net--VM1 | 16:16 |
| adcpdk | so I could not ping the router from the ext-net (aka br-mgmt) | 16:16 |
| openstackgerrit | Merged openstack/openstack-ansible-os_watcher stable/pike: Zuul: Remove project name https://review.openstack.org/542239 | 16:16 |
| *** sar has joined #openstack-ansible | 16:17 | |
| adcpdk | I am still learning even how to use IRC | 16:17 |
| adcpdk | changed name to dim as it's easier to type | 16:17 |
| *** armaan has quit IRC | 16:18 | |
| *** armaan has joined #openstack-ansible | 16:18 | |
| *** woodard has quit IRC | 16:20 | |
| idlemind | adcpdk i think if you wanted to use br-mgmt you'd have to do some snat'ing from a loopback (dummy) interface that you actually use for your "flat" network that gets tied into OpenStack. It's probably easier at this point to look at a different approach. The Br-VLAN in the default config lends itself to becoming a qinq type of interface. That's what I use for my ext-net(s) in my lab right now but it may be difficult to | 16:20 |
| idlemind | set that up on the outside side unless you're already familair with how to handle qinq stuff. | 16:20 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova master: If debug is enabled, provide verbose output for host discovery https://review.openstack.org/541443 | 16:21 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/pike: If debug is enabled, provide verbose output for host discovery https://review.openstack.org/542287 | 16:21 |
| idlemind | adcpdk what you want to do is pick a vlan in your physical environment (or create a new one) that isn't used for anything else right now (ignore trying to re-use br-mgmt) | 16:21 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/ocata: If debug is enabled, provide verbose output for host discovery https://review.openstack.org/542288 | 16:21 |
| *** chhagarw has quit IRC | 16:21 | |
| idlemind | adcpdk then trunk that to your compute and neutron hosts (network_hosts). on those hosts adjust the network configuration to create a vlan interface on enps0f0 and use that in the host_bind_override and uncomment the "flat" network in user_config.yml | 16:22 |
| idlemind | adcpdk then when you create your "ext-net" in horizon (or openstack cli) specify the network name as "flat" as it is in user_config.yml by default. then set the gateway to the layer 3 device on the real lan (layer 3 switch or router) | 16:23 |
| idlemind | adcpdk i know it's a lot of text i'll see if i can make a dirty visio | 16:23 |
| *** TxGirlGeek has joined #openstack-ansible | 16:27 | |
| *** vnogin has joined #openstack-ansible | 16:31 | |
| openstackgerrit | Taseer Ahmed proposed openstack/openstack-ansible-os_congress master: Introduce os_congress role in gerrit https://review.openstack.org/522491 | 16:35 |
| *** vnogin has quit IRC | 16:35 | |
| *** vnogin has joined #openstack-ansible | 16:36 | |
| *** taseer2 is now known as Taseer | 16:37 | |
| *** sar has quit IRC | 16:38 | |
| *** sar has joined #openstack-ansible | 16:38 | |
| idlemind | adcpdk https://imgur.com/a/on1eI | 16:39 |
| idlemind | let me know if that doesn't end up just confusing it more | 16:39 |
| openstackgerrit | Merged openstack/openstack-ansible stable/ocata: Zuul: Remove project name https://review.openstack.org/542027 | 16:42 |
| evrardjp | mhayden: the bug is tracked upstream now: https://jira.mariadb.org/browse/MDEV-15254 | 16:43 |
| evrardjp | the description is incorrect though, the joining is fine, as long as you don't have to do a sst | 16:44 |
| evrardjp | everything fine until you need to transfer a byte | 16:44 |
| *** eumel8 has quit IRC | 16:44 | |
| evrardjp | so I will dig deeper, I am sure it's the script that changed that is causing the issue: variable wrongly renamed or something | 16:44 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_tempest master: Properly check for an existing log directory https://review.openstack.org/542307 | 16:45 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Implement a scenario with no containers https://review.openstack.org/504224 | 16:45 |
| *** dariko has joined #openstack-ansible | 16:46 | |
| *** armaan has quit IRC | 16:47 | |
| *** armaan has joined #openstack-ansible | 16:47 | |
| nurdie | how do i load /etc/nova/policy.json changes | 16:47 |
| *** rama_y has joined #openstack-ansible | 16:47 | |
| *** Ritesh has quit IRC | 16:47 | |
| *** openstackgerrit has quit IRC | 16:48 | |
| adcpdk | thank you idlemind! This is amazing, I got your point and created br33 binded to vlan33 enps0f0.33 on all three nodes and deployment host just in case | 16:53 |
| idlemind | adcpdk it needs to be the vlan interface, enps0f0.33 not a bridge on the actual openstack nodes k | 16:54 |
| adcpdk | oh I see | 16:54 |
| adcpdk | I will try and let you know | 16:54 |
| adcpdk | btw you shouldn't stay hungry longer 711 rules | 16:54 |
| idlemind | ya something about bridges can't be added to bridges and behind the scenes openstack creates a bridge for the "flat" network and addes that "host_bind_interface" to the newly created bridge | 16:55 |
| idlemind | rando thought does review.openstack.org have an api that can be queried by scrubs like me | 16:56 |
| adcpdk | I will let you know once I re-ran that os-neutron ansible playbook | 16:56 |
| idlemind | adcpdk cool cool good luck! | 16:57 |
| *** weezS has joined #openstack-ansible | 16:58 | |
| *** TxGirlGeek has quit IRC | 17:00 | |
| *** TxGirlGeek has joined #openstack-ansible | 17:01 | |
| adcpdk | idlemind, so this should work now? | 17:01 |
| adcpdk | ``` | 17:01 |
| adcpdk | - network: | 17:01 |
| adcpdk | container_bridge: "br-vlan" | 17:01 |
| adcpdk | container_type: "veth" | 17:01 |
| adcpdk | container_interface: "eth12" | 17:01 |
| adcpdk | host_bind_override: "enp1s0f0.33" | 17:01 |
| adcpdk | type: "flat" | 17:01 |
| adcpdk | net_name: "flat" | 17:01 |
| adcpdk | group_binds: | 17:01 |
| adcpdk | - neutron_linuxbridge_agent | 17:01 |
| adcpdk | ``` | 17:01 |
| adcpdk | https://pastebin.com/mC6NgRyX | 17:02 |
| adcpdk | sorry for pasting this all above | 17:02 |
| idlemind | yup as far as i know, one of the experts could explain the logic behind container_bridge: "br-vlan" I haven't figured out why that's there in that fashion yet | 17:02 |
| odyssey4me | adcpdk I would suggest that you use a paste service or gist to communicate multi-line things, otherwise you might end up being kicked for flooding by a bot or something | 17:02 |
| adcpdk | odyssey4me sorry, I though my agent will transfer my multiline, but it didn't. I used pastebin instead, my apologies | 17:03 |
| odyssey4me | no worries :) | 17:03 |
| odyssey4me | always good to see people helping each other out :) | 17:04 |
| dcdamien | Hi! Does anyone have an idea why this -> http://paste.openstack.org/show/666731/ in /etc/openstack_deploy/env.d/santracity-proxy.yml and this -> http://paste.openstack.org/show/666735/ in /etc/openstack_deploy/conf.d/santracity-proxy.yml doesn't make such containers when running lxc-containers-create.yml playbook? | 17:04 |
| idlemind | odyssey4me do you know what container_bridge: "br-vlan" is used for in the flat network? is it just because we don't know the name of the bridge neutron will create? | 17:04 |
| idlemind | or is br-vlan (native) also supposed to map to the vlan being presented via host_bind_override? | 17:05 |
| odyssey4me | I'm afraid that I'm not well versed in the voodoo of networking configs in OSA. :) | 17:06 |
| odyssey4me | cloudnull may be able to help there, and some others know their way around too | 17:06 |
| idlemind | thx | 17:08 |
| odyssey4me | dcdamien the physical skep items need totally different names, and those are the ones you need in your conf.d file | 17:08 |
| odyssey4me | dcdamien see https://github.com/openstack/openstack-ansible/blob/master/inventory/env.d/aodh.yml as an example | 17:09 |
| idlemind | i have a sneaky suspicion the native vlan of br-vlan has to match the host_bind_override to make it all work | 17:09 |
| odyssey4me | idlemind yes, IIRC that is correct | 17:09 |
| odyssey4me | there's a ton of stuff in https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/openstack_user_config.yml.example which explains all that, along with a networking appendix | 17:09 |
| odyssey4me | https://docs.openstack.org/project-deploy-guide/openstack-ansible/latest/overview-network-arch.html | 17:10 |
| odyssey4me | https://docs.openstack.org/project-deploy-guide/openstack-ansible/latest/app-networking.html | 17:10 |
| idlemind | k adcpdk we'll have to rethink your approach a little to get to a functional flat then | 17:10 |
| idlemind | ah ok, i see how this is supposed to work in the aio ... the bootsrap is supposed to create a veth called eth12 on the host that plugs into br-vlan and that can be referenced in the openstack_user_config as eth12 but because it's done by bootstrap-aio.sh in adcpdk's case that eth12 (veth) never gets created | 17:14 |
| *** gkadam has quit IRC | 17:15 | |
| odyssey4me | dcdamien I think what you're hitting is https://bugs.launchpad.net/openstack-ansible/+bug/1623584 | 17:16 |
| openstack | Launchpad bug 1623584 in openstack-ansible "Dynamic inventory clash between host group name and container group name" [Low,Confirmed] - Assigned to Nolan Brubaker (nolan-brubaker) | 17:16 |
| adcpdk | idlemind thanks for looking in to that for me | 17:17 |
| idlemind | np i needed to figure it out too | 17:17 |
| idlemind | just been putting it off | 17:17 |
| *** vnogin has quit IRC | 17:17 | |
| adcpdk | AIO worked fine for me, I only started having problems deploying testing environment from the scratch: https://docs.openstack.org/project-deploy-guide/openstack-ansible/draft/app-config-test.html#introduction | 17:18 |
| adcpdk | I will be in 30 minutes, will need to grab my lunch | 17:18 |
| idlemind | ya cuz the aio scripting creates the eth12 veth interface on br-vlan | 17:18 |
| idlemind | so when you created the nodes in a more traditional fashion you have to create the eth12 veth on your own | 17:19 |
| *** peri has quit IRC | 17:20 | |
| idlemind | is bootstrap-host/tasks/prepare_networking.yml meant to be run to configure host networking or is it assumed we do that via other means (manual, our own playbooks)? | 17:22 |
| *** openstackgerrit has joined #openstack-ansible | 17:25 | |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-ceph_client stable/pike: Zuul: Remove project name https://review.openstack.org/542325 | 17:25 |
| evrardjp | nurdie: you can override nova policy with what you want, see also our config_template overrides in the documentation | 17:27 |
| dcdamien | odyssey4me: thanks, this is it | 17:29 |
| *** TxGirlGeek has quit IRC | 17:32 | |
| *** TxGirlGeek has joined #openstack-ansible | 17:32 | |
| *** sar has quit IRC | 17:34 | |
| Guy | :q! | 17:35 |
| evrardjp | :w! | 17:35 |
| Guy | lol | 17:35 |
| evrardjp | You can't leave us! | 17:35 |
| evrardjp | that's not nice! | 17:35 |
| evrardjp | :D | 17:35 |
| Guy | wouldnt dream of it! | 17:36 |
| mhayden | ESC ZZ | 17:36 |
| Guy | dont suppose anyone has a sample haproxy.cfg file | 17:36 |
| evrardjp | mhayden: YES | 17:36 |
| Guy | I cant get the backend up even though the port is listening | 17:36 |
| evrardjp | That's what I do! | 17:36 |
| Guy | sweet! | 17:37 |
| mhayden | i used to be a CTRL-X y kinda guy | 17:37 |
| mhayden | in nanoland | 17:37 |
| evrardjp | f3 f2 | 17:37 |
| evrardjp | but yeah ZZ is the deal | 17:37 |
| cloudnull | im all about that meta asdhklasdlkjhasdlkjhasdkljhasdlkjhaashjdkjalhdasklj kljha skldjh aslkjdhas lkdjh to exit my editor | 17:37 |
| evrardjp | cloudnull: I always knew you loved emacs. | 17:37 |
| cloudnull | hahahaha | 17:37 |
| evrardjp | it was well hidden, you're trying to put that away. But it's like stella, and budweiser, always in your heart at home. | 17:38 |
| evrardjp | :p | 17:38 |
| evrardjp | hahahaha | 17:38 |
| evrardjp | Guy: what's your issue with haproxy? | 17:38 |
| Guy | well... | 17:38 |
| Guy | https://www.irccloud.com/pastebin/dIni5n8J/ | 17:39 |
| evrardjp | well it doesn't look like an haproxy issue | 17:40 |
| evrardjp | what about your galera cluster? | 17:40 |
| Guy | I can connect to the host with mysql -u root -p -h | 17:40 |
| evrardjp | is the little boy happy ? | 17:40 |
| Guy | galera is cool | 17:40 |
| evrardjp | show status; ? | 17:40 |
| Guy | I can get to the database | 17:40 |
| evrardjp | to see if it cluster fines? | 17:40 |
| evrardjp | if it clusters correctly? | 17:40 |
| evrardjp | darn that's a difficult sentence | 17:41 |
| Guy | :) | 17:41 |
| evrardjp | also I don't know what you have configured for networking | 17:41 |
| Guy | show status returns sycned and operationsal | 17:42 |
| Guy | um | 17:42 |
| evrardjp | but if using /24, 172.29.238.236 and 172.29.236.13 aren't on the same net. Not sure which one does what and if they have access to other things... I am just stabbing in the dark there. | 17:42 |
| evrardjp | Guy: interesting | 17:42 |
| evrardjp | so it's behaving fine, showing a proper cluster | 17:42 |
| Guy | yeah | 17:43 |
| evrardjp | (I guess you checked on all ) | 17:43 |
| *** woodard has joined #openstack-ansible | 17:43 | |
| Guy | its a /22 | 17:43 |
| Guy | yes sir, checked all that | 17:43 |
| Guy | I think its the ha backend | 17:43 |
| cloudnull | evrardjp: lolol. I did buy a stella challis in your honor http://www.stellaartois.com/en_us/water.html | 17:43 |
| Guy | I can connect directly to the cluster from keystone to galera | 17:43 |
| evrardjp | what is that cloudnull ? | 17:44 |
| cloudnull | stella, the best beer from belgium is changing the lives of folks around the world | 17:44 |
| evrardjp | we are good people! | 17:45 |
| cloudnull | hahahaha | 17:45 |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-os_ceilometer stable/newton: Zuul: Remove project name https://review.openstack.org/542348 | 17:45 |
| evrardjp | stella exports for people that don't know beer, and it works | 17:45 |
| *** vnogin has joined #openstack-ansible | 17:45 | |
| evrardjp | Ask the uk folks. | 17:45 |
| evrardjp | :p | 17:45 |
| *** gillesMo has quit IRC | 17:45 | |
| evrardjp | ok I have to go... dinner time! | 17:46 |
| evrardjp | will continue on the galera thing tomorrow | 17:46 |
| Guy | enjoy! | 17:46 |
| cloudnull | take care evrardjp | 17:46 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-os_nova master: Zuul: Remove project name https://review.openstack.org/541117 | 17:46 |
| evrardjp | for now, the only thing I have is that: https://jira.mariadb.org/browse/MDEV-15254 which is exactly the same problem. I'll be checking socat, and the script itself tomorrow! | 17:47 |
| idlemind | cloudnull evrardjp the tests directory has some tasks for setting up host networking i imagine this is optimized for our automated test suites and shouldn't be re-used; instead we should create and manage our own methods for getting the network setup prior to starting a deploy (per docs) correct? | 17:48 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-galera_server master: Set proper yum repo priority on CentOS 7 https://review.openstack.org/541924 | 17:48 |
| idlemind | in particular it gives us the commands needed to get the veth pair up and rolling for br-vlan which would be needed on our compute_host(s) and network_host(s) right? | 17:49 |
| *** vnogin has quit IRC | 17:50 | |
| odyssey4me | idlemind yep | 17:50 |
| odyssey4me | many have tried to create a generic thing to build the networking bit, and many have failed - there are just too many possibilities | 17:51 |
| idlemind | ya i'd imagine | 17:51 |
| idlemind | i'm still struggling with how i would want to define mine in code | 17:51 |
| odyssey4me | that said, with modern ansible it should be a *lot* easier | 17:51 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-tests master: Collect files from /etc/yum.repos.d/ https://review.openstack.org/542349 | 17:51 |
| odyssey4me | and there's nothing that says we couldn't have a set of supported use-cases, and if that doesn't do what you need then you don't have to use them | 17:51 |
| idlemind | i'd like to stick w/network manager based commands and configs but i'm thinking the buck kinda stops at veths ... | 17:52 |
| *** pbandark has joined #openstack-ansible | 17:55 | |
| *** sar has joined #openstack-ansible | 17:55 | |
| mhayden | just found that centos isn't using openstack mirrors for RDO/openstack-pike :| | 17:57 |
| mhayden | the pip_install roles checks to see if they're present and bails if they are | 17:57 |
| idlemind | orly | 17:57 |
| idlemind | so what version am i running? | 17:57 |
| *** TxGirlGeek has quit IRC | 18:01 | |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-os_zaqar stable/ocata: Zuul: Remove project name https://review.openstack.org/542351 | 18:05 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-openstack_hosts master: Ensure openstack_hosts sets nodepool variables https://review.openstack.org/542352 | 18:06 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Add nspawn container driver https://review.openstack.org/477017 | 18:10 |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_hosts master: Keep persistent systemd journals https://review.openstack.org/542188 | 18:11 |
| adcpdk | idlemind my veth12 is running fine on node02 (compute) | 18:15 |
| adcpdk | I just needed to pre-up it | 18:15 |
| idlemind | adcpdk neat so then you could move to using eth12 in your host_bind_override again | 18:15 |
| idlemind | was that your original aio node? or did you have to do the creation stuff for it | 18:16 |
| *** hw_wutianwei has quit IRC | 18:25 | |
| *** hw_wutianwei has joined #openstack-ansible | 18:25 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_barbican stable/ocata: Zuul: Remove project name https://review.openstack.org/542269 | 18:25 |
| idlemind | was that your original aio node? or did you have to do the creation stuff for it | 18:25 |
| idlemind | grr copy and paste | 18:26 |
| idlemind | adcpdk the last thing, i had some trouble w/ssl based connections in ipv4 w/my external networks (or more likely my internal networks) not cleanly handling fragmentation. you might need or want to set the mtu of your ext-net to 1450 to accomodate for vxlan dropping 50 bytes off of the stock mtu not sure if any of the pro's see that in their deploys as well | 18:26 |
| idlemind | mhayden something work putting in an appendix for centos duders that might be following the tutorials and get tripped up by br-vlan for flat and vlan and not having an eth12 present? ... http://paste.openstack.org/show/666775/ | 18:27 |
| idlemind | * something worth | 18:27 |
| mhayden | oh, for an aio? | 18:27 |
| idlemind | even a multi-node they all need an eth12 to map to the native vlan of br-vlan | 18:28 |
| mhayden | i remember submitting this quandary to cloudnull about 2.5 years ago :) | 18:28 |
| mhayden | when i was learning the ropes | 18:28 |
| idlemind | ya because most ppl setup a bond or nested bridges with vlans like i did and don't notice or understand the veth: eth12 dealo in the tuts | 18:29 |
| idlemind | (like me) | 18:29 |
| idlemind | i more or less fished this from the tests that creates it for aio and mutli-node and translated the ansible back into a chunk that could be pasted at the ole terminal | 18:29 |
| *** TxGirlGeek has joined #openstack-ansible | 18:30 | |
| idlemind | i personally am using separate ansible to prep my hosts after kickstart but this could be useful none the less | 18:30 |
| odyssey4me | cloudnull mhayden got a minute to peek at https://review.openstack.org/542307 ? | 18:31 |
| idlemind | but might just be nice for those reading along at home to see a link or note box about what the heck is happening with veth and eth12 and here's a script to get the veth running on your centos box | 18:31 |
| openstackgerrit | Merged openstack/openstack-ansible-os_swift stable/newton: Zuul: Remove project name https://review.openstack.org/542162 | 18:31 |
| openstackgerrit | Merged openstack/openstack-ansible-ceph_client stable/pike: Zuul: Remove project name https://review.openstack.org/542325 | 18:31 |
| mhayden | odyssey4me: got it | 18:32 |
| odyssey4me | ta | 18:32 |
| odyssey4me | cloudnull more deletions if you'd like :) https://review.openstack.org/#/q/topic:bug/1670419+(status:open+OR+status:merged) | 18:32 |
| odyssey4me | mhayden cloudnull that os_tempest patch is the last thing standing in the way of https://review.openstack.org/504224 :) | 18:34 |
| odyssey4me | (other than reviews, of course) | 18:34 |
| cloudnull | nice ! | 18:35 |
| odyssey4me | I'm tempted to actually make that aio_metal scenario voting ;) | 18:36 |
| cloudnull | I actually think we should make it experimental for now. | 18:36 |
| odyssey4me | it's passing. and does so in under an hour ;) | 18:36 |
| odyssey4me | I'm tempted to add another test on top - an upgrade test using no containers. :) | 18:37 |
| cloudnull | I mean octavia is passing most of the time, but that's still experimental. | 18:37 |
| odyssey4me | that's only 'cos no-one has made it not experimental | 18:37 |
| cloudnull | i have no strong opinions about it but it's a new test case | 18:38 |
| odyssey4me | I gave up asking when that's going to happen some time ago. | 18:38 |
| cloudnull | so long as it's not voting i think it's ok for now | 18:38 |
| odyssey4me | sure | 18:38 |
| *** mbuil has quit IRC | 18:43 | |
| *** TxGirlGeek has quit IRC | 18:43 | |
| *** stuartgr has quit IRC | 18:46 | |
| *** armaan has quit IRC | 18:47 | |
| adcpdk | idlemind I think I need to run infrustructure playbook now | 18:48 |
| adcpdk | my mysql is not working))) | 18:48 |
| idlemind | adcpdk :( that's never fun | 18:48 |
| idlemind | if you're running 3 mysql containers (like the aio - i think) on 1 host you have to do some magic after a host reboot to get galera back and going | 18:49 |
| *** vnogin has joined #openstack-ansible | 18:49 | |
| adcpdk | From what I understand now is that either I can specify enps0f0.30 in host_bind_override OR I can still have eth12 on it but instead play with /etc/network/interfaces spcifically the eth12 (with pre-ups) section on compute node ```bridge_ports enp1s0f0.33 br-vlan-veth``` | 18:50 |
| adcpdk | I know what you are talking about, however I am not running AIO, thus the problem does not exist, I screwed it up by changing something in nova playbook | 18:51 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_aodh master: Update the Tempest plugin to point towards the new plugin https://review.openstack.org/541270 | 18:52 |
| *** TxGirlGeek has joined #openstack-ansible | 18:56 | |
| *** kstev has quit IRC | 18:57 | |
| *** poopcat has joined #openstack-ansible | 18:57 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_nova stable/ocata: If debug is enabled, provide verbose output for host discovery https://review.openstack.org/542288 | 18:57 |
| *** vnogin has quit IRC | 18:58 | |
| idlemind | adcpdk now that i understand more completely what container-bridge br-vlan and the host-override veth are doing ... i'd recommend for your build that you go with the veth and end up w/a eth12 on each compute node (via pre-up / post-down scripts) if you look at the tests for bootstrap-aio.yml in openstack-ansible you'll see how to do it for all platforms (ubuntu, suse and centos) i wrote how i just did it in a bash | 19:00 |
| idlemind | script in a previous paste for my 2 centos compute nodes | 19:00 |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: Use native packages to deploy keepalived https://review.openstack.org/542187 | 19:02 |
| openstackgerrit | Merged openstack/openstack-ansible stable/ocata: Use native packages to deploy keepalived https://review.openstack.org/542193 | 19:02 |
| openstackgerrit | Merged openstack/openstack-ansible-repo_server stable/ocata: Zuul: Remove project name https://review.openstack.org/542200 | 19:03 |
| spotz | evrardjp: for some reason I thought you were pinging this morning so I didn't ping | 19:05 |
| *** hamza21 has joined #openstack-ansible | 19:10 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_zaqar stable/ocata: Zuul: Remove project name https://review.openstack.org/542351 | 19:12 |
| *** kstev has joined #openstack-ansible | 19:13 | |
| adcpdk | idlemind could share what you did in bash? | 19:20 |
| idlemind | This would be for CentOS http://paste.openstack.org/show/666775/ | 19:20 |
| adcpdk | so this one would work then? | 19:22 |
| adcpdk | http://paste.openstack.org/show/666802/ | 19:22 |
| idlemind | i'm no ubuntu pro but that looks right | 19:23 |
| idlemind | you can verify locally w/tools like brctl show br-vlan after a reboot | 19:24 |
| idlemind | (or start and stops of networking service) | 19:24 |
| adcpdk | so in this case whatever you specify in ``` bridge_ports enp1s0f0.33 br-vlan-veth``` will be your flat interface I assume, I don't understand master convention you used in ```ip link set br-vlan-veth master br-vlan | 19:26 |
| adcpdk | ``` | 19:26 |
| idlemind | br-vlan-veth pairs w/eth12 which is referenced in host_bind_interface_override in user_config.yml | 19:27 |
| adcpdk | i see | 19:27 |
| adcpdk | I will let you know of my results | 19:27 |
| adcpdk | did it work for you? | 19:27 |
| idlemind | haven't rebuilt neutron yet | 19:28 |
| idlemind | myself | 19:28 |
| idlemind | https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/openstack_user_config.yml.example#L300 | 19:28 |
| idlemind | see that line in openstack_user_config.yml? | 19:28 |
| adcpdk | yes | 19:29 |
| idlemind | that's the name you use for the "interface" when you create your external network in horizon (or cli) | 19:29 |
| adcpdk | that I understan | 19:29 |
| idlemind | k | 19:29 |
| adcpdk | understand* | 19:29 |
| idlemind | good deal | 19:29 |
| idlemind | i'm going to step away for a minute too | 19:29 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova master: Remove placement_database related options/code https://review.openstack.org/541685 | 19:29 |
| adcpdk | thank you! | 19:29 |
| *** lbragstad has quit IRC | 19:31 | |
| *** lbragstad has joined #openstack-ansible | 19:32 | |
| foutatoro | Hi, I deployed openstack in 2 nodes with LVM as cinder's backend but when I run the command "lvs" the cinder-volume doesn't appear in storage_node | 19:36 |
| foutatoro | here is the error http://paste.openstack.org/show/666809/ | 19:36 |
| foutatoro | did somebody get that before ? | 19:36 |
| idlemind | foutatoro i create the cinder-volume before | 19:37 |
| idlemind | for me i name it "cinder-volumes" though (just make sure it all matches up to your user_config.yml | 19:37 |
| idlemind | and it would be a vg | 19:38 |
| adcpdk | I had a similar problem 2 days ago, my VG called node03-vg it's totally unacceptable, it needs to be called "cinder-volumes" or it needs to be changed in user_config | 19:38 |
| idlemind | so vgs | 19:38 |
| adcpdk | foutatoro please don't follow my mistakes and don't try to rename root VG with vgrename | 19:39 |
| foutatoro | idlemind: yes the name in openstack_user_conf is "cinder-volumes" | 19:39 |
| adcpdk | try add another drive to the system as a physical volume and create VG using it | 19:39 |
| idlemind | foutatoro so create your vg and name it cinder-volumes (i do all of that via kickstart in centos but you can do it at the command-line after the fact) | 19:40 |
| adcpdk | with the name referenced above | 19:40 |
| idlemind | you can use a plain file on existing storage w/the loop device if you're just trying to get it working in a pinch (it'll be mucho slow) | 19:42 |
| idlemind | http://www.anthonyldechiaro.com/blog/2010/12/19/lvm-loopback-how-to/ | 19:42 |
| foutatoro | adcpdk adcpdk : I think we can not create the volume the is not associated with an existing physical volume ? | 19:42 |
| idlemind | if you have space on an existing pv you can add a vg to it | 19:43 |
| idlemind | (i think) | 19:43 |
| adcpdk | say you have /dev/sda (hard drive1) and you create a physical volume of of that. Then you can create a VG e.g. root-vg, or cinder-volumes. | 19:43 |
| mnaser | is anyone working on figuring out why database access seems to be broken on centos 7 in functional tests | 19:44 |
| adcpdk | The problem I had was that my system was already installed with VG name other than cinder-volumes and I had to add another physical drive to the server, create PV out of it and create additional VG using that drive, which I called cinder-volumes | 19:44 |
| mnaser | depending on https://review.openstack.org/#/c/541924/ still doesn't work unfortunately | 19:45 |
| mnaser | fatal: [infra1]: FAILED! => {"changed": true, "cmd": ["/openstack/venvs/keystone-testing/bin/keystone-manage", "db_sync", "--check"], "delta": "0:01:41.815048", "end": "2018-02-08 19:41:51.952171", "failed_when_result": true, "msg": "non-zero return code", "rc": 1, "start": "2018-02-08 19:40:10.137123", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} | 19:45 |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-os_keystone stable/ocata: Zuul: Remove project name https://review.openstack.org/542383 | 19:45 |
| mnaser | hmm ok this is interesting | 19:45 |
| mnaser | http://logs.openstack.org/70/541270/11/check/openstack-ansible-functional-centos-7/f0b3e0c/logs/openstack/infra1/keystone/keystone.log.txt.gz#_2018-02-08_19_41_51_848 | 19:46 |
| mnaser | http://logs.openstack.org/70/541270/11/check/openstack-ansible-functional-centos-7/f0b3e0c/logs/openstack/infra1/mysql_logs/galera_server_error.log.txt.gz | 19:46 |
| mnaser | the server is literally shut down? | 19:46 |
| mnaser | http://logs.openstack.org/70/541270/11/check/openstack-ansible-functional-centos-7/f0b3e0c/logs/redhat-rpm-list-installed-infra1.txt.txt.gz | 19:47 |
| mnaser | mariadb-common 10.1.30-1.el7.centos combined with 10.1.31-1.el7.centos .. that cannot be good | 19:47 |
| idlemind | side note about speed of centos gates, should we disable fastestmirror to ensure the same mirror gets hit all the time and reduce cache hits after the first? | 19:50 |
| idlemind | or does apt-cacher filter some of the mirror variances out | 19:50 |
| openstackgerrit | Merged openstack/openstack-ansible-os_tempest master: Properly check for an existing log directory https://review.openstack.org/542307 | 19:51 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-galera_client master: Prevent latest mariadb version https://review.openstack.org/542119 | 19:51 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_aodh master: Update the Tempest plugin to point towards the new plugin https://review.openstack.org/541270 | 19:52 |
| mnaser | evrardjp: fyi i made a fix in your change, incorrect variable | 19:52 |
| mnaser | just so we can speed things up because i want to get the aodh fix in | 19:53 |
| evrardjp | which one? and thank you :) | 19:54 |
| evrardjp | mnaser: we 've just pushed a release note to say remove 10.1.31 | 19:54 |
| mnaser | evrardjp: https://review.openstack.org/542119 this one | 19:54 |
| evrardjp | quite a new thing | 19:54 |
| mnaser | galera_major_version in suse vars file instead of galera_client_major_version | 19:55 |
| evrardjp | oh yeah I thought I fixed that everywhere | 19:55 |
| mnaser | wow its already in check queue | 19:56 |
| mnaser | gate is moving quick today | 19:56 |
| *** vnogin has joined #openstack-ansible | 19:59 | |
| mgariepy | wow. installing on hdd is fun.. | 20:07 |
| mgariepy | it's super fast. | 20:08 |
| evrardjp | haha | 20:11 |
| evrardjp | I can't remember when that arrived to me for the last time | 20:11 |
| evrardjp | ssds are the deal right | 20:11 |
| mgariepy | i know. | 20:12 |
| mgariepy | the only good thing about that is the server posting seems really fast comparing to the install .. | 20:12 |
| evrardjp | do the mongo db style | 20:13 |
| evrardjp | put everything into memory | 20:13 |
| evrardjp | that's blazing fast | 20:13 |
| mnaser | evrardjp: https://review.openstack.org/#/c/542119/ done | 20:13 |
| * mnaser crosses fingers his aodh change passes with that dependency | 20:14 | |
| evrardjp | shouldn't be a problem | 20:14 |
| *** vnogin has quit IRC | 20:14 | |
| evrardjp | galera works with just one machine | 20:14 |
| evrardjp | it's jsut the clustering that's borked on .31 | 20:15 |
| mnaser | evrardjp: well aodh has been broken on installing keystone this entire time :\ | 20:16 |
| mnaser | in centos-7 | 20:16 |
| evrardjp | mmm | 20:16 |
| evrardjp | in ubuntu how is it going? | 20:16 |
| evrardjp | link? | 20:17 |
| evrardjp | mhayden: mgariepy cloudnull could one of you merge this: https://review.openstack.org/#/c/542119/ ? please? | 20:17 |
| evrardjp | then I can bump the roles, and deal with the fallout tomorrow | 20:18 |
| mnaser | evrardjp: https://review.openstack.org/#/c/541270/ | 20:18 |
| mnaser | everything passes | 20:18 |
| evrardjp | mnaser: it's late for me, I will do something else of my life | 20:18 |
| evrardjp | :D | 20:19 |
| mnaser | except centos | 20:19 |
| mnaser | no problem | 20:19 |
| evrardjp | ok that's good then! | 20:19 |
| mnaser | i'm at patchset #12 but yeah its just centos, even suse is ok | 20:19 |
| evrardjp | yeah so the fix of major isn't good? | 20:19 |
| mnaser | i just did a depends-on the fix i just modified, i think it might be related because the client and shared rpm was .31 and the common was .30 etc | 20:19 |
| mnaser | http://logs.openstack.org/70/541270/11/check/openstack-ansible-functional-centos-7/f0b3e0c/logs/redhat-rpm-list-installed-infra1.txt.txt.gz | 20:20 |
| mnaser | see mismatching versions top of file | 20:20 |
| evrardjp | yeah | 20:20 |
| *** chyka has joined #openstack-ansible | 20:20 | |
| evrardjp | your patch seem ok | 20:20 |
| evrardjp | yeah the depends on is required to make it work | 20:20 |
| evrardjp | as of now | 20:20 |
| evrardjp | won't be the case in a few hours | 20:21 |
| evrardjp | I mean you wouldn't need those depends-on | 20:21 |
| evrardjp | but let's wait for the logs of this with your last changes | 20:21 |
| mnaser | or maybe there's some other issue :( its stuck on "TASK [os_keystone : Check current state of Keystone DB]" again | 20:21 |
| mnaser | the same place it failed before | 20:21 |
| adcpdk | did anyone had this problem before? E: Failed to fetch http://mirror.rackspace.com/mariadb/repo/10.1/ubuntu/pool/main/m/mariadb-10.1/mariadb-server-10.1_10.1.31+maria-1~xenial_amd64.deb Hash Sum mismatch | 20:22 |
| evrardjp | I can try it tomorrow locally | 20:23 |
| odyssey4me | adcpdk yep, that happens when the mirror updated and removed packages between the apt update and the apt install | 20:23 |
| evrardjp | I'd rather you figure it out though, I'd prefer focusing on fixing the package | 20:23 |
| odyssey4me | I wonder if mariadb isbusy pulling that version now | 20:23 |
| odyssey4me | evrardjp if you're bored, two more of these need reviews: https://review.openstack.org/#/q/topic:bug/1670419+(status:open+OR+status:merged) | 20:24 |
| evrardjp | odyssey4me: that would be nice | 20:24 |
| evrardjp | I am not bored, but my wife has something on the tele | 20:24 |
| evrardjp | :D | 20:24 |
| mnaser | 2018-02-08 20:22:56.284491 | centos-7 | fatal: [infra1]: FAILED! => {"changed": true, "cmd": ["/openstack/venvs/keystone-testing/bin/keystone-manage", "db_sync", "--check"], "delta": "0:01:41.449436", "end": "2018-02-08 20:22:56.147893", "failed_when_result": true, "msg": "non-zero return code", "rc": 1, "start": "2018-02-08 20:21:14.698457", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} | 20:24 |
| mnaser | same thing again | 20:24 |
| evrardjp | I am just tired of xtradb | 20:24 |
| *** chyka has quit IRC | 20:24 | |
| adcpdk | odyssey4me so I should just wait and try again later? | 20:25 |
| odyssey4me | mnaser it sounds like the keystone sync is failing - which will need follow up... unless there's some misconfiguration in that test | 20:25 |
| mnaser | odyssey4me: the sync isnt failing, the check for sync is, but also looking at keystone log, it complains about connection refused to mysql | 20:26 |
| evrardjp | only fails on centos though | 20:26 |
| mnaser | odyssey4me: which then checking the mysql logs.. i can see the service was shutdown | 20:26 |
| odyssey4me | adcpdk usually, yes - but we've been discussing how broken mariadb 10.1.31 is all day, if you didn't notice ;) so if that's a multi-node install it'll fail unless you implement a repo override | 20:26 |
| odyssey4me | adcpdk take a peek at https://review.openstack.org/542144 | 20:27 |
| mnaser | http://logs.openstack.org/70/541270/11/check/openstack-ansible-functional-centos-7/f0b3e0c/logs/openstack/infra1/keystone/keystone.log.txt.gz | 20:27 |
| mnaser | so that tells me mysql isnt running? | 20:27 |
| evrardjp | either using .30, or using a different sst mechanism | 20:27 |
| evrardjp | mnaser: maybe STONITH | 20:27 |
| mnaser | which is weirdly confirmed by this - http://logs.openstack.org/70/541270/11/check/openstack-ansible-functional-centos-7/f0b3e0c/logs/openstack/infra1/mysql_logs/galera_server_error.log.txt.gz | 20:27 |
| mnaser | "[Note] /usr/sbin/mysqld: Normal shutdown" | 20:27 |
| mnaser | its... gracefully shutting down? no errors.. | 20:27 |
| evrardjp | we do a restart at some point, so maybe it doesn't matter | 20:28 |
| evrardjp | what's the keystone log? | 20:28 |
| evrardjp | odyssey4me: I want to get this mariadb out of the way as fast as possible | 20:28 |
| mnaser | http://logs.openstack.org/70/541270/11/check/openstack-ansible-functional-centos-7/f0b3e0c/logs/openstack/infra1/messages.txt.gz#_Feb__8_19_32_37 | 20:28 |
| mnaser | mysql gets shutdown | 20:29 |
| evrardjp | wonder if I should simply go back to a poor mechanism | 20:29 |
| adcpdk | I see thank you odyssey4me | 20:29 |
| mnaser | then tries to start again here http://logs.openstack.org/70/541270/11/check/openstack-ansible-functional-centos-7/f0b3e0c/logs/openstack/infra1/messages.txt.gz#_Feb__8_19_33_08 | 20:29 |
| odyssey4me | evrardjp yeah, I hear you | 20:29 |
| mnaser | http://logs.openstack.org/70/541270/11/check/openstack-ansible-functional-centos-7/f0b3e0c/logs/openstack/infra1/messages.txt.gz#_Feb__8_19_33_08 | 20:30 |
| mnaser | then it's uninstalled?!?! | 20:30 |
| evrardjp | mnaser: compare with ubuntu. if ubuntu works, there is something to fix with centos, just like that. | 20:30 |
| mnaser | Erased: MariaDB-server-10.1.30-1.el7.centos.x86_64 | 20:30 |
| odyssey4me | I'm thinking that we just continue pinning to a specific version - and update that from time to time like the rest of the infra stuff. | 20:30 |
| evrardjp | odyssey4me: that's what I proposed earlier | 20:30 |
| evrardjp | :) | 20:30 |
| mnaser | if you look at the log something uninstalls mariadb | 20:30 |
| odyssey4me | Yeah, I don't like it - but it seems like we're already doing it all over the place so we may as well continue. | 20:31 |
| mnaser | ansible-yum: Invoked with allow_downgrade=False name=['mariadb-common', 'mariadb-config', 'mariadb-server', 'mariadb-libs', 'mariadb'] list=None install_repoquery=True conf_file=None disable_gpg_check=False state=absent disablerepo=None update_cache=False enablerepo=None exclude=None security=False validate_certs=True installroot=/ skip_broken=False | 20:31 |
| mnaser | whaaat | 20:31 |
| * mnaser goes back to ara | 20:31 | |
| *** hw_wutianwei has quit IRC | 20:31 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-openstack_hosts master: Ensure openstack_hosts sets nodepool variables https://review.openstack.org/542352 | 20:31 |
| evrardjp | mnaser: that's because of incompatible packages | 20:31 |
| odyssey4me | mnaser the removal should only be happening for a package version that doesn't match the mariadb version that's supposed to be installed | 20:31 |
| odyssey4me | maybe this will solve the issue you're seeing: https://review.openstack.org/541924 | 20:32 |
| *** armaan has joined #openstack-ansible | 20:32 | |
| evrardjp | https://github.com/openstack/openstack-ansible-galera_server/blob/master/tasks/galera_install.yml#L70-L79 | 20:32 |
| evrardjp | mnaser: ^ | 20:32 |
| mnaser | already depending on that odyssey4me | 20:32 |
| odyssey4me | mnaser our mariadb stuff is great when it works, but is a bit bonkers :/ sorry | 20:33 |
| mnaser | ot | 20:33 |
| mnaser | it's okay, just gotta figure it out | 20:33 |
| *** hw_wutianwei has joined #openstack-ansible | 20:33 | |
| mnaser | Feb 8 19:31:54 localhost yum[10211]: Installed: MariaDB-server-10.1.30-1.el7.centos.x86_64 .. and later .. Feb 8 19:33:31 localhost yum[15791]: Erased: MariaDB-server-10.1.30-1.el7.centos.x86_64 | 20:33 |
| odyssey4me | you should see the ocata version of the role - it's horrible | 20:33 |
| *** armaan has quit IRC | 20:33 | |
| evrardjp | mnaser: yeah that's why I wanted to move to a more smart package installation | 20:34 |
| odyssey4me | https://media.giphy.com/media/374pcIBVEGb6g/giphy.gif | 20:34 |
| evrardjp | :) | 20:35 |
| mnaser | why is there a logic that attempts to delete/remove those packages? | 20:35 |
| evrardjp | not sure what you mean | 20:35 |
| odyssey4me | 'cos the docs said when upgrading, you should remove the old, then install the new | 20:35 |
| odyssey4me | and then of course we also have the stupid situation that you may have a package from the wrong repo | 20:35 |
| mnaser | https://github.com/openstack/openstack-ansible-galera_client/blob/master/vars/redhat-7.yml#L42-L48 | 20:38 |
| mnaser | should we drop that? | 20:38 |
| mnaser | the client shouldnt try to uninstall server rpms? | 20:39 |
| odyssey4me | hmm, that's weird - why is that there mhayden ? | 20:39 |
| mnaser | what seems to happen is | 20:39 |
| openstackgerrit | Merged openstack/openstack-ansible-galera_client master: Prevent latest mariadb version https://review.openstack.org/542119 | 20:39 |
| odyssey4me | https://github.com/openstack/openstack-ansible-galera_client/commit/72cc31a4d6107a69f3d9d5f2bd287112385df1c2 | 20:39 |
| mnaser | galera_client runs, removes nothing, galera_server runs, goes by smoothly | 20:39 |
| mnaser | os_keystone starts, it depends on galera_client, it uninstalls the mysql server | 20:40 |
| mnaser | i guess other tests dont have keystone and galera on the same container? | 20:40 |
| odyssey4me | this time it's hwoarang's fault :) | 20:40 |
| evrardjp | woop | 20:40 |
| evrardjp | I think I merged it. | 20:40 |
| mnaser | yeah so | 20:41 |
| mnaser | https://github.com/openstack/openstack-ansible-os_magnum/blob/master/tests/inventory | 20:41 |
| evrardjp | oh no, it's major which merged it! | 20:41 |
| evrardjp | that's still major breakage | 20:41 |
| mnaser | this one doesnt get messed up because keystone doesn't get installed on the same container as galera | 20:41 |
| odyssey4me | https://media.giphy.com/media/naxep4vNBAOL6/giphy.gif | 20:41 |
| idlemind | lol major breakage | 20:41 |
| mnaser | oh 2 days ago | 20:41 |
| odyssey4me | ah yes of course - the galera_client role does no package actions on the galera_server container... | 20:42 |
| mnaser | i dont know if we have to actaully do that but at least remove mariadb-server | 20:43 |
| odyssey4me | so the issue here is that the inventory isn't set out the way we expect it to... ie with galera_server on a host different to all the stuff using galera_client | 20:43 |
| mnaser | ok so should i add another container for keystone? | 20:43 |
| mnaser | but also this means galera_client is dangerous to run now | 20:43 |
| odyssey4me | the roles should typically have the infra in a container, then a general openstack container for keystone and whatever else, then any additional containers to test whatever the role wants to test | 20:44 |
| mnaser | and i think galera_client runs inside the galera container | 20:44 |
| mhayden | well, on CentOS, remember that upstream mariadb packages are "MariaDB-*" while the ones from centos are "mariadb-" | 20:44 |
| mhayden | so they're different | 20:44 |
| mhayden | those lines remove the stock centos ones if they're present | 20:44 |
| mhayden | and replaces them with the ones from upstream | 20:44 |
| mnaser | so that might make a "surprise, your mariadb server is uninstalled by galera_client and installed by galera_server" | 20:44 |
| odyssey4me | mnaser it does, but with this flag: https://github.com/openstack/openstack-ansible-galera_server/blob/master/meta/main.yml#L43 | 20:45 |
| mhayden | we could pile those into a transaction and make the package install/removal faster | 20:45 |
| mnaser | odyssey4me: ah ok good point | 20:45 |
| odyssey4me | galera_client should be respecting that flag at all times | 20:45 |
| mnaser | yeah but https://github.com/openstack/openstack-ansible-os_keystone/blob/master/meta/main.yml#L44-L46 would still uninstall it | 20:46 |
| mnaser | so i guess i should add a third container | 20:46 |
| mnaser | infra1, openstack1, gnocchi, aodh | 20:46 |
| odyssey4me | mnaser yep, that'd be best | 20:47 |
| odyssey4me | evrardjp this is infuriating, and perhaps confirms that we have a race condition - https://review.openstack.org/#/c/542288/ (os_nova ocata) passed this time | 20:50 |
| openstackgerrit | Merged openstack/openstack-ansible-galera_server master: Set proper yum repo priority on CentOS 7 https://review.openstack.org/541924 | 20:50 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_aodh master: Update the Tempest plugin to point towards the new plugin https://review.openstack.org/541270 | 20:51 |
| mnaser | ok | 20:51 |
| mnaser | i think this might be the one | 20:51 |
| evrardjp | odyssey4me: I am sorry if that's infuriating. So much to do, so little time. | 20:53 |
| evrardjp | I started to take a more systematic approach by building etherpads, but it doesn't work either. | 20:53 |
| evrardjp | (see telemetry stack fixing, zuul v3, ...) | 20:54 |
| *** ChanServ has quit IRC | 20:57 | |
| odyssey4me | time for me to head out - night folks! | 20:57 |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Stop any running VMs when re-deploying https://review.openstack.org/541416 | 20:59 |
| *** hamzy_ has quit IRC | 21:00 | |
| evrardjp | odyssey4me: good night | 21:06 |
| *** acormier has quit IRC | 21:06 | |
| evrardjp | mnaser: have you proposed a revert? | 21:06 |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: Remove placement_database related options/code https://review.openstack.org/541692 | 21:07 |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Replace pxelinux with iPXE support in MNAIO https://review.openstack.org/539327 | 21:07 |
| *** acormier has joined #openstack-ansible | 21:07 | |
| evrardjp | of https://review.openstack.org/#/c/541581/ | 21:07 |
| *** Guy has quit IRC | 21:07 | |
| *** acormier has quit IRC | 21:08 | |
| *** acormier_ has joined #openstack-ansible | 21:08 | |
| *** acormier_ has quit IRC | 21:08 | |
| *** acormier has joined #openstack-ansible | 21:08 | |
| *** dariko has quit IRC | 21:09 | |
| *** hamza21 has quit IRC | 21:10 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova master: Zuul: Remove project name https://review.openstack.org/541117 | 21:14 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-openstack_hosts master: Ensure openstack_hosts sets nodepool variables https://review.openstack.org/542352 | 21:18 |
| openstackgerrit | Merged openstack/openstack-ansible-os_ceilometer stable/newton: Zuul: Remove project name https://review.openstack.org/542348 | 21:24 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-tests master: Use package_state=present for CentOS 7 testing https://review.openstack.org/541016 | 21:25 |
| *** dariko has joined #openstack-ansible | 21:25 | |
| geekinutah | Folks, where would be the right place to do some pre-deployment checks | 21:29 |
| geekinutah | for example, using contrainer_interface "eth0" is not really a viable strategy :-) | 21:29 |
| geekinutah | would like to contribute some sort of check as penance | 21:29 |
| *** dave-mccowan has quit IRC | 21:30 | |
| *** ChanServ has joined #openstack-ansible | 21:31 | |
| *** barjavel.freenode.net sets mode: +o ChanServ | 21:31 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_keystone stable/ocata: Zuul: Remove project name https://review.openstack.org/542383 | 21:33 |
| *** dariko has quit IRC | 21:35 | |
| *** adcpdk has quit IRC | 21:35 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-tests stable/pike: test-set-nodepool-vars: Add nodepool mirror discovery for openSUSE https://review.openstack.org/542456 | 21:42 |
| mhayden | so pike was not checking for nodepool properly and wasn't utilizing mirrors for centos/opensuse ^^ | 21:42 |
| idlemind | mhayden that is specific to the gates right? | 21:44 |
| mhayden | right | 21:44 |
| idlemind | so is this another hope that they speed up | 21:44 |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-os_barbican stable/pike: Zuul: Remove project name https://review.openstack.org/542457 | 21:46 |
| *** jamesdenton has quit IRC | 21:46 | |
| *** dave-mccowan has joined #openstack-ansible | 21:47 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-galera_client master: Revert "RedHat: Ensure that conflicting MariaDB packages are removed" https://review.openstack.org/542458 | 21:57 |
| *** armaan has joined #openstack-ansible | 21:57 | |
| mnaser | woo | 21:59 |
| mnaser | progress | 21:59 |
| mnaser | cradox failing to install with "rados_inconsistent_pg_listq6SIPm.c:(.text+0x15): undefined reference to `rados_inconsistent_pg_list'" .. anyone seen something like this? | 21:59 |
| openstackgerrit | Merged openstack/openstack-ansible master: Implement a scenario with no containers https://review.openstack.org/504224 | 22:05 |
| openstackgerrit | Merged openstack/openstack-ansible master: Remove placement_database related options/code https://review.openstack.org/541691 | 22:05 |
| *** woodard_ has joined #openstack-ansible | 22:06 | |
| *** woodard has quit IRC | 22:06 | |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-openstack_hosts stable/pike: Zuul: Remove project name https://review.openstack.org/542460 | 22:06 |
| *** woodard_ has quit IRC | 22:08 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-openstack_hosts master: [WIP] Test package_state = present https://review.openstack.org/542461 | 22:11 |
| *** sar has quit IRC | 22:14 | |
| *** chhagarw has joined #openstack-ansible | 22:17 | |
| *** chhagarw has quit IRC | 22:21 | |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-os_zaqar stable/pike: Zuul: Remove project name https://review.openstack.org/542466 | 22:26 |
| *** chyka has joined #openstack-ansible | 22:27 | |
| *** chyka has quit IRC | 22:32 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Add nspawn container driver https://review.openstack.org/477017 | 22:37 |
| *** armaan has quit IRC | 22:44 | |
| *** armaan has joined #openstack-ansible | 22:44 | |
| openstackgerrit | James E. Blair proposed openstack/openstack-ansible-galera_client stable/newton: Zuul: Remove project name https://review.openstack.org/542471 | 22:46 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Add nspawn container driver https://review.openstack.org/477017 | 22:48 |
| *** TxGirlGeek has quit IRC | 22:58 | |
| *** foutatoro has quit IRC | 22:58 | |
| *** bauruine has quit IRC | 23:00 | |
| *** armaan has quit IRC | 23:00 | |
| *** TxGirlGeek has joined #openstack-ansible | 23:02 | |
| *** acormier has quit IRC | 23:08 | |
| *** acormier has joined #openstack-ansible | 23:08 | |
| *** chyka has joined #openstack-ansible | 23:08 | |
| mnaser | what role takes care of installing repos for ceph? | 23:11 |
| mnaser | i ask because gnocchi builds with cradox which requires >= jewel but centos ships with hammer by default | 23:11 |
| *** dave-mccowan has quit IRC | 23:12 | |
| mnaser | so need to include whatever role that takes care of adding the latest repo so it installs a modern librados-devel | 23:12 |
| *** acormier has quit IRC | 23:13 | |
| *** rodolof has quit IRC | 23:19 | |
| *** pbandark has quit IRC | 23:21 | |
| *** woodard has joined #openstack-ansible | 23:23 | |
| errr | howdy woodard | 23:32 |
| *** kstev has quit IRC | 23:32 | |
| *** chyka has quit IRC | 23:34 | |
| *** Jeffrey4l has quit IRC | 23:34 | |
| *** weezS has quit IRC | 23:39 | |
| *** chyka has joined #openstack-ansible | 23:49 | |
| *** TxGirlGeek has quit IRC | 23:50 | |
| *** chyka has quit IRC | 23:51 | |
| *** tux_ has quit IRC | 23:51 | |
| *** tux_ has joined #openstack-ansible | 23:52 | |
| *** TxGirlGeek has joined #openstack-ansible | 23:56 | |
| *** tux_ has quit IRC | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
