| openstackgerrit | Jimmy McCrory proposed openstack/openstack-ansible stable/ocata: Fix newton workaround variables https://review.openstack.org/575270 | 00:04 |
|---|---|---|
| *** poul has joined #openstack-ansible | 00:05 | |
| *** poul has quit IRC | 00:09 | |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts master: lxc_install_dnf: Fix symlink https://review.openstack.org/574317 | 00:14 |
| *** vnogin has joined #openstack-ansible | 00:14 | |
| *** hw_wutianwei_ has quit IRC | 00:15 | |
| *** vnogin has quit IRC | 00:19 | |
| *** gkadam has quit IRC | 00:23 | |
| *** poul has joined #openstack-ansible | 00:23 | |
| *** cjloader has joined #openstack-ansible | 00:23 | |
| *** cjloader has quit IRC | 00:27 | |
| *** poul has quit IRC | 00:28 | |
| *** pmannidi has joined #openstack-ansible | 00:31 | |
| *** poul has joined #openstack-ansible | 00:41 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Use elasticsearch coordinator nodes as smart LBs https://review.openstack.org/574870 | 00:42 |
| *** poul has quit IRC | 00:46 | |
| *** chyka has joined #openstack-ansible | 00:46 | |
| *** tux_ has quit IRC | 00:47 | |
| *** cshen has joined #openstack-ansible | 00:48 | |
| *** nurdie_ has quit IRC | 00:48 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_keystone master: Install upstream NGINX repository https://review.openstack.org/575273 | 00:50 |
| *** chyka has quit IRC | 00:51 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Use elasticsearch coordinator nodes as smart LBs https://review.openstack.org/574870 | 00:52 |
| *** cshen has quit IRC | 00:52 | |
| *** gregoryo has joined #openstack-ansible | 00:53 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Use elasticsearch coordinator nodes as smart LBs https://review.openstack.org/574870 | 00:53 |
| *** nurdie has joined #openstack-ansible | 00:58 | |
| *** nurdie has quit IRC | 00:58 | |
| *** poul has joined #openstack-ansible | 00:58 | |
| *** nurdie has joined #openstack-ansible | 00:59 | |
| *** poul has quit IRC | 01:03 | |
| *** nurdie has quit IRC | 01:03 | |
| *** JamesZhu has quit IRC | 01:05 | |
| *** hw_wutianwei_ has joined #openstack-ansible | 01:05 | |
| *** JamesZhu has joined #openstack-ansible | 01:06 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible stable/queens: Update the output for "openstack floating" command https://review.openstack.org/575276 | 01:09 |
| *** vnogin has joined #openstack-ansible | 01:09 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible stable/queens: Fix the appropriate kernel modules path https://review.openstack.org/575278 | 01:12 |
| *** vnogin has quit IRC | 01:13 | |
| *** poul has joined #openstack-ansible | 01:16 | |
| *** poul has quit IRC | 01:21 | |
| openstackgerrit | Merged openstack/openstack-ansible-haproxy_server master: Use realpath filter instead of the stat/dir dance https://review.openstack.org/575145 | 01:24 |
| *** jeferobles has joined #openstack-ansible | 01:25 | |
| mnaser | small backport - https://review.openstack.org/#/q/I07bf9f4bce7bb53ecc305d260142e878797d6c04 | 01:26 |
| openstackgerrit | Merged openstack/openstack-ansible-repo_build master: Add release notes link to README https://review.openstack.org/574570 | 01:27 |
| *** poul has joined #openstack-ansible | 01:34 | |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts master: Adding rpc_pipefs for lxc-openstack aa profile https://review.openstack.org/573259 | 01:36 |
| openstackgerrit | Merged openstack/openstack-ansible-haproxy_server stable/pike: Options to enable SSL for backend connections https://review.openstack.org/570530 | 01:36 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_magnum master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575281 | 01:37 |
| *** poul has quit IRC | 01:39 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_swift master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575282 | 01:39 |
| *** jeferobles has quit IRC | 01:40 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_searchlight master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575283 | 01:40 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_panko master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575285 | 01:41 |
| *** gyee has quit IRC | 01:41 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_nova master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575286 | 01:41 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_neutron master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575287 | 01:43 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_monasca-agent master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575288 | 01:44 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_monasca master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575289 | 01:45 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_masakari master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575290 | 01:46 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_barbican master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575291 | 01:47 |
| *** jeferobles has joined #openstack-ansible | 01:47 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_keystone master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575292 | 01:48 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_heat master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575293 | 01:48 |
| openstackgerrit | Merged openstack/ansible-hardening master: Fixes invalid URL in STIG Deviations summary page https://review.openstack.org/575161 | 01:49 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_glance master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575294 | 01:49 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_designate master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575296 | 01:49 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_cinder master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575297 | 01:50 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_ceilometer master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575298 | 01:51 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_blazar master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575300 | 01:51 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_aodh master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575301 | 01:52 |
| *** poul has joined #openstack-ansible | 01:52 | |
| *** jeferobles has quit IRC | 01:54 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-tests master: Remove all non-voting jobs https://review.openstack.org/575303 | 01:56 |
| *** poul has quit IRC | 01:56 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-tests master: Remove all non-voting jobs https://review.openstack.org/575303 | 01:59 |
| *** nurdie has joined #openstack-ansible | 02:00 | |
| *** vnogin has joined #openstack-ansible | 02:03 | |
| *** nurdie has quit IRC | 02:05 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_keystone master: Use correct variable for sso_callback_template deployment https://review.openstack.org/572502 | 02:06 |
| *** vnogin has quit IRC | 02:07 | |
| *** poul has joined #openstack-ansible | 02:10 | |
| *** poul has quit IRC | 02:15 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Use elasticsearch coordinator nodes as smart LBs https://review.openstack.org/574870 | 02:23 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_keystone master: Install upstream NGINX repository https://review.openstack.org/575273 | 02:27 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_keystone master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575292 | 02:27 |
| *** poul has joined #openstack-ansible | 02:28 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_keystone master: [DNM] test staging uwsgi pkgs from rdo https://review.openstack.org/575309 | 02:29 |
| *** indistylo has joined #openstack-ansible | 02:31 | |
| *** poul has quit IRC | 02:33 | |
| openstackgerrit | michael rice proposed openstack/openstack-ansible-os_keystone stable/pike: Use correct variable for sso_callback_template deployment https://review.openstack.org/575310 | 02:33 |
| *** boxrick_ has quit IRC | 02:36 | |
| *** boxrick_ has joined #openstack-ansible | 02:38 | |
| openstackgerrit | Merged openstack/openstack-ansible-haproxy_server stable/queens: Options to enable SSL for backend connections https://review.openstack.org/570529 | 02:41 |
| *** chhagarw has joined #openstack-ansible | 02:43 | |
| *** poul has joined #openstack-ansible | 02:46 | |
| *** vurtz has joined #openstack-ansible | 02:49 | |
| *** poul has quit IRC | 02:50 | |
| *** tux_ has joined #openstack-ansible | 02:51 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Use elasticsearch coordinator nodes as smart LBs https://review.openstack.org/574870 | 02:52 |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova stable/queens: Fix nova_console_novnc_ssl.yml include path https://review.openstack.org/572434 | 02:54 |
| openstackgerrit | Merged openstack/openstack-ansible stable/ocata: Update all SHAs for 15.1.23 https://review.openstack.org/574050 | 02:54 |
| *** radeks has joined #openstack-ansible | 02:56 | |
| *** vnogin has joined #openstack-ansible | 02:57 | |
| *** cshen has joined #openstack-ansible | 02:58 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_heat master: Deprecate auth_uri option https://review.openstack.org/558379 | 02:59 |
| *** vnogin has quit IRC | 03:01 | |
| *** cshen has quit IRC | 03:03 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova stable/pike: Fix nova_console_novnc_ssl.yml include path https://review.openstack.org/572435 | 03:04 |
| openstackgerrit | zhongshengping proposed openstack/openstack-ansible-os_magnum master: Deprecate auth_uri option https://review.openstack.org/558380 | 03:13 |
| *** dxiri has quit IRC | 03:13 | |
| *** dxiri has joined #openstack-ansible | 03:15 | |
| *** sgrasley has joined #openstack-ansible | 03:23 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: Add more meaningful/user readable failure https://review.openstack.org/571671 | 03:30 |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: Add some troubleshooting informations https://review.openstack.org/571670 | 03:30 |
| openstackgerrit | Merged openstack/openstack-ansible master: releasenotes: Declare openSUSE Leap 42.3 as supported distribution https://review.openstack.org/571212 | 03:30 |
| openstackgerrit | Merged openstack/openstack-ansible master: Add utility playbook to report all listening ports https://review.openstack.org/570211 | 03:30 |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: Fix the appropriate kernel modules path https://review.openstack.org/573750 | 03:30 |
| *** poul has joined #openstack-ansible | 03:40 | |
| *** poul has quit IRC | 03:45 | |
| *** dave-mccowan has quit IRC | 03:45 | |
| *** vnogin has joined #openstack-ansible | 03:51 | |
| *** chyka has joined #openstack-ansible | 03:52 | |
| *** vnogin has quit IRC | 03:55 | |
| *** udesale has joined #openstack-ansible | 03:56 | |
| *** chyka has quit IRC | 03:57 | |
| *** tux_ has quit IRC | 03:57 | |
| *** poul has joined #openstack-ansible | 03:58 | |
| *** poul has quit IRC | 04:02 | |
| *** indistylo has quit IRC | 04:03 | |
| *** poopcat has quit IRC | 04:03 | |
| *** poul has joined #openstack-ansible | 04:16 | |
| *** jeferobles has joined #openstack-ansible | 04:18 | |
| *** poul has quit IRC | 04:20 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Use elasticsearch coordinator nodes as smart LBs https://review.openstack.org/574870 | 04:30 |
| *** poul has joined #openstack-ansible | 04:33 | |
| *** poul has quit IRC | 04:38 | |
| *** jeferobles has quit IRC | 04:38 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Use elasticsearch coordinator nodes as smart LBs https://review.openstack.org/574870 | 04:38 |
| *** vnogin has joined #openstack-ansible | 04:45 | |
| *** threestrands has quit IRC | 04:47 | |
| *** vnogin has quit IRC | 04:49 | |
| *** radeks has quit IRC | 04:51 | |
| *** cshen has joined #openstack-ansible | 04:58 | |
| *** cshen has quit IRC | 05:03 | |
| *** pcaruana has quit IRC | 05:09 | |
| *** chhavi__ has joined #openstack-ansible | 05:22 | |
| *** chhagarw has quit IRC | 05:24 | |
| *** poul has joined #openstack-ansible | 05:28 | |
| *** nurdie has joined #openstack-ansible | 05:30 | |
| *** poul has quit IRC | 05:32 | |
| *** radeks has joined #openstack-ansible | 05:32 | |
| *** dxiri has quit IRC | 05:33 | |
| *** vnogin has joined #openstack-ansible | 05:39 | |
| *** vnogin has quit IRC | 05:44 | |
| *** jeferobles has joined #openstack-ansible | 05:45 | |
| *** poul has joined #openstack-ansible | 05:46 | |
| *** poul has quit IRC | 05:50 | |
| *** vurtz has quit IRC | 05:53 | |
| *** mma has joined #openstack-ansible | 05:59 | |
| *** poul has joined #openstack-ansible | 06:03 | |
| *** poul has quit IRC | 06:08 | |
| *** vurtz has joined #openstack-ansible | 06:10 | |
| *** poul has joined #openstack-ansible | 06:14 | |
| *** pcaruana has joined #openstack-ansible | 06:17 | |
| *** cshen has joined #openstack-ansible | 06:17 | |
| *** nigelmaddock has joined #openstack-ansible | 06:18 | |
| *** poul has quit IRC | 06:19 | |
| *** maybebuggy has joined #openstack-ansible | 06:21 | |
| nigelmaddock | any tips on this one? | 06:21 |
| nigelmaddock | lxc-start: conf.c: instantiate_veth: 2685 failed to create veth pair (4951328f_eth0 and vethBNHNMD): File exists | 06:21 |
| nigelmaddock | i've checked and the link doesn't exist. | 06:22 |
| *** threestrands has joined #openstack-ansible | 06:27 | |
| *** fghaas has joined #openstack-ansible | 06:30 | |
| openstackgerrit | Olivier Bourdon proposed openstack/openstack-ansible master: Add option to change fs type on bootstrap device https://review.openstack.org/566299 | 06:36 |
| *** exodusftw has quit IRC | 06:37 | |
| *** shardy has joined #openstack-ansible | 06:38 | |
| *** epalper has joined #openstack-ansible | 06:40 | |
| *** DanyC has quit IRC | 06:41 | |
| *** DanyC has joined #openstack-ansible | 06:41 | |
| *** pmannidi has quit IRC | 06:42 | |
| *** exodusftw has joined #openstack-ansible | 06:43 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Unfreeze master https://review.openstack.org/573398 | 06:44 |
| *** hassenius has joined #openstack-ansible | 06:45 | |
| *** DanyC has quit IRC | 06:46 | |
| *** hassenius has quit IRC | 06:49 | |
| *** pmannidi has joined #openstack-ansible | 06:52 | |
| *** nurdie has quit IRC | 06:54 | |
| *** nurdie has joined #openstack-ansible | 06:54 | |
| *** nurdie has quit IRC | 06:59 | |
| *** mbuil has joined #openstack-ansible | 07:09 | |
| evrardjp_ | odyssey4me: are you there? | 07:11 |
| *** evrardjp_ is now known as evrardjp | 07:12 | |
| *** smorant has joined #openstack-ansible | 07:12 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_nova master: Fix database creation tasks https://review.openstack.org/574834 | 07:13 |
| evrardjp | how did you check nigelmaddock ? | 07:14 |
| nigelmaddock | ip link show | 07:15 |
| evrardjp | nigelmaddock: it's generally working fine, so I wonder what happened to your machine | 07:15 |
| evrardjp | nigelmaddock: could you check you container lxc config? | 07:15 |
| *** poul has joined #openstack-ansible | 07:16 | |
| *** poul_ has joined #openstack-ansible | 07:16 | |
| evrardjp | for a device configured twice or something | 07:17 |
| evrardjp | are you sure this is the last issue in your log too? | 07:18 |
| *** poul has quit IRC | 07:20 | |
| nigelmaddock | it's the first error I hit | 07:21 |
| nigelmaddock | lxc-start 20180614031309.707 ERROR lxc_conf - conf.c:instantiate_veth:2685 - failed to create veth pair (4951328f_eth0 and veth2FU2X9): File exists | 07:21 |
| nigelmaddock | lxc-start 20180614031309.710 INFO lxc_monitor - monitor.c:lxc_monitor_sock_name:201 - using monitor socket name "lxc/ad055575fe28ddd5//var/lib/lxc" (length of socket name 33 must be <= 105) | 07:21 |
| nigelmaddock | lxc-start 20180614031309.766 ERROR lxc_conf - conf.c:lxc_create_network:3029 - failed to create netdev | 07:21 |
| nigelmaddock | lxc-start 20180614031309.766 ERROR lxc_start - start.c:lxc_spawn:1103 - Failed to create the network. | 07:21 |
| nigelmaddock | lxc-start 20180614031309.766 ERROR lxc_start - start.c:__lxc_start:1358 - Failed to spawn container "infra1_nova_api_container-4951328f". | 07:21 |
| *** gkadam has joined #openstack-ansible | 07:22 | |
| *** jeferobles has quit IRC | 07:24 | |
| *** rmart04 has joined #openstack-ansible | 07:26 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-tests master: [DNM] Test do not build cryptography https://review.openstack.org/574182 | 07:27 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_keystone master: [DNM] Test keystone with cryptography wheel build https://review.openstack.org/574183 | 07:28 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: Update all SHAs for milestone 3 https://review.openstack.org/574006 | 07:28 |
| evrardjp | nigelmaddock: ok that seams reasonable, ip seems you have a dangling veth somewhere | 07:30 |
| evrardjp | it* | 07:30 |
| nigelmaddock | hmm, there seems to be some configuration in the main config file, then there is an include for eth0 and eth2 | 07:30 |
| nigelmaddock | for this instance. | 07:31 |
| evrardjp | yeah that's normak | 07:31 |
| evrardjp | normal | 07:31 |
| nigelmaddock | ok | 07:31 |
| evrardjp | but is there any overlap or something? | 07:31 |
| evrardjp | Did you manually edit those? | 07:31 |
| evrardjp | what's your openstack_user_config like? | 07:31 |
| evrardjp | else you can just reboot, see if the veth issue is still there | 07:32 |
| evrardjp | you're on which os nigelmaddock ? | 07:32 |
| *** cjloader has joined #openstack-ansible | 07:32 | |
| nigelmaddock | yah, rebooted. Ubuntu 16.04 | 07:33 |
| nigelmaddock | let me upload the config somewhere | 07:33 |
| *** pvradu has joined #openstack-ansible | 07:33 | |
| *** vnogin has joined #openstack-ansible | 07:34 | |
| nigelmaddock | https://pastebin.com/wDbKKu3C | 07:36 |
| nigelmaddock | (appreciate the look) | 07:36 |
| *** cjloader has quit IRC | 07:37 | |
| *** tosky has joined #openstack-ansible | 07:37 | |
| *** vurtz has quit IRC | 07:38 | |
| *** vnogin has quit IRC | 07:38 | |
| *** nigelmaddock has quit IRC | 07:39 | |
| *** nigelmaddock has joined #openstack-ansible | 07:40 | |
| nigelmaddock | [sorry vpn timeout!] | 07:40 |
| admin0 | morning \o | 07:43 |
| evrardjp | nigelmaddock: that looks pretty standard to me | 07:48 |
| evrardjp | maybe run lxc-start -n <name> without the -d to see what happens there? | 07:49 |
| *** flaper87 has quit IRC | 07:49 | |
| evrardjp | I have to run | 07:49 |
| *** armaan has joined #openstack-ansible | 07:49 | |
| *** DanyC has joined #openstack-ansible | 07:50 | |
| *** sgrasley has quit IRC | 07:50 | |
| nigelmaddock | thanks evrardjp.. been there, no more info there. | 07:52 |
| nigelmaddock | (already looked there, same complaint about file exists, then failed to create netdev | 07:52 |
| nigelmaddock | appreciate the look Jean-Philippe! | 07:53 |
| *** vnogin has joined #openstack-ansible | 07:53 | |
| evrardjp | :) | 07:55 |
| evrardjp | well I just glanced over it | 07:56 |
| evrardjp | nigelmaddock: wanna try experimental tech? : p | 07:56 |
| evrardjp | (using nspawn vs lxc) | 07:57 |
| *** nattanon has joined #openstack-ansible | 07:57 | |
| nigelmaddock | much of a change? | 07:57 |
| evrardjp | lxc has been treating us good for years, and I haven't got that issue for a while. | 07:57 |
| evrardjp | the dangling veth I mean | 07:58 |
| evrardjp | nigelmaddock: well it's all cgroup-ed processes after all. | 07:58 |
| *** flaper87 has joined #openstack-ansible | 07:58 | |
| evrardjp | but yeah,different tooling | 07:58 |
| evrardjp | I am still confused, this _MUST_ work | 07:58 |
| evrardjp | oh wait | 07:59 |
| evrardjp | idea | 07:59 |
| evrardjp | could you do lxc-ls -f ? | 07:59 |
| evrardjp | my idea revolves around that fact you'd be having a collision in randome generations. | 07:59 |
| nigelmaddock | NAME STATE AUTOSTART GROUPS IPV4 IPV6 | 07:59 |
| nigelmaddock | infra1_cinder_api_container-8d753d50 STOPPED 1 onboot, openstack - - | 07:59 |
| nigelmaddock | etc | 08:00 |
| evrardjp | -e | 08:00 |
| evrardjp | yeah, but the etc interests me :) | 08:00 |
| evrardjp | on a pastebin maybe? | 08:00 |
| evrardjp | (you run as root, right?) | 08:00 |
| evrardjp | which branch are you on? | 08:01 |
| nigelmaddock | yah root.. i've blown the containers away a few times. | 08:01 |
| evrardjp | how did you delete them? | 08:01 |
| evrardjp | lxc-destroy? | 08:02 |
| nigelmaddock | yep | 08:02 |
| evrardjp | did you then remove the files? | 08:02 |
| nigelmaddock | 17.0.5 | 08:02 |
| nigelmaddock | the /tmp or elsewhere? | 08:02 |
| evrardjp | nigelmaddock: ok | 08:02 |
| evrardjp | nope the ones in lxc config folders | 08:02 |
| nattanon | Sorry, Do someone ever met console on horizon make everything double on window instance ? | 08:02 |
| evrardjp | everything double nattanon ? | 08:03 |
| evrardjp | not sure to understand | 08:03 |
| nattanon | for example typing 'a' output will be 'aa' | 08:03 |
| evrardjp | oh | 08:03 |
| evrardjp | novnc or spice? | 08:03 |
| nattanon | novnc | 08:03 |
| evrardjp | mmm | 08:03 |
| evrardjp | never got that | 08:03 |
| *** gregoryo has quit IRC | 08:04 | |
| nattanon | i look at the debugging on my browser it show me that horizon console request websocket 2 times | 08:04 |
| evrardjp | But last time I ran a windows instance was a few years back :p | 08:04 |
| evrardjp | ok that might be the cause :p | 08:04 |
| evrardjp | nigelmaddock: do you have the output of the lxc-ls -f handy? | 08:05 |
| evrardjp | out of curiosity | 08:05 |
| nattanon | I also saw the process on nova-novncproxy double. | 08:05 |
| evrardjp | nigelmaddock: also try to grep 4951328f in that folder | 08:05 |
| evrardjp | I have to run | 08:06 |
| nigelmaddock | np. | 08:06 |
| nigelmaddock | need to rerun the playbook from that point, just killed the containers again | 08:06 |
| nigelmaddock | it does kill them from /var/lib/lxc | 08:06 |
| nattanon | evrardjp: Do you have any idea for this caused by horizon or something else ? | 08:07 |
| admin0 | nattanon, what OS are you booting up ? | 08:09 |
| nattanon | admin0: window2012R2, window2016 | 08:10 |
| admin0 | i saw it before, but it was due to OS related | 08:10 |
| admin0 | meaing the OS echos back whatever is typed | 08:10 |
| admin0 | try with a cirros or other image and see if that is still the case | 08:10 |
| nattanon | admin: all window image got the same things. Linux also got 2 socket but it work normally. | 08:10 |
| *** hassenius has joined #openstack-ansible | 08:11 | |
| nattanon | admin0: For fefdora centos ubuntu cirros got the same situation but it work fine no more double for all the things. | 08:11 |
| *** hasseniu_ has joined #openstack-ansible | 08:12 | |
| admin0 | nattanon, "got same situation " and "no more double" and opposite of one another :) | 08:12 |
| admin0 | are* | 08:12 |
| admin0 | so issue is only with windows images ? | 08:12 |
| nattanon | admin0: Sorry let me try again. | 08:12 |
| nattanon | admin0: yes | 08:13 |
| nattanon | admin0: only window. | 08:13 |
| admin0 | nattanon, how was the image generated ? | 08:14 |
| evrardjp | nigelmaddock: yes it is supposed to kill them from the folder | 08:15 |
| evrardjp | but who knows... I was worried your veth pairs are not maching the container name | 08:15 |
| *** hassenius has quit IRC | 08:15 | |
| evrardjp | we've wired them together to make life of deployers easy | 08:15 |
| evrardjp | (i.e. containername-b33f would have b33f-eth{0,1,...} veths | 08:16 |
| evrardjp | nigelmaddock: I am concerned a recent change in the lxc_hosts/container_create roles could have changed something that only get caught on reboot, which we do not test in gates (patches welcomed there!) | 08:17 |
| nattanon | admin0: It our iso licensed then using qemu-img for converting, but my team just finished try that image on mirantis it work fine. | 08:17 |
| *** hamzy_ has joined #openstack-ansible | 08:18 | |
| evrardjp | nattanon: then check the difference with mirantis in terms of configs/processes maybe? | 08:18 |
| evrardjp | check novnc/horizon versions too? | 08:18 |
| evrardjp | does it work directly outside horizon? | 08:18 |
| nigelmaddock | evrardjp; yep, that pairing looks correct | 08:19 |
| nigelmaddock | lxc-start 20180614031230.963 INFO lxc_conf - conf.c:instantiate_veth:2739 - Attached '6a1e5523_eth0': to the bridge 'lxcbr0': | 08:19 |
| nigelmaddock | lxc-start 20180614031230.964 ERROR lxc_conf - conf.c:instantiate_veth:2685 - failed to create veth pair (6a1e5523_eth0 and vethCJ0VT9): File exists | 08:19 |
| admin0 | nattanon, try one of this image a well: https://cloudbase.it/windows-cloud-images/#download .. see where the issue lies | 08:19 |
| admin0 | could be some drivers etc missing | 08:20 |
| *** hamzy has quit IRC | 08:20 | |
| nattanon | evrardjp: Okay, I will try to connect using outside vnc server. | 08:21 |
| *** hasseniu_ has quit IRC | 08:21 | |
| nattanon | admin0: Okay i will try now. | 08:21 |
| nigelmaddock | evrardjp: this is my first foray into Openstack, getting there but its a learning curve :) | 08:22 |
| hwoarang | mnaser: evrardjp thank you for looking into https://review.openstack.org/#/c/574834/ | 08:22 |
| hwoarang | it was blocking my distro work for nova:) | 08:22 |
| evrardjp | nigelmaddock it shouldn't be. Everything should be easy on that side. | 08:24 |
| *** armaan has quit IRC | 08:24 | |
| evrardjp | you don't have lxd/nspawn machine running at the same time, right? | 08:24 |
| evrardjp | hwoarang: I am puzzled. | 08:25 |
| evrardjp | How the hell did that one get merged | 08:25 |
| evrardjp | I am at fault there | 08:25 |
| evrardjp | but this should never have merged. | 08:25 |
| hwoarang | lately we find many gaps in testing | 08:25 |
| evrardjp | it even passed gating >< | 08:26 |
| nigelmaddock | evrardjp; not running but it could be there. these targets are fresh installs as per the deployment guide. | 08:26 |
| nigelmaddock | root@ing-target0:/var/lib/lxc# dpkg --list |grep nspawn | 08:26 |
| nigelmaddock | ii systemd-container 229-4ubuntu21.2 amd64 systemd container/nspawn tools | 08:26 |
| evrardjp | we need to move to testinfra. | 08:26 |
| *** armaan has joined #openstack-ansible | 08:26 | |
| evrardjp | machinectl list? | 08:26 |
| evrardjp | nigelmaddock: ^ | 08:26 |
| evrardjp | nigelmaddock: I am very confused because it should run by default. | 08:26 |
| nigelmaddock | MACHINE CLASS SERVICE | 08:26 |
| nigelmaddock | 0 machines listed. | 08:26 |
| evrardjp | ok so that's good | 08:27 |
| evrardjp | lxc ls (for lxd checking, yes, that's confusing cli) | 08:27 |
| nigelmaddock | 'lxc ls' isn't a good option for that version | 08:27 |
| evrardjp | lxc list maybe? | 08:28 |
| evrardjp | I can't remember | 08:28 |
| evrardjp | :p | 08:28 |
| nigelmaddock | yah yah, was just making sure it was the right binary | 08:28 |
| nigelmaddock | empty list | 08:28 |
| *** hassenius has joined #openstack-ansible | 08:29 | |
| nigelmaddock | root@ing-target0:/var/lib/lxc# lxc list | 08:29 |
| nigelmaddock | +------+-------+------+------+------+-----------+ | 08:29 |
| nigelmaddock | | NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS | | 08:29 |
| nigelmaddock | +------+-------+------+------+------+-----------+ | 08:29 |
| *** hasseniu_ has joined #openstack-ansible | 08:29 | |
| evrardjp | nigelmaddock: ok so we are good, and we've isolated that to a lxc issue. | 08:29 |
| evrardjp | and now the ip link show ? | 08:30 |
| *** armaan has quit IRC | 08:30 | |
| evrardjp | nigelmaddock: I am started a 17.0.5 just to make sure it's not in our code :p | 08:30 |
| evrardjp | nigelmaddock: you're the first one to report this | 08:30 |
| nigelmaddock | 10: lxcbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000 | 08:30 |
| nigelmaddock | link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff | 08:30 |
| nigelmaddock | 35: lxdbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 | 08:30 |
| nigelmaddock | link/ether 22:8a:0b:1b:ed:9e brd ff:ff:ff:ff:ff:ff | 08:30 |
| nigelmaddock | thats the bridges outside the other ones | 08:31 |
| evrardjp | mmm | 08:31 |
| evrardjp | and no veth, and all the containers are marked as failed to boot? | 08:31 |
| nigelmaddock | correct. all fail to boot, and no veth ever get created | 08:32 |
| evrardjp | apparmor maybe? | 08:32 |
| nigelmaddock | I have started a standalone container ok | 08:32 |
| evrardjp | ok | 08:32 |
| evrardjp | have you created your networksto bridge them properly? | 08:33 |
| evrardjp | could you show a full output of ip link show? | 08:33 |
| *** electrofelix has joined #openstack-ansible | 08:33 | |
| *** hassenius has quit IRC | 08:33 | |
| *** pmannidi has quit IRC | 08:33 | |
| nigelmaddock | https://pastebin.com/xknLREwx | 08:34 |
| *** armaan has joined #openstack-ansible | 08:34 | |
| admin0 | why you have lxd instead of lxc ? | 08:36 |
| evrardjp | a real mystery for me, but I am very tired. | 08:36 |
| evrardjp | admin0: he has both | 08:36 |
| evrardjp | maybe removing lxd could help, but I am not sure | 08:36 |
| admin0 | won't lxc not work when lxd is there ? | 08:36 |
| nigelmaddock | hmm, the lxd only appears after the playbook run | 08:37 |
| evrardjp | I don't know but I am going to try when my 17.0.5 is up | 08:37 |
| admin0 | but that was my observation a long time back .. so i use lxd only on computes and if they are lxd | 08:38 |
| admin0 | i have not tried on 17.0.5 | 08:38 |
| *** ianychoi has quit IRC | 08:39 | |
| nigelmaddock | just removing those containers again and bouncing the box... | 08:39 |
| *** armaan has quit IRC | 08:41 | |
| nigelmaddock | yep lxcbr0 is there at boot | 08:41 |
| nigelmaddock | based on the ansible /etc/network/interfaces.d/lxc-net-bridge.cfg include | 08:42 |
| hwoarang | evrardjp: would you like to give an easy +2 on https://review.openstack.org/#/c/574694/ ? | 08:42 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone stable/queens: Use correct variable for sso_callback_template deployment https://review.openstack.org/575374 | 08:45 |
| nigelmaddock | ahhh... they are being created but something is happening. | 08:46 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_keystone stable/ocata: Use correct variable for sso_callback_template deployment https://review.openstack.org/575375 | 08:46 |
| nigelmaddock | I ran a ip link show _during_ the playbook and it showed them | 08:46 |
| nigelmaddock | 12: vethH0SMJM@if11: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master lxcbr0 state UP mode DEFAULT group default qlen 1000 | 08:46 |
| nigelmaddock | link/ether fe:9a:9a:5e:9c:f4 brd ff:ff:ff:ff:ff:ff link-netnsid 3 | 08:46 |
| nigelmaddock | etc | 08:46 |
| nigelmaddock | then they disappeared again | 08:46 |
| nigelmaddock | so to "file exists" could be due to duplicate creation ? | 08:50 |
| *** armaan has joined #openstack-ansible | 08:57 | |
| *** sgrasley has joined #openstack-ansible | 08:59 | |
| *** pvradu has quit IRC | 09:00 | |
| *** pvradu has joined #openstack-ansible | 09:01 | |
| evrardjp | nigelmaddock: yeah you see in the line before that it creates it and fails | 09:02 |
| evrardjp | (the line before in your log) | 09:02 |
| evrardjp | lxcbr0 is normal | 09:02 |
| evrardjp | lxdbr0 is ... I will check! | 09:02 |
| nigelmaddock | lxdbr0 isn't there now however. | 09:02 |
| nigelmaddock | maybe that was an artifact of my other container creation | 09:02 |
| admin0 | why not remove the whole lxd* stuff and keep only lxc stuff | 09:03 |
| admin0 | apt | 09:03 |
| nigelmaddock | the lxdbr0 only appeared after I tried to start things by hand | 09:04 |
| nigelmaddock | i'll have a look at removing the lxd tools | 09:05 |
| *** radeks has quit IRC | 09:05 | |
| *** pvradu has quit IRC | 09:05 | |
| evrardjp | nigelmaddock: I am still deploying on 17.0.5 | 09:08 |
| evrardjp | no issue, I've been through the playbooks | 09:08 |
| *** pvradu has joined #openstack-ansible | 09:09 | |
| *** sgrasley has quit IRC | 09:10 | |
| *** cronix has quit IRC | 09:11 | |
| *** armaan has quit IRC | 09:11 | |
| *** hasseniu_ has quit IRC | 09:11 | |
| *** hassenius has joined #openstack-ansible | 09:11 | |
| *** hasseniu_ has joined #openstack-ansible | 09:12 | |
| *** armaan has joined #openstack-ansible | 09:13 | |
| *** hassenius has quit IRC | 09:16 | |
| *** chyka has joined #openstack-ansible | 09:17 | |
| *** chyka has quit IRC | 09:21 | |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts master: tasks: lxc_volume: Always create the machines.raw file https://review.openstack.org/574694 | 09:23 |
| admin0 | evrardjp, do you know if i need to do cinder using NFS, is this file https://docs.openstack.org/openstack-ansible-os_cinder/latest/configure-cinder.html -- /etc/shares/nfs_share_volume1 -- is that something i provide or is generated .. and if i need to provide, what is the content | 09:25 |
| *** vnogin has quit IRC | 09:27 | |
| evrardjp | admin0: I think you have to create it | 09:28 |
| admin0 | in the deploy container ? | 09:28 |
| admin0 | in the deploy system .. or in the compute nodes or in controllers ? | 09:28 |
| admin0 | and what is the content like ? | 09:28 |
| *** vnogin has joined #openstack-ansible | 09:29 | |
| evrardjp | admin0: I never used it myself, so you have to dig in the code | 09:29 |
| evrardjp | my bad | 09:30 |
| evrardjp | https://github.com/openstack/openstack-ansible-os_cinder/blob/701fd0ba57bf83b3273855e012d50aa1d4be5ad2/tasks/cinder_post_install.yml#L69-L75 | 09:30 |
| *** udesale_ has joined #openstack-ansible | 09:30 | |
| *** gkadam_ has joined #openstack-ansible | 09:30 | |
| evrardjp | https://github.com/openstack/openstack-ansible-os_cinder/blob/701fd0ba57bf83b3273855e012d50aa1d4be5ad2/templates/nfs_shares.j2#L3 | 09:30 |
| evrardjp | so it's good, don't worry | 09:30 |
| evrardjp | you still need to have your nfs server handy | 09:31 |
| MasterofJOKers | hi. are there logs for this channel online somewhere? | 09:31 |
| evrardjp | yes | 09:31 |
| evrardjp | on classical openstack logs | 09:31 |
| evrardjp | let me find that for you | 09:31 |
| MasterofJOKers | thx | 09:31 |
| evrardjp | (let me show you how I got it at the same time) | 09:31 |
| evrardjp | in our docs, on onboarding, we are talking about irc: https://docs.openstack.org/openstack-ansible/latest/contributor/project-onboarding.html | 09:31 |
| evrardjp | it gives a link to this: http://eavesdrop.openstack.org/irclogs/%23openstack-ansible/ | 09:32 |
| evrardjp | (for daily logs) | 09:32 |
| evrardjp | and this: http://eavesdrop.openstack.org/#OpenStack_Ansible_Deployment_Meeting | 09:32 |
| evrardjp | for meetings | 09:32 |
| evrardjp | MasterofJOKers: hope it helps. | 09:32 |
| *** jmnk has quit IRC | 09:32 | |
| *** jmnk_ has joined #openstack-ansible | 09:32 | |
| MasterofJOKers | yes, thank you | 09:32 |
| *** gkadam__ has joined #openstack-ansible | 09:33 | |
| *** udesale__ has joined #openstack-ansible | 09:33 | |
| *** gkadam has quit IRC | 09:34 | |
| *** udesale has quit IRC | 09:34 | |
| *** boxrick_ is now known as boxrick | 09:35 | |
| *** gkadam_ has quit IRC | 09:36 | |
| *** udesale_ has quit IRC | 09:36 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/ansible-hardening master: SUSE: Add support for openSUSE Leap 15 https://review.openstack.org/575381 | 09:38 |
| openstackgerrit | Merged openstack/ansible-hardening master: Update links in README https://review.openstack.org/574585 | 09:50 |
| openstackgerrit | Merged openstack/openstack-ansible-os_barbican master: Add release notes link to README https://review.openstack.org/575046 | 09:50 |
| *** threestrands has quit IRC | 09:54 | |
| *** sgrasley has joined #openstack-ansible | 09:54 | |
| *** cshen has quit IRC | 09:57 | |
| nattanon | evrardjp: It's proove | 10:00 |
| nattanon | evrardjp: for my problem all typing are twice in horizon console | 10:01 |
| nattanon | evrardjp: It because of twice connection to novnc server | 10:02 |
| evrardjp | ok, and why is that? | 10:03 |
| nattanon | evrardjp: I had check configure between mirantis and my OSA(pike) | 10:03 |
| evrardjp | so horizon is opening the connection twice? | 10:03 |
| nattanon | evrardjp: Yes | 10:03 |
| evrardjp | but novnc is fine? | 10:03 |
| evrardjp | so it means it's a horizon bug? | 10:03 |
| nattanon | evrardjp: My configuration quite the same | 10:03 |
| evrardjp | or a config issue? | 10:03 |
| nattanon | evrardjp: Now i'm trying to hard to find out | 10:04 |
| nattanon | evrardjp: It not solved yet. | 10:04 |
| evrardjp | but you've tested and proven that it works fine with novnc directly | 10:06 |
| nattanon | admin0: i prove their is not related to the image. It happen because double connection to novnc server via websocket. | 10:06 |
| nattanon | evrardjp: novnc directly it on process. | 10:08 |
| *** JamesZhu has quit IRC | 10:09 | |
| nattanon | evrardjp: i found it accidentally when i trying to clear cache when the first socket are established it working fine but just a few sec later second socket is established also then all my action are twice. | 10:10 |
| *** JamesZhu has joined #openstack-ansible | 10:10 | |
| hwoarang | evrardjp: cloudnull logan-: would you like to restore your votes on https://review.openstack.org/#/c/573758/ ? only commit message changed | 10:14 |
| nattanon | evrardjp: horizon is loading all file twice for my case js and css also loaded twice. So i guest it caused by horizon. | 10:18 |
| evrardjp | hwoarang: didn't we fix the tox.ini not triggeringjobs? | 10:22 |
| hwoarang | no this is the patch for it | 10:22 |
| evrardjp | nope | 10:23 |
| evrardjp | I meant the patch for the ignored files | 10:23 |
| evrardjp | can I have votes on https://review.openstack.org/#/c/574276/2 ? | 10:24 |
| evrardjp | odyssey4me: mgariepy cloudnull andymccr | 10:24 |
| hwoarang | yeah that's the correct one :( | 10:25 |
| hwoarang | sorry | 10:25 |
| evrardjp | no worries | 10:25 |
| evrardjp | I am restoring votes | 10:25 |
| hwoarang | got lost with this spahetti of dependencies | 10:25 |
| hwoarang | *spaghetti | 10:25 |
| evrardjp | I'd rather have that for 573758 | 10:26 |
| evrardjp | no worries | 10:26 |
| evrardjp | you tackled a lot of things | 10:26 |
| *** radeks has joined #openstack-ansible | 10:26 | |
| *** blinkiz has joined #openstack-ansible | 10:34 | |
| blinkiz | https://paste.ubuntu.com/p/9Fn47b6hS6/ Am having problem with IPv6 addresses and /opt/get-pip.py. Why is get-pip.py looking for something at 127.0.0.1? Apparently it is wrapping 127.0.0.1 in square brackets ([]) when am using an IPv6 address as internal_lb_vip_address. | 10:35 |
| *** nicolasbock has joined #openstack-ansible | 10:37 | |
| *** jwitko_ has quit IRC | 10:44 | |
| evrardjp | blinkiz: http://git.openstack.org/cgit/openstack/openstack-ansible/tree/inventory/group_vars/repo_all.yml | 10:50 |
| evrardjp | that's bad | 10:50 |
| evrardjp | could you help fix it? | 10:50 |
| evrardjp | blinkiz: we need to replace internal_lb_vip_address | ipwrap with 'localhost' | 10:51 |
| evrardjp | or at least file a bug :) | 10:52 |
| evrardjp | good catch | 10:52 |
| evrardjp | are you the one that proposed the ipwrap fix btw? | 10:52 |
| evrardjp | submitting a bug we triaged this week? | 10:52 |
| *** admin0 has quit IRC | 10:52 | |
| openstackgerrit | Merged openstack/openstack-ansible-tests master: zuul.d: jobs: Trigger functional jobs on tox.ini changes https://review.openstack.org/574276 | 11:07 |
| *** sxc731 has joined #openstack-ansible | 11:07 | |
| blinkiz | evrardjp, yes, am the one working on ipv6 support when I have the time here at my workplace. | 11:09 |
| blinkiz | evrardjp, thanks for the lead on where to solve the problem. I try a solution and create a bug report if it seems to work alright | 11:10 |
| *** rmart04 has quit IRC | 11:12 | |
| *** rmart04 has joined #openstack-ansible | 11:12 | |
| *** armaan has quit IRC | 11:15 | |
| *** cshen has joined #openstack-ansible | 11:21 | |
| *** ansmith has quit IRC | 11:26 | |
| evrardjp | cool | 11:35 |
| *** nicolasbock has quit IRC | 11:42 | |
| *** udesale_ has joined #openstack-ansible | 11:42 | |
| mnaser | https://review.openstack.org/#/c/575273/ | 11:44 |
| mnaser | Another epel cleanup, so keystone works without it | 11:44 |
| *** udesale__ has quit IRC | 11:45 | |
| hwoarang | mnaser: i am curious. are we safe to pull so many repos on centos? | 11:48 |
| hwoarang | some packages from epel, some from rdo, we also have galera, rabbitmq, now nginx | 11:48 |
| hwoarang | so many moving parts... | 11:48 |
| *** udesale_ has quit IRC | 11:48 | |
| hwoarang | out of our control | 11:49 |
| mnaser | hwoarang: I don’t think we have much choice. However, the long term with everything will be in rdo except nginx | 11:50 |
| mnaser | Rabbit and Galera are already there | 11:50 |
| hwoarang | ok that would be nice | 11:50 |
| mnaser | Maybe if we do uwsgi cleanly from rdo, we can then convince them to bring nginx in | 11:51 |
| mnaser | Which should eliminate most dependencies. | 11:51 |
| hwoarang | ok that would be nice | 11:53 |
| hwoarang | mnaser: for this patch in particular i suggest you use the include thing and only allow nginx from that repo | 11:54 |
| hwoarang | i have no idea what else they may ship there in the future | 11:54 |
| hwoarang | also, the benefit of using epel is that it's mirrored in infra so we get better stability compared to external repos | 11:55 |
| hwoarang | :/ | 11:55 |
| hwoarang | maybe worth using it for nginx too | 11:55 |
| hwoarang | let me comment on the bug | 11:55 |
| *** fghaas has left #openstack-ansible | 12:03 | |
| openstackgerrit | Nicolas Bock proposed openstack/openstack-ansible master: Internal IPv6 addresses need to be wrapped in `[]` https://review.openstack.org/574893 | 12:04 |
| *** vnogin has quit IRC | 12:08 | |
| evrardjp | blinkiz: this patch from Nicolas Bock would help you, as internal LB vip addresses would be declared without the [], and would be autowrapped everywhere you need to: https://review.openstack.org/#/c/574893/3 | 12:09 |
| *** vnogin has joined #openstack-ansible | 12:10 | |
| *** nicolasbock has joined #openstack-ansible | 12:11 | |
| *** JamesZhu has quit IRC | 12:14 | |
| *** JamesZhu has joined #openstack-ansible | 12:14 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_neutron master: tox.ini: Use common vars file for distibution installations https://review.openstack.org/575418 | 12:22 |
| *** JamesZhu has quit IRC | 12:22 | |
| *** JamesZhu has joined #openstack-ansible | 12:22 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_keystone master: vars: Add required devel packages https://review.openstack.org/574170 | 12:29 |
| mnaser | hwoarang: thanks for your comments, appreciate if cores can look into https://review.openstack.org/#/c/575273/ so we can come to a conclusion so we can resolve this issue | 12:45 |
| mnaser | because centos is blocked till this is fixed | 12:45 |
| hwoarang | mnaser: maybe we need something similar in horizon? | 12:46 |
| *** ianychoi has joined #openstack-ansible | 12:46 | |
| hwoarang | iirc it uses nginx | 12:46 |
| hwoarang | and any other role which still uses httpd | 12:46 |
| hwoarang | :/ | 12:46 |
| mnaser | uses nginx*? | 12:46 |
| mnaser | horizon uses httpd with mod_wsgi afaik | 12:47 |
| mnaser | nginx has repos | 12:47 |
| hwoarang | is httpd and mod_wsgi in standard centos repos? | 12:47 |
| mnaser | hwoarang: yep | 12:48 |
| hwoarang | ok great | 12:48 |
| *** ansmith has joined #openstack-ansible | 12:59 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova master: Fix database creation tasks https://review.openstack.org/574834 | 13:00 |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_keystone master: [DNM] test staging uwsgi pkgs from rdo https://review.openstack.org/575309 | 13:01 |
| *** maybebuggy has quit IRC | 13:03 | |
| evrardjp | wow | 13:03 |
| evrardjp | milestone | 13:03 |
| *** lihi has quit IRC | 13:06 | |
| mgariepy | evrardjp, saw my comments on https://review.openstack.org/#/c/574722/ ? | 13:06 |
| *** lihi has joined #openstack-ansible | 13:06 | |
| mnaser | evrardjp: milestone? | 13:09 |
| *** dave-mccowan has joined #openstack-ansible | 13:10 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_glance master: tox.ini: Use common vars file for distibution installations https://review.openstack.org/575428 | 13:13 |
| *** armaan has joined #openstack-ansible | 13:15 | |
| evrardjp | mnaser: yeah we have uwsgi packages in rdo! | 13:22 |
| evrardjp | temporarily! | 13:22 |
| *** jamesdenton has joined #openstack-ansible | 13:23 | |
| evrardjp | I'd like to refactor that log collection to be a role, and used in gates, or if some ppl want it, run on their cloud | 13:24 |
| evrardjp | because that starts to get hard to read | 13:24 |
| evrardjp | mgariepy: ^ | 13:24 |
| evrardjp | but in the meantime we are probably good | 13:24 |
| *** armaan has quit IRC | 13:29 | |
| *** armaan has joined #openstack-ansible | 13:29 | |
| tinwood | Hi. I'm very hoping someone here can help me. I'm tinwood (alex) and maintain the nova-lxd plugin for nova and "we" have a failing test: openstack-ansible-nova-lxd. The problem is, I can't find the test anywhere!! | 13:31 |
| *** armaan has quit IRC | 13:32 | |
| *** armaan has joined #openstack-ansible | 13:32 | |
| *** nattanon has quit IRC | 13:36 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_keystone master: tox.ini: Use common vars file for distibution installations https://review.openstack.org/573758 | 13:37 |
| *** armaan has quit IRC | 13:37 | |
| evrardjp | hello tinwood | 13:38 |
| tinwood | hi | 13:38 |
| evrardjp | so nova lxd is tested in our nova repo | 13:38 |
| evrardjp | https://github.com/openstack/openstack-ansible-os_nova | 13:38 |
| evrardjp | (do you want the long story or the short story?) | 13:39 |
| evrardjp | https://github.com/openstack/openstack-ansible-os_nova/blob/master/tox.ini#L115-L123 | 13:39 |
| tinwood | evrardjp, ah yes. What I was trying to do was to reproduce the failing test in a devstack-gate instance (I have a script that runs it up). | 13:39 |
| evrardjp | https://github.com/openstack/openstack-ansible-os_nova/blob/master/tests/os_nova-overrides-lxd.yml | 13:39 |
| tinwood | But I couldn't track down the test. | 13:39 |
| evrardjp | so go into that repo | 13:40 |
| evrardjp | ./run_tests.sh lxd | 13:40 |
| evrardjp | should do the trick | 13:40 |
| evrardjp | have a vm | 13:40 |
| evrardjp | so that you won't crush your current machine :) | 13:40 |
| evrardjp | sorry | 13:40 |
| evrardjp | it's not lxd | 13:40 |
| evrardjp | it's | 13:40 |
| tinwood | so that's essentially: openstack-ansible-nova-lxd | 13:40 |
| *** jwitko has joined #openstack-ansible | 13:40 | |
| evrardjp | ./run_tests.sh func_lxd | 13:40 |
| tinwood | Okay, I'll give that a go. | 13:41 |
| evrardjp | openstack-ansible-lxd-ubuntu-xenial | 13:41 |
| evrardjp | that's the name of our job | 13:41 |
| tinwood | evrardjp, that's the bit that's confusing me. The test that fails: https://review.openstack.org/#/c/572724/ | 13:42 |
| *** armaan has joined #openstack-ansible | 13:42 | |
| evrardjp | see here: https://github.com/openstack/openstack-ansible-os_nova/blob/master/zuul.d/project.yaml#L29 and https://github.com/openstack/openstack-ansible-os_nova/blob/master/zuul.d/jobs.yaml#L17-L20 | 13:42 |
| evrardjp | oh ok | 13:42 |
| evrardjp | you're talking about something else | 13:42 |
| evrardjp | my bad | 13:42 |
| evrardjp | it's not our job to test lxd you want | 13:42 |
| tinwood | actually, it's defined in nova-lxd .zuul.d | 13:42 |
| evrardjp | it's your job that tests lxd with our deploy | 13:42 |
| tinwood | but I couldn't work out what it actually "did" | 13:42 |
| evrardjp | the other way :) | 13:42 |
| tinwood | yup. | 13:43 |
| evrardjp | yeah so | 13:43 |
| evrardjp | let me explain that differently then :) | 13:43 |
| tinwood | e.g. here: https://github.com/openstack/nova-lxd/blob/master/.zuul.yaml | 13:43 |
| tinwood | actually you've already explained it! | 13:43 |
| evrardjp | https://github.com/openstack/nova-lxd/blob/master/.zuul.yaml#L17 is a cross repo job I will skip for now | 13:43 |
| evrardjp | it takes the https://github.com/openstack/nova-lxd/blob/master/.zuul.yaml#L20 I've explained before | 13:44 |
| evrardjp | and runs https://github.com/openstack/nova-lxd/blob/master/.zuul.yaml#L22 I've explained too | 13:44 |
| evrardjp | for https://github.com/openstack/nova-lxd/blob/master/.zuul.yaml#L17 to work | 13:44 |
| tinwood | tox_env func_lxd is the key, and openstack-ansible-os_nova. | 13:44 |
| tinwood | So it's the same thing. | 13:44 |
| evrardjp | we need to tell where to come from, which is why https://github.com/openstack/nova-lxd/blob/master/.zuul.yaml#L23 | 13:44 |
| evrardjp | yeah | 13:44 |
| tinwood | excellent! | 13:44 |
| evrardjp | just more wiring | 13:44 |
| evrardjp | :) | 13:44 |
| tinwood | evrardjp, thanks -- now I *think* I understand. i'll run up a vm and see if I can reproduce. | 13:45 |
| tinwood | thanks very much! | 13:45 |
| evrardjp | tinwood: but that should have helped you https://github.com/openstack/nova-lxd/blob/master/.zuul.yaml#L13-L14 :) | 13:45 |
| tinwood | evrardjp, you know sometimes how you just "don't see it"? | 13:45 |
| tinwood | This was one of those occasions! | 13:46 |
| *** armaan has quit IRC | 13:46 | |
| evrardjp | yeah, I've seen you coming by a few times, always discreet :) | 13:46 |
| tinwood | when I say "you" I, of course, mean "me" | 13:46 |
| evrardjp | haha :) | 13:47 |
| evrardjp | I understood :p | 13:47 |
| evrardjp | that seems a scary issue btw | 13:48 |
| evrardjp | Synchronize the nova API DB schema | 13:48 |
| tinwood | evrardjp, yes, I though so. It seems to be a permissions thing. I wanted to dig into it before assigning "blame" so-as-to-speak. | 13:48 |
| evrardjp | tinwood: please be aware we have patches going on in nova | 13:49 |
| evrardjp | we broke it | 13:49 |
| evrardjp | maybe they merged now | 13:49 |
| evrardjp | so might want to recheck if that's a new issue | 13:49 |
| tinwood | I'll do a recheck now to see. | 13:49 |
| tinwood | It is non-voting, but I like to see everything green! | 13:50 |
| *** gameon has joined #openstack-ansible | 13:50 | |
| *** pylot has quit IRC | 13:50 | |
| *** pylot has joined #openstack-ansible | 13:51 | |
| evrardjp | it was merged 50 minutes ago | 13:51 |
| evrardjp | so if your test was before... | 13:51 |
| gameon | Greetings all - I am looking at moving my OpenStack cluster and as part of it, will need to adjust the FQDNs of all hypervisors and controllers. Has anyone done this? I need to preserve the data and such that resides on the hyperconverged Ceph OSDs as part of the hypervisors. Is this (easily) possible using OSA? | 13:51 |
| tinwood | evrardjp, it was; I've just asked for a recheck; I'll report back. Thanks very much for your help; I feel like another piece of the jigsaw has clicked into place. | 13:52 |
| evrardjp | TIL about goss: https://github.com/aelsabbahy/goss/blob/master/docs/manual.md -- I find it more in-line with what we are doing than testinfra IMO. What do you think folks? hwoarang mgariepy mnaser odyssey4me cloudnull | 13:52 |
| *** dave-mccowan has quit IRC | 13:52 | |
| evrardjp | gameon: mmm | 13:52 |
| *** nigelmaddock has quit IRC | 13:53 | |
| evrardjp | just the fqdns? | 13:53 |
| evrardjp | were you using the fqdns for the external lb vip address? (I suppose so) | 13:53 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_glance master: tox.ini: Use common vars file for distibution installations https://review.openstack.org/575428 | 13:53 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_neutron master: tox.ini: Use common vars file for distibution installations https://review.openstack.org/575418 | 13:54 |
| *** Tahvok has quit IRC | 13:54 | |
| evrardjp | gameon: I won't lie, it will be impactful, but not from OSA side, from openstack side already. | 13:54 |
| gameon | At the moment, the IPs and FQDNs will be changing | 13:54 |
| evrardjp | oh the IPs too? | 13:54 |
| evrardjp | wow. | 13:54 |
| evrardjp | you like to make it harder :) | 13:54 |
| gameon | I could potentially retain the IPs, but the FQDNs would be xyz.ENV.xyz.xyz.tld - the ENV would change | 13:54 |
| evrardjp | if you retain the IP your life will be easier | 13:55 |
| evrardjp | because you won't break your rabbitmq cluster | 13:55 |
| evrardjp | and galera | 13:55 |
| evrardjp | and many others | 13:55 |
| gameon | IPs I could retain I think - it's just FQDNs - I suppose one workaround would be to setup CNAMEs to the existing records though I'm not really keen on that approach | 13:55 |
| evrardjp | fqdns are used for endpoints, so your openstack apis will be broken | 13:56 |
| evrardjp | (probably it depends on your config) | 13:56 |
| evrardjp | the other case is for agents that run with your fqdn | 13:56 |
| evrardjp | it's not gonna be simple, but I did it in the past, and it was no real big deal. | 13:56 |
| evrardjp | just things to take care of | 13:57 |
| evrardjp | rabbitmq/neutron agents/cinder agents/... | 13:57 |
| *** pylot has quit IRC | 13:57 | |
| *** pylot has joined #openstack-ansible | 13:57 | |
| gameon | OK - sounds like I'll need to plan it through. Did you use OSA or was it more manually configuring each component? | 13:58 |
| guilhermesp | gameon: specially ceph monitors... changing the ceph monitors ip mapping wound't be good http://docs.ceph.com/docs/luminous/rados/operations/add-or-rm-mons/#changing-a-monitor-s-ip-address | 13:58 |
| gameon | Thank you both - IPs will have to be retained in that case | 13:59 |
| mgariepy | and also this when changing ips of ceph mons : https://bugs.launchpad.net/nova/+bug/1452641 | 13:59 |
| openstack | Launchpad bug 1452641 in OpenStack Compute (nova) "Static Ceph mon IP addresses in connection_info can prevent VM startup" [Medium,Confirmed] - Assigned to Corey Bryant (corey.bryant) | 13:59 |
| *** pylot has quit IRC | 14:00 | |
| *** shardy has quit IRC | 14:01 | |
| *** Tahvok has joined #openstack-ansible | 14:02 | |
| gameon | That's really useful all - thank you again, I'll give it some thought, fortunatley its a way off | 14:03 |
| *** smorant1 has joined #openstack-ansible | 14:14 | |
| *** smorant has quit IRC | 14:14 | |
| *** smorant has joined #openstack-ansible | 14:14 | |
| *** biffi has quit IRC | 14:16 | |
| *** smorant1 has quit IRC | 14:18 | |
| *** cjloader has joined #openstack-ansible | 14:19 | |
| *** vnogin has quit IRC | 14:22 | |
| *** vnogin has joined #openstack-ansible | 14:23 | |
| *** dave-mccowan has joined #openstack-ansible | 14:25 | |
| *** tux_ has joined #openstack-ansible | 14:34 | |
| mnaser | looks cool evrardjp | 14:36 |
| mnaser | fyi i am working with infra to get a hold on a ceph deploy node that is failing with out of disk | 14:37 |
| mnaser | and trying to replicate/figure out whats going on | 14:37 |
| *** nurdie has joined #openstack-ansible | 14:38 | |
| *** throwsb1 has joined #openstack-ansible | 14:41 | |
| *** hamzy_ is now known as hamzy | 14:44 | |
| mnaser | yaaaay | 14:46 |
| mnaser | hwoarang: https://review.openstack.org/#/c/575309/ :D | 14:46 |
| mnaser | i'll update rdo | 14:46 |
| hwoarang | looks promising | 14:47 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_barbican master: tox.ini: Use common vars file for distibution installations https://review.openstack.org/575452 | 14:57 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_aodh master: tox.ini: Use common vars file for distibution installations https://review.openstack.org/575454 | 14:58 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_cinder master: tox.ini: Use common vars file for distibution installations https://review.openstack.org/575455 | 14:59 |
| *** epalper has quit IRC | 14:59 | |
| *** epalper has joined #openstack-ansible | 14:59 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_ceilometer master: tox.ini: Use common vars file for distibution installations https://review.openstack.org/575456 | 15:00 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_designate master: tox.ini: Use common vars file for distibution installations https://review.openstack.org/575457 | 15:00 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_heat master: tox.ini: Use common vars file for distibution installations https://review.openstack.org/575459 | 15:01 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-os_swift master: tox.ini: Use common vars file for distibution installations https://review.openstack.org/575460 | 15:03 |
| *** rmart04 has quit IRC | 15:04 | |
| *** vnogin has quit IRC | 15:06 | |
| *** dxiri has joined #openstack-ansible | 15:09 | |
| mnaser | hwoarang: https://review.rdoproject.org/r/#/c/14240/ :) | 15:11 |
| mnaser | we will have to switch our repos to install the rdo trunk packages (instead of using queens, which is the right thing to do) | 15:11 |
| mnaser | but we'll hopefully get uwsgi with that and then all of the other work you did for distro installs should hopefully "just work" | 15:11 |
| hwoarang | ok | 15:11 |
| *** radeks_ has joined #openstack-ansible | 15:14 | |
| *** sxc731 has quit IRC | 15:14 | |
| *** radeks has quit IRC | 15:14 | |
| *** admin0 has joined #openstack-ansible | 15:17 | |
| admin0 | hi guys .. as soon as I add the nfs line in the config, inventory breaks -- https://gist.github.com/a1git/00cb386302d8906e845a0406450d738e -- what might i be doing wrong there .. | 15:18 |
| admin0 | i also tried the - { ip, share } format .. same result | 15:18 |
| *** jeferobles has joined #openstack-ansible | 15:19 | |
| *** sxc731 has joined #openstack-ansible | 15:23 | |
| *** pcaruana has quit IRC | 15:24 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-plugins master: Remove the dist_sort module https://review.openstack.org/575469 | 15:26 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_hosts master: SUSE: Add support for openSUSE Leap 15 https://review.openstack.org/573176 | 15:26 |
| *** DanyC has quit IRC | 15:28 | |
| hwoarang | mnaser: i agree with switching repos to rocky/master but i can't see ubuntu having one http://ubuntu-cloud.archive.canonical.com/ubuntu/dists/xenial-updates/ | 15:33 |
| hwoarang | but i might be looking at the wrong place | 15:33 |
| hwoarang | suse provides one as well so we are good | 15:33 |
| cloudnull | admin0 that looks right | 15:33 |
| mnaser | i think ubuntu has one, or used to, i need to look for it | 15:33 |
| cloudnull | admin0 I may be missing somee context, but what failing ? | 15:34 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add task to install requirements and change pathing https://review.openstack.org/575224 | 15:35 |
| admin0 | when i run setup hosts, it gets: * Failed to parse /opt/openstack-ansible/inventory/dynamic_inventory.py with ini plugin: /opt/openstack- | 15:36 |
| admin0 | ansible/inventory/dynamic_inventory.py:18: Expected key=value host variable assignment, got: argparse | 15:36 |
| logan- | mnaser hwoarang: only for bionic http://ubuntu-cloud.archive.canonical.com/ubuntu/dists/bionic-updates/ | 15:36 |
| admin0 | as soon as i # the nfs part .. | 15:36 |
| hwoarang | logan-: ohh :( shame. maybe it's an oversight | 15:36 |
| logan- | i think we had the same problem in newton | 15:37 |
| openstackgerrit | Jimmy McCrory proposed openstack/openstack-ansible stable/ocata: Fix newton workaround variables https://review.openstack.org/575270 | 15:37 |
| hwoarang | i doubt that they will drop xenial from rocky :/ | 15:38 |
| hwoarang | but who knows | 15:38 |
| logan- | heh https://github.com/openstack/openstack-ansible-pip_install/blob/newton-eol/vars/ubuntu-14.04.yml#L17 | 15:39 |
| logan- | https://github.com/openstack/openstack-ansible-pip_install/blob/newton-eol/vars/ubuntu-16.04.yml#L17 | 15:39 |
| *** sawblade6 has joined #openstack-ansible | 15:47 | |
| hwoarang | :( | 15:48 |
| *** lihi has quit IRC | 15:50 | |
| cloudnull | admin0 mind running the dynamic_inventory.py script ? | 15:50 |
| cloudnull | is there a stacktrace? | 15:50 |
| *** lihi has joined #openstack-ansible | 15:52 | |
| *** jeferobles has quit IRC | 15:53 | |
| *** Leo_m has joined #openstack-ansible | 15:56 | |
| hwoarang | good https://gitlab.freedesktop.org/spice/spice-html5 returns 503. the perks of moving way from github to self-hosting | 15:56 |
| mnaser | ugh | 15:58 |
| mnaser | seriously | 15:58 |
| mnaser | >:( | 15:58 |
| logan- | been at least an hour | 15:58 |
| logan- | i had a job fail an hour ago because of that :/ | 15:59 |
| *** chyka has joined #openstack-ansible | 16:00 | |
| *** DanyC has joined #openstack-ansible | 16:02 | |
| *** cshen has quit IRC | 16:04 | |
| *** DanyC has joined #openstack-ansible | 16:05 | |
| *** chyka has quit IRC | 16:07 | |
| admin0 | cloudnull, here: https://gist.github.com/a1git/c95923bd2d6cc3e8d315a701b8ab0bef | 16:14 |
| admin0 | oh | 16:15 |
| admin0 | i see it | 16:15 |
| admin0 | tab :( | 16:15 |
| *** smorant has quit IRC | 16:17 | |
| *** vurtz has joined #openstack-ansible | 16:19 | |
| *** gyee has joined #openstack-ansible | 16:19 | |
| *** udesale has joined #openstack-ansible | 16:20 | |
| *** udesale has quit IRC | 16:28 | |
| *** yiorgos_272 has joined #openstack-ansible | 16:32 | |
| *** sxc731 has quit IRC | 16:35 | |
| prometheanfire | everything in the group_vars/all/ folder can be accessed by all roles? or just group_vars/all/all.yml, iirc it's the former | 16:36 |
| *** poul_ has quit IRC | 16:36 | |
| *** sgrasley has quit IRC | 16:42 | |
| logan- | the former | 16:43 |
| *** sgrasley has joined #openstack-ansible | 16:43 | |
| *** pvradu has quit IRC | 16:46 | |
| prometheanfire | cool, thanks for confirming | 16:46 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Change openjdk from 8 to 9 https://review.openstack.org/575496 | 16:51 |
| evrardjp | cloudnull: mnaser maybe we should ask infra to mirror this one | 16:53 |
| evrardjp | what do you think? | 16:54 |
| mnaser | for spice-html5? | 16:54 |
| mnaser | i dunno if they ever mirrored git repos | 16:54 |
| evrardjp | it's probably not a bad idea to do so. | 16:54 |
| evrardjp | who knows what will happen to gnocchixyz? : p | 16:55 |
| mnaser | bahaha | 16:55 |
| mnaser | okay let me ask | 16:55 |
| evrardjp | yeah or novnc | 16:55 |
| evrardjp | if both go away, we are done. | 16:55 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Change openjdk from 8 to 9 https://review.openstack.org/575496 | 16:55 |
| evrardjp | both = novnc and spice | 16:56 |
| ansmith | evrardjp: hello, just wanted to check back re. the zun/qdrouterd base jobs | 17:03 |
| openstackgerrit | Merged openstack/openstack-ansible master: Bind internal services to internal IPs only https://review.openstack.org/569141 | 17:04 |
| evrardjp | I am sorry I didn't get the chance yesterday evening, nor today yet. | 17:04 |
| evrardjp | I should stop working and do that. | 17:04 |
| ansmith | evrardjp: or is that something I can help with, not sure what it entails | 17:04 |
| *** fghaas has joined #openstack-ansible | 17:05 | |
| evrardjp | nothing much, you did all the work already apparently. | 17:08 |
| evrardjp | the rest is done with infra. | 17:09 |
| evrardjp | so we can just bring the thing | 17:09 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Change openjdk from 8 to 9 https://review.openstack.org/575496 | 17:09 |
| *** gkadam__ is now known as gkadam | 17:10 | |
| *** chhavi__ has quit IRC | 17:15 | |
| openstackgerrit | Cam J. Loader proposed openstack/openstack-ansible master: Make more glance vars accessible globally https://review.openstack.org/575504 | 17:16 |
| *** DanyC has quit IRC | 17:18 | |
| evrardjp | ansmith: we'll see what https://review.openstack.org/#/c/575505/1 gives. | 17:21 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add alias and rollups to the curator https://review.openstack.org/575507 | 17:23 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_zun master: Add zuul jobs https://review.openstack.org/574139 | 17:27 |
| *** sawblade6 has quit IRC | 17:30 | |
| mnaser | evrardjp, cloudnull: if you're around, i'd like you to comment on https://review.openstack.org/#/c/575273/ | 17:30 |
| *** dave-mccowan has quit IRC | 17:30 | |
| cloudnull | mnaser lgtm | 17:31 |
| mnaser | cloudnull: hwoarang had some comments | 17:31 |
| mnaser | not sure what you thin kabout them | 17:31 |
| evrardjp | ok | 17:31 |
| * mnaser personally prefers going to upstream repo rather than epel | 17:31 | |
| mnaser | and if we start seeing timeouts, we can get infra to mirror things | 17:31 |
| evrardjp | mnaser: is it only for keystone we'll need that? | 17:32 |
| mnaser | evrardjp: we'll probably need it for repo_server too | 17:32 |
| mnaser | and thats it | 17:32 |
| evrardjp | ok | 17:32 |
| evrardjp | it doesn't pass centos7 though | 17:33 |
| evrardjp | oh still uwsgi | 17:33 |
| *** epalper has quit IRC | 17:34 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_nova master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575286 | 17:35 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_nova master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575286 | 17:36 |
| *** pvradu has joined #openstack-ansible | 17:37 | |
| *** gameon has quit IRC | 17:42 | |
| *** pcaruana has joined #openstack-ansible | 17:43 | |
| *** sawblade6 has joined #openstack-ansible | 17:44 | |
| *** radeks_ has quit IRC | 17:49 | |
| openstackgerrit | Cam J. Loader proposed openstack/openstack-ansible-os_glance master: Remove vars from defaults/main.yml https://review.openstack.org/575513 | 17:54 |
| *** cjloader_ has joined #openstack-ansible | 18:00 | |
| *** cjloader has quit IRC | 18:00 | |
| *** poopcat has joined #openstack-ansible | 18:01 | |
| *** pcaruana has quit IRC | 18:01 | |
| *** mbuil has quit IRC | 18:02 | |
| *** pcaruana has joined #openstack-ansible | 18:06 | |
| *** electrofelix has quit IRC | 18:19 | |
| openstackgerrit | Mohammed Naser proposed openstack/openstack-ansible-os_panko master: Switch OpenSUSE jobs to be voting https://review.openstack.org/575285 | 18:26 |
| *** cjloader_ has quit IRC | 18:27 | |
| mnaser | https://review.openstack.org/#/q/topic:remove-nv-jobs | 18:28 |
| mnaser | some easy love | 18:28 |
| *** cjloader has joined #openstack-ansible | 18:28 | |
| *** cjloader has quit IRC | 18:30 | |
| *** cjloader has joined #openstack-ansible | 18:30 | |
| *** hachi has joined #openstack-ansible | 18:45 | |
| *** hachi has quit IRC | 18:48 | |
| *** pvradu has quit IRC | 18:48 | |
| *** dave-mccowan has joined #openstack-ansible | 18:52 | |
| *** sawblade6 has quit IRC | 19:01 | |
| *** sawblade6 has joined #openstack-ansible | 19:06 | |
| openstackgerrit | Antony Messerli proposed openstack/openstack-ansible-ops master: Adds support for leaping from Mitaka https://review.openstack.org/575527 | 19:06 |
| jrosser | cloudnull: this can't be "latest" so fails an upgrade https://github.com/openstack/openstack-ansible-ops/blob/master/elk_metrics_6x/common_task_install_elk_repo.yml#L19 | 19:11 |
| openstackgerrit | Cam J. Loader proposed openstack/openstack-ansible-os_ironic master: Add glance auth information to ironic.conf for communication https://review.openstack.org/573822 | 19:18 |
| *** cshen has joined #openstack-ansible | 19:24 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_monasca master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575289 | 19:34 |
| admin0 | hi guys .. i configured a iscsi for cinder and when i try to create a new volume, i get: Filtering removed all hosts for the request with volume ID 'ae536b80-3ee7-4f87-b021-e1d7b162454a'. Filter results: AvailabilityZoneFilter: (start: 3, end: 3), CapacityFilter: (start: 3, end: 3), CapabilitiesFilter: (start: 3, end: 0) | 19:37 |
| admin0 | 19:37 | |
| admin0 | what/where is this CapabilitiesFilter: coming from ? | 19:38 |
| admin0 | started in 17.0.5 | 19:38 |
| *** fghaas has quit IRC | 19:46 | |
| *** gkadam has quit IRC | 19:48 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_glance master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575294 | 19:48 |
| openstackgerrit | Merged openstack/openstack-ansible-os_designate master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575296 | 19:48 |
| openstackgerrit | Merged openstack/openstack-ansible-os_heat master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575293 | 19:48 |
| openstackgerrit | Merged openstack/openstack-ansible-os_masakari master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575290 | 19:48 |
| openstackgerrit | Merged openstack/openstack-ansible-os_monasca-agent master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575288 | 19:48 |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575287 | 19:48 |
| openstackgerrit | Merged openstack/openstack-ansible-os_blazar master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575300 | 19:48 |
| openstackgerrit | Merged openstack/openstack-ansible-os_searchlight master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575283 | 19:49 |
| openstackgerrit | Merged openstack/openstack-ansible-os_aodh master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575301 | 19:49 |
| openstackgerrit | Andy Smith proposed openstack/openstack-ansible-tests master: Add qdrouterd role https://review.openstack.org/575537 | 19:53 |
| openstackgerrit | Cam J. Loader proposed openstack/openstack-ansible-os_glance master: Remove vars from defaults/main.yml https://review.openstack.org/575513 | 19:56 |
| openstackgerrit | Cam J. Loader proposed openstack/openstack-ansible-os_ironic master: Add glance auth information to ironic.conf for communication https://review.openstack.org/573822 | 20:00 |
| *** tasker has joined #openstack-ansible | 20:02 | |
| tasker | hi. so, spice-html5 moved their upstream repo from github to git.freedesktop.org. are there similar overrides for spice_html5 to those available for nova, cinder, et al: nova_git_repo, nova_git_install_branch, magnum_git_repo, octavia_git_repo ? | 20:05 |
| *** vurtz has quit IRC | 20:09 | |
| openstackgerrit | Cam J. Loader proposed openstack/openstack-ansible master: Make more glance vars accessible globally https://review.openstack.org/575504 | 20:09 |
| ivve | yo guys, are there any override variables available for replacing *.svg files in horizon (splash login logo and the top openstack logo) | 20:14 |
| ivve | nevermind found it! :) | 20:15 |
| openstackgerrit | Merged openstack/openstack-ansible-os_swift master: Switch non-voting jobs to use Zuul stanza https://review.openstack.org/575282 | 20:18 |
| tasker | ivve: where was it? I'd like to use that too. | 20:18 |
| *** cmart has joined #openstack-ansible | 20:19 | |
| ivve | https://docs.openstack.org/openstack-ansible-os_horizon/pike/ | 20:19 |
| ivve | horizon_custom_uploads | 20:19 |
| tasker | ahh -- my spice overrides are: nova_spicehtml5_git_repo, nova_spicehtml5_git_install_branch | 20:19 |
| ivve | im guessing just cram it into user_vars and os-horizon pb | 20:20 |
| ivve | testing, just moving the custom.svg files | 20:20 |
| cmart | evrardjp, you around? | 20:20 |
| tasker | or is it spicehtml5_git_repo, spicehtml5_git_install_branch? | 20:21 |
| throwsb1 | Hi, I am getting the following FAILED on new build on queens "fatal: [infra1_keystone_container-e3370511]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false, "failed": true}" | 20:23 |
| throwsb1 | is this a know issue | 20:24 |
| ivve | hmm actually user_vars didn't work | 20:24 |
| ivve | guessing i have to modify the original horizon yaml | 20:24 |
| jamesdenton | throwsb1 yes | 20:25 |
| jamesdenton | I experienced that issue, and its related to the healthcheck applied to the Galera VIP. | 20:25 |
| jamesdenton | update haproxy.conf and comment out the healthcheck and restart haproxy. | 20:25 |
| throwsb1 | jamesdenton: thanks. let me try. | 20:26 |
| jamesdenton | galera is unavailable through the vip, essentially. | 20:26 |
| throwsb1 | is this a new issue? | 20:26 |
| jamesdenton | I don't know how widespread it is. I saw it over a week ago but didn't open a bug. | 20:27 |
| ivve | nvm im stupid again, just errors in syntax :D | 20:27 |
| jamesdenton | There's an HTTP HEAD check applied to the backend node. While it works with curl, haproxy doesn't like it. That was my experience anyway. | 20:28 |
| throwsb1 | jamesdenton: ok. I am working with 2 diff environments the first I had no issues and it is on 17.0.4 or 5 in a cluster build. this new build is not clustered and running the latest. | 20:28 |
| throwsb1 | ok | 20:29 |
| jamesdenton | Hmm. Mine was an AIO, though i wouldn't think that would cause a difference. | 20:29 |
| throwsb1 | I wouldn't think | 20:29 |
| throwsb1 | this new one is 2 servers. infra and compute | 20:30 |
| jamesdenton | I guess you can figure it out real quick - try to hit galera via the vip and see if it response. | 20:30 |
| jamesdenton | *responds | 20:30 |
| *** ansmith has quit IRC | 20:31 | |
| *** poopcat has quit IRC | 20:31 | |
| throwsb1 | I ran the galera playbook check from deploy server and it was fine | 20:32 |
| throwsb1 | which health check did you disable? galera or keystone? | 20:33 |
| jamesdenton | can you telnet to the vbip on 3306? | 20:33 |
| jamesdenton | i found in the haproxy log that galera backend node was down and had to disable the check on the galera backend config. | 20:34 |
| *** Talion has joined #openstack-ansible | 20:34 | |
| *** poopcat has joined #openstack-ansible | 20:35 | |
| throwsb1 | yea it is down. also hatop is showing down | 20:36 |
| jamesdenton | k | 20:37 |
| jamesdenton | so there's a good chance its failing that check. i would just comment it out or change it to tcp | 20:37 |
| throwsb1 | I can telnet direct to the server container on 3306 | 20:38 |
| throwsb1 | definitely lb issue | 20:38 |
| jamesdenton | Not really sure why it's happening, to be honest. And i'm not sure why HTTP HEAD over L4 in this case | 20:39 |
| throwsb1 | that seems to do it | 20:40 |
| throwsb1 | I will look at the healthcheck on other install to see if something changed | 20:40 |
| jamesdenton | i have another deploy w/ master, same check, and it works | 20:40 |
| jamesdenton | *shrug* | 20:40 |
| throwsb1 | AIO ? | 20:40 |
| throwsb1 | what version of OSA? | 20:41 |
| throwsb1 | hmmmm, haproxy looks the same | 20:45 |
| throwsb1 | same between 2 environments | 20:45 |
| jamesdenton | the one i had issues with was 17.0.5 | 20:50 |
| *** tux_ has quit IRC | 20:50 | |
| jamesdenton | stable/queens at the time | 20:50 |
| jamesdenton | yeah, it's weird. | 20:50 |
| throwsb1 | jamesdenton: thanks for the pointer. I searched the treads for this channel and there were 2 mentions on 5/11 and 5/12 | 20:50 |
| *** tux_ has joined #openstack-ansible | 20:50 | |
| throwsb1 | no fix | 20:50 |
| jamesdenton | you might find me asking about the healthcheck last week w/ no corresponding error. | 20:51 |
| jamesdenton | looks like a legit issue though. can raise a bug | 20:51 |
| throwsb1 | yea | 20:52 |
| throwsb1 | if the last playbook deploy finishes successfully, I will try to uncomment out and restart and see if it returns. I am just wondering if a restart of ha, without commenting it out fixed it | 20:53 |
| jrosser | i've got something similar going on with horizon healthcheck now no longer working when it used to | 20:53 |
| jrosser | which i really don't understand | 20:53 |
| throwsb1 | hmmm, mine was on keystone. | 20:54 |
| jrosser | i have deleted/recreated my horizon containers (for other reasons) and it's never liked it since that | 20:54 |
| throwsb1 | can you connect to galera through external vip 3306? | 20:54 |
| throwsb1 | or just look at hatop and galera backend? | 20:54 |
| *** tux_ has quit IRC | 20:55 | |
| Talion | Hello, is there any deployment model mostly/commonly chosen for production ? | 20:58 |
| admin0 | if cinder has nfs, does the cinder volume containers also mount the nfs partition ? | 20:58 |
| cloudnull | jrosser ah good catch | 21:12 |
| cloudnull | updating | 21:12 |
| tasker | is there a way to turn on `neutron_fwaas_dashboard` via OSA? | 21:13 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Change openjdk from 8 to 9 https://review.openstack.org/575496 | 21:15 |
| jrosser | cloudnull: still going with it - not managed to get it all to hang together quite yet | 21:16 |
| cloudnull | tasker yes you should be set that option in a user variables file | 21:17 |
| tasker | okay. what's the option? | 21:17 |
| cloudnull | all of the required options will be enabled if fwaas is part of the neutron plugin base | 21:18 |
| cloudnull | https://github.com/openstack/openstack-ansible-os_neutron/blob/master/defaults/main.yml#L283-L296 | 21:19 |
| cloudnull | thats the option set | 21:19 |
| cloudnull | so if you just need fwaasV2 you can set that in the base list and run the playbooks to enable the various components for fwaas | 21:20 |
| *** throwsb1 has quit IRC | 21:20 | |
| cloudnull | jrosser for elk 6.3 or the change to use smartLBs ? | 21:20 |
| jrosser | maybe i did too much at once! | 21:21 |
| jrosser | elk 6.3 + openjdk9 = fail | 21:21 |
| jrosser | something stops the service starting with a java exception, and the log isnt terribly clear what is wrong | 21:21 |
| cloudnull | interesting | 21:22 |
| jrosser | and to back out of that it doesnt seem quite as simple as apt purge the version 9 stuff | 21:22 |
| cloudnull | Im running through this now myself, ill see if i get the same results. | 21:22 |
| jrosser | there is residual crud which means elasticsearch still wont start with the same error | 21:23 |
| *** ansmith has joined #openstack-ansible | 21:23 | |
| jrosser | so i've gone round one by one delete/create/install on the elasticsearch containers and it seems to be coming up better with 6.3 + openjdk8 | 21:24 |
| cloudnull | jdk9 is a short term relaese | 21:24 |
| cloudnull | so maybe its better to keep jdk8 for now | 21:24 |
| jrosser | it feels like the cluster is sufficiently complex now with 3 log nodes and 3 kibana nodes that some overview is needed | 21:25 |
| jrosser | theres a lot of places to go look and check that everything has discovered everything else | 21:25 |
| *** mriedem has joined #openstack-ansible | 21:27 | |
| mriedem | https://review.openstack.org/#/c/575153/ to add the upgrade job back in passed the check queue but didn't go into gate queue, might need another core to +W it to kick it into the gate queue | 21:28 |
| mriedem | sometimes zuul is funny | 21:28 |
| *** mriedem has left #openstack-ansible | 21:29 | |
| *** mriedem has joined #openstack-ansible | 21:29 | |
| *** jwitko has quit IRC | 21:29 | |
| mriedem | so, the nova-status upgrade check tool is doing it's job it looks like: | 21:31 |
| mriedem | https://review.openstack.org/#/c/575125/5 | 21:31 |
| mriedem | Details: Placement API version 1.24 needed, you have 1.17. | 21:31 |
| mriedem | is OSA upgrading placement after upgrading nova? or not at all? | 21:31 |
| *** DanyC has joined #openstack-ansible | 21:31 | |
| mriedem | because placement needs to be upgraded before nova | 21:32 |
| *** cjloader_ has joined #openstack-ansible | 21:32 | |
| *** cjloader has quit IRC | 21:32 | |
| mriedem | from grenade | 21:33 |
| mriedem | https://github.com/openstack-dev/grenade/blob/master/projects/60_nova/upgrade.sh#L90 | 21:33 |
| mriedem | maybe this is the problem? https://github.com/openstack/openstack-ansible-os_nova/blob/eda296a37ea3f2fd8ffbafa2da573573c5c98068/defaults/main.yml#L506 | 21:36 |
| mriedem | placement isn't first in the start order | 21:36 |
| *** cjloader_ has quit IRC | 21:37 | |
| *** prometheanfire has quit IRC | 21:43 | |
| *** nurdie has quit IRC | 21:44 | |
| *** dave-mccowan has quit IRC | 21:45 | |
| *** nurdie has joined #openstack-ansible | 21:48 | |
| *** cshen has quit IRC | 21:49 | |
| jrosser | cloudnull: it lives! | 21:50 |
| cloudnull | jrosser nice! | 21:52 |
| cloudnull | I have a diagram I created for the basic elk things - https://drive.google.com/file/d/1Z9081k4t8hW-_XQtztxhRZ6WVhPyX6KN/view?usp=sharing - Just kinda helped me grok what I was working toward with the smart LB changes. | 21:53 |
| *** nurdie has quit IRC | 21:53 | |
| cloudnull | IDK if thats of any use to folks looking at the cluster and trying to understand the components | 21:54 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Change openjdk from 8 to 9 https://review.openstack.org/575496 | 21:56 |
| *** cmart has quit IRC | 21:57 | |
| *** jwitko has joined #openstack-ansible | 22:00 | |
| *** tasker has quit IRC | 22:03 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-ops master: Add upgrade task options https://review.openstack.org/575496 | 22:03 |
| *** jbadiapa_ has joined #openstack-ansible | 22:03 | |
| jrosser | cloudnull: theres a handy "monitoring" tab in kibana now which gives a simple good/bad cluster health overview | 22:04 |
| cloudnull | jrosser is your cluster online using the new topology ? | 22:04 |
| cloudnull | that's very nice ! | 22:04 |
| jrosser | as far as i can tell | 22:04 |
| cloudnull | a monitoring tab would be great, right now I have a collection of curl commands :) | 22:05 |
| jrosser | i just redid haproxy to point to the kibana nodes rather than the elasticsearch ones | 22:05 |
| jrosser | and i now see kibana 6.3 rather than my old one | 22:05 |
| cloudnull | cool! | 22:05 |
| jrosser | and i've rerun the whole of site.yml so the beats should be set up in the new way too | 22:05 |
| openstackgerrit | Matt Riedemann proposed openstack/openstack-ansible-os_nova master: Fix start order for nova-placement-api service https://review.openstack.org/575558 | 22:05 |
| *** jbadiapa has quit IRC | 22:06 | |
| cloudnull | excellent ! | 22:06 |
| *** jwitko has quit IRC | 22:06 | |
| *** sawblade6 has quit IRC | 22:08 | |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Add task to install requirements and change pathing https://review.openstack.org/575224 | 22:11 |
| *** DanyC has quit IRC | 22:23 | |
| *** nicolasbock has quit IRC | 22:34 | |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Use elasticsearch coordinator nodes as smart LBs https://review.openstack.org/574870 | 22:37 |
| *** Leo_m has quit IRC | 22:40 | |
| *** jbadiapa_ has quit IRC | 22:46 | |
| *** tosky has quit IRC | 22:49 | |
| *** DanyC has joined #openstack-ansible | 22:54 | |
| *** threestrands has joined #openstack-ansible | 22:56 | |
| *** threestrands has quit IRC | 22:56 | |
| *** threestrands has joined #openstack-ansible | 22:56 | |
| *** threestrands has quit IRC | 22:57 | |
| *** threestrands has joined #openstack-ansible | 22:57 | |
| mriedem | during an upgrade job run, are the service logs all in systemd? | 22:58 |
| mriedem | it appears that either nova_api_placement either isn't getting upgraded to master (should be while it's pip installed from nova master), | 22:58 |
| *** DanyC has quit IRC | 22:58 | |
| mriedem | or it's not getting restarted, but it should be since it's part of nova_all | 22:59 |
| mriedem | oh nvm http://logs.openstack.org/25/575125/5/check/openstack-ansible-upgrade-ubuntu-xenial/222b6ac/logs/openstack/openstack1/nova/nova-placement-api.log.txt.gz | 23:00 |
| mriedem | not sure how i missed those | 23:00 |
| *** jbadiapa_ has joined #openstack-ansible | 23:02 | |
| openstackgerrit | Matt Riedemann proposed openstack/openstack-ansible-os_nova master: Add execreloads for nova-placement-api https://review.openstack.org/575566 | 23:04 |
| openstackgerrit | Matt Riedemann proposed openstack/openstack-ansible-os_nova master: Add execreloads for nova-placement-api https://review.openstack.org/575566 | 23:04 |
| *** prometheanfire has joined #openstack-ansible | 23:28 | |
| *** cshen has joined #openstack-ansible | 23:44 | |
| *** markvoelker has quit IRC | 23:46 | |
| *** dxiri has quit IRC | 23:47 | |
| *** dxiri has joined #openstack-ansible | 23:48 | |
| *** cshen has quit IRC | 23:49 | |
| *** dxiri has quit IRC | 23:52 | |
| *** nurdie has joined #openstack-ansible | 23:54 | |
| *** DanyC has joined #openstack-ansible | 23:55 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!