| *** weezS has joined #openstack-ansible | 00:00 | |
| tasker | having a hard time finding this; how do I put haproxy in MAINT mode? and does that apply to the backend URLs or the frontend? | 00:05 |
|---|---|---|
| tasker | well, now I found this, https://serverfault.com/questions/249316/how-can-i-remove-balanced-node-from-haproxy-via-command-line | 00:07 |
| *** weezS has quit IRC | 00:10 | |
| tasker | but still, am i disabling the front-end of the endpoint or the back-end (the IP for container)? | 00:10 |
| *** yifei has joined #openstack-ansible | 00:18 | |
| *** gouthamr has joined #openstack-ansible | 00:18 | |
| tasker | do I want to keep the "lxc container data"? isn't this things like the name of the container and its IP? | 00:29 |
| *** marc_ab has quit IRC | 00:32 | |
| *** woodard has quit IRC | 00:35 | |
| *** woodard has joined #openstack-ansible | 00:35 | |
| *** thorst_afk has joined #openstack-ansible | 00:39 | |
| *** thorst_afk has quit IRC | 00:44 | |
| *** thorst_afk has joined #openstack-ansible | 00:47 | |
| *** thorst_afk has quit IRC | 00:47 | |
| *** thorst_afk has joined #openstack-ansible | 00:49 | |
| *** thorst_afk has quit IRC | 00:49 | |
| *** hw_wutianwei has joined #openstack-ansible | 00:55 | |
| *** chyka has joined #openstack-ansible | 00:59 | |
| *** zcourts has joined #openstack-ansible | 01:03 | |
| *** chyka has quit IRC | 01:04 | |
| *** zcourts_ has quit IRC | 01:05 | |
| *** huxinhui_ has joined #openstack-ansible | 01:13 | |
| *** zcourts has quit IRC | 01:21 | |
| *** vnogin has joined #openstack-ansible | 01:28 | |
| *** vnogin has quit IRC | 01:33 | |
| *** marc_ab has joined #openstack-ansible | 01:52 | |
| *** dxiri has joined #openstack-ansible | 01:52 | |
| *** thorst_afk has joined #openstack-ansible | 01:54 | |
| *** dxiri_ has quit IRC | 01:55 | |
| *** thorst_afk has quit IRC | 01:59 | |
| *** thorst_afk has joined #openstack-ansible | 02:05 | |
| *** thorst_afk has quit IRC | 02:05 | |
| *** pmannidi has joined #openstack-ansible | 02:08 | |
| *** esberglu has joined #openstack-ansible | 02:20 | |
| *** esberglu has quit IRC | 02:20 | |
| *** thegreenhundred has joined #openstack-ansible | 02:31 | |
| *** thorst_afk has joined #openstack-ansible | 02:36 | |
| *** thorst_afk has quit IRC | 02:41 | |
| cloudnull | hey tasker still around ? | 02:48 |
| cloudnull | did you figure out the answer to putting hap in maint mode? | 02:49 |
| tasker | ello | 03:07 |
| tasker | no. | 03:07 |
| tasker | and I think I've messed up deeper. | 03:07 |
| tasker | I'm doing "setup-everything" and the play "Galera container config" is failing at "install pip" with -- HTTPError: 404 Client Error: Not Found for url: http://haproxy:8181/os-releases/14.2.7/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt | 03:08 |
| tasker | so, I think maybe I ripped out the repo container too early or didn't put it back soon enough. | 03:09 |
| tasker | oh, geez. I just looked at the clock. | 03:09 |
| tasker | eh -- my cats are warriors. they'll survive; or eat each other. | 03:09 |
| *** esberglu has joined #openstack-ansible | 03:14 | |
| *** esberglu has quit IRC | 03:18 | |
| cloudnull | hahaha | 03:23 |
| cloudnull | if haproxy is down you will get that errir | 03:23 |
| cloudnull | **error | 03:23 |
| tasker | oh | 03:24 |
| cloudnull | down, or not routing traffic | 03:24 |
| tasker | curses, foiled | 03:25 |
| tasker | I thought it might because there was no repo container on the other end. | 03:25 |
| cloudnull | that also would cause it | 03:25 |
| cloudnull | you can check, | 03:25 |
| cloudnull | on the infra node. | 03:25 |
| cloudnull | lxc-ls -f | grep repo | 03:25 |
| tasker | just did | 03:26 |
| cloudnull | or from your deployment host, `ansible -m shell -a 'lxc-ls -f | grep repo' infra_all` | 03:26 |
| tasker | the file exists, so it's haproxy. | 03:26 |
| cloudnull | hatop is available on the hap host | 03:26 |
| tasker | hatop? | 03:26 |
| cloudnull | haptop -s /var/run/hatop.stat | 03:26 |
| cloudnull | or something similar | 03:26 |
| tasker | sweet. connection refused. | 03:27 |
| cloudnull | hatop is a CLI based management tool | 03:27 |
| tasker | ok, great! | 03:27 |
| tasker | I can take a look at this tomorrow. | 03:27 |
| tasker | it's 2230 here. | 03:27 |
| cloudnull | here too | 03:27 |
| cloudnull | :D | 03:27 |
| * cloudnull is in TX | 03:27 | |
| tasker | <- MO | 03:27 |
| tasker | are you at least safe or dry? | 03:27 |
| cloudnull | its scotch'o'clock in my book | 03:28 |
| cloudnull | yup. | 03:28 |
| cloudnull | in san antonio | 03:28 |
| tasker | that's a good idea. | 03:28 |
| cloudnull | we got a lot of wind and some rain but NOTHING like Houston | 03:28 |
| tasker | mostly sunny and highs of 80 for two weeks. | 03:29 |
| *** marc_ab has quit IRC | 03:29 | |
| tasker | it'S crazy awesome out here; almost spooky. | 03:29 |
| cloudnull | sounds pretty great | 03:29 |
| tasker | this is _not_ a missouri august. | 03:29 |
| tasker | alright cloud, I'm out. I'll be back tomorrow to whip some haproxy around. | 03:30 |
| tasker | thanks so much for your help! | 03:30 |
| cloudnull | np | 03:30 |
| cloudnull | have a good night | 03:30 |
| tasker | you as well. | 03:30 |
| tasker | -- hey, does socat need to be installed for hatop to work? | 03:31 |
| cloudnull | no. | 03:31 |
| cloudnull | there are two files. | 03:31 |
| cloudnull | i cant remember if its stat or sock | 03:31 |
| cloudnull | one sec I can check | 03:31 |
| tasker | .stat | 03:31 |
| tasker | and .pid | 03:32 |
| tasker | unless it's somewhere else entirely | 03:32 |
| cloudnull | hatop -s /var/run/haproxy.stat | 03:33 |
| cloudnull | https://snag.gy/rDzT03.jpg | 03:33 |
| tasker | oh that is cool. | 03:34 |
| tasker | after some searching, i mananged to enable the web stats page, but couldn't find a way to use it as management. | 03:34 |
| tasker | yeah - connection refused. I'll have to look into it, but tomorrow. | 03:34 |
| tasker | ok, seriously, I'm out now. | 03:35 |
| tasker | thanks, again! | 03:35 |
| cloudnull | take care | 03:35 |
| *** udesale has joined #openstack-ansible | 03:37 | |
| *** thorst_afk has joined #openstack-ansible | 03:38 | |
| openstackgerrit | Jimmy McCrory proposed openstack/openstack-ansible-specs master: Add spec for hybrid messaging https://review.openstack.org/499882 | 03:39 |
| *** gkadam_ has joined #openstack-ansible | 03:41 | |
| *** thorst_afk has quit IRC | 03:42 | |
| *** dxiri has quit IRC | 03:50 | |
| *** dxiri has joined #openstack-ansible | 03:50 | |
| *** gouthamr has quit IRC | 03:53 | |
| *** dxiri has quit IRC | 03:55 | |
| *** chhavi has joined #openstack-ansible | 04:11 | |
| *** germs has quit IRC | 04:20 | |
| *** galstrom_zzz is now known as galstrom | 04:21 | |
| *** chyka has joined #openstack-ansible | 04:27 | |
| *** chyka has quit IRC | 04:31 | |
| *** thorst_afk has joined #openstack-ansible | 04:38 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-specs master: Hyperconverged Containers https://review.openstack.org/499886 | 04:40 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible-specs master: Hyperconverged Containers https://review.openstack.org/499886 | 04:42 |
| *** thorst_afk has quit IRC | 04:43 | |
| *** galstrom is now known as galstrom_zzz | 04:44 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: [WIP] Further hyper-converge our container usage https://review.openstack.org/499396 | 04:53 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible stable/pike: Fix LXC container start order https://review.openstack.org/499888 | 04:53 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible stable/ocata: Fix LXC container start order https://review.openstack.org/499890 | 04:55 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible stable/newton: Fix LXC container start order https://review.openstack.org/499891 | 04:56 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Run gate playbooks in parallel https://review.openstack.org/497742 | 04:56 |
| *** pmannidi has quit IRC | 04:59 | |
| *** DanyC has joined #openstack-ansible | 05:22 | |
| *** thorst_afk has joined #openstack-ansible | 05:39 | |
| *** thorst_afk has quit IRC | 05:44 | |
| *** marst has joined #openstack-ansible | 05:52 | |
| *** DanyC has quit IRC | 05:55 | |
| *** marst has quit IRC | 05:57 | |
| *** esberglu has joined #openstack-ansible | 05:59 | |
| *** esberglu has quit IRC | 06:03 | |
| *** Oku_OS-away is now known as Oku_OS | 06:11 | |
| *** drifterza has joined #openstack-ansible | 06:11 | |
| *** pcaruana has joined #openstack-ansible | 06:17 | |
| *** thorst_afk has joined #openstack-ansible | 06:40 | |
| *** thorst_afk has quit IRC | 06:45 | |
| *** mbuil has joined #openstack-ansible | 06:48 | |
| *** mohit048 has joined #openstack-ansible | 06:51 | |
| *** esberglu has joined #openstack-ansible | 06:52 | |
| *** esberglu has quit IRC | 06:53 | |
| *** esberglu has joined #openstack-ansible | 06:53 | |
| *** esberglu has quit IRC | 06:53 | |
| *** esberglu has joined #openstack-ansible | 06:54 | |
| *** maybebuggy has joined #openstack-ansible | 06:54 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible stable/pike: [DOC} Add openSUSE to Software requirements https://review.openstack.org/499913 | 06:56 |
| *** esberglu has quit IRC | 06:58 | |
| *** shardy has joined #openstack-ansible | 07:09 | |
| hwoarang | good morning | 07:32 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible stable/pike: [DOC] Add openSUSE Leap references https://review.openstack.org/499920 | 07:35 |
| *** DanyC has joined #openstack-ansible | 07:37 | |
| *** DanyC has quit IRC | 07:40 | |
| *** DanyC has joined #openstack-ansible | 07:40 | |
| *** thorst_afk has joined #openstack-ansible | 07:41 | |
| *** thorst_afk has quit IRC | 07:45 | |
| *** vnogin has joined #openstack-ansible | 08:00 | |
| *** drifterza has quit IRC | 08:03 | |
| *** chyka has joined #openstack-ansible | 08:03 | |
| *** chyka has quit IRC | 08:08 | |
| neith | morning | 08:09 |
| *** qiliang27 has joined #openstack-ansible | 08:15 | |
| odyssey4me | o/ | 08:15 |
| odyssey4me | oh nice, ocata/pike/master all passed the deploy aio periodics this morning | 08:15 |
| odyssey4me | unfortunately the upgrades have not passed | 08:15 |
| *** liangqi has joined #openstack-ansible | 08:16 | |
| *** drifterza has joined #openstack-ansible | 08:18 | |
| DimGR | odyssey4me remember me asking few days ago how to upgrade magnum ocata to magnum pike ? well i did it and it works excellent | 08:26 |
| odyssey4me | DimGR oh, great :) | 08:26 |
| odyssey4me | you should blog post how to do it ;) | 08:26 |
| DimGR | i sent a patch explaining this to magnum for review | 08:27 |
| DimGR | not only magnum pike solved major issues like dns and cinder that were useless in magnum ocata , but the role although for pike built with no errors on ocata | 08:28 |
| DimGR | so its a success | 08:28 |
| *** drifterza has quit IRC | 08:31 | |
| *** drifterza has joined #openstack-ansible | 08:32 | |
| *** thorst_afk has joined #openstack-ansible | 08:42 | |
| *** drifterza has quit IRC | 08:44 | |
| *** thorst_afk has quit IRC | 08:46 | |
| *** jafeha has joined #openstack-ansible | 08:49 | |
| *** electrofelix has joined #openstack-ansible | 08:50 | |
| openstackgerrit | Taseer Ahmed proposed openstack/openstack-ansible-specs master: (WIP) Blueprint for Congress integration with OSA https://review.openstack.org/499066 | 08:53 |
| *** drifterza has joined #openstack-ansible | 08:56 | |
| *** vnogin has quit IRC | 09:08 | |
| *** vnogin has joined #openstack-ansible | 09:09 | |
| *** udesale__ has joined #openstack-ansible | 09:27 | |
| *** yifei has quit IRC | 09:27 | |
| *** udesale has quit IRC | 09:27 | |
| *** udesale has joined #openstack-ansible | 09:29 | |
| *** udesale__ has quit IRC | 09:31 | |
| andymccr | looking into the hardening failures on AIOs | 09:35 |
| *** vnogin has quit IRC | 09:40 | |
| *** udesale has quit IRC | 09:40 | |
| *** udesale has joined #openstack-ansible | 09:41 | |
| *** thorst_afk has joined #openstack-ansible | 09:42 | |
| *** udesale has quit IRC | 09:44 | |
| *** thorst_afk has quit IRC | 09:47 | |
| odyssey4me | andymccr I expect that perhaps https://review.openstack.org/499609 may need to be ported back all the way to ocata? | 09:48 |
| andymccr | odyssey4me: yeah lemme take care of that! | 09:48 |
| openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-lxc_hosts stable/pike: For CentOS ensure lxc-net is started and enabled https://review.openstack.org/499949 | 09:48 |
| openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-lxc_hosts stable/ocata: For CentOS ensure lxc-net is started and enabled https://review.openstack.org/499950 | 09:49 |
| andymccr | odyssey4me: maybe not | 09:51 |
| andymccr | it looks like it broke things | 09:51 |
| andymccr | http://logs.openstack.org/57/499557/2/check/gate-openstack-ansible-openstack-ansible-ceph-centos-7-nv/1174ad1/console.html#_2017-09-01_09_33_34_035114 | 09:52 |
| andymccr | that seems inconsistent though - also it looks like the hardening issue was transient thankfully | 09:55 |
| odyssey4me | you might need to add some retries there | 09:56 |
| odyssey4me | I think I saw that being done on some other system start things | 09:56 |
| andymccr | gonna test some - if i can get it to work then i'll submit a patch on top before backporting | 09:58 |
| andymccr | for now i've abandoned those backports | 09:58 |
| *** drifterza has quit IRC | 10:00 | |
| odyssey4me | okie dokey | 10:01 |
| *** drifterza has joined #openstack-ansible | 10:03 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder stable/ocata: Update paste, policy and rootwrap configurations 2017-08-25 https://review.openstack.org/497778 | 10:09 |
| *** vnogin has joined #openstack-ansible | 10:10 | |
| openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-os_tacker stable/pike: Update roles & vars for stable/pike https://review.openstack.org/499955 | 10:11 |
| *** chhavi has quit IRC | 10:14 | |
| *** rstarmer has quit IRC | 10:15 | |
| *** rstarmer has joined #openstack-ansible | 10:15 | |
| *** rstarmer has quit IRC | 10:15 | |
| *** rstarmer has joined #openstack-ansible | 10:16 | |
| *** rstarmer has quit IRC | 10:16 | |
| *** rstarmer has joined #openstack-ansible | 10:17 | |
| *** rstarmer has quit IRC | 10:17 | |
| openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-os_tacker master: Revert "Update roles & vars for stable/pike" https://review.openstack.org/499958 | 10:17 |
| *** rstarmer has joined #openstack-ansible | 10:18 | |
| *** rstarmer has quit IRC | 10:18 | |
| *** rstarmer has joined #openstack-ansible | 10:18 | |
| *** rstarmer has quit IRC | 10:19 | |
| *** hw_wutianwei has quit IRC | 10:21 | |
| *** qiliang27 has quit IRC | 10:22 | |
| *** iahmed has joined #openstack-ansible | 10:25 | |
| *** stuartgr has joined #openstack-ansible | 10:31 | |
| *** hw_wutianwei has joined #openstack-ansible | 10:34 | |
| openstackgerrit | Andy McCrae proposed openstack/openstack-ansible-lxc_hosts master: Retry lxc-net service start on CentOS7 https://review.openstack.org/499962 | 10:35 |
| *** vnogin has quit IRC | 10:42 | |
| *** thorst_afk has joined #openstack-ansible | 10:43 | |
| *** drifterza has quit IRC | 10:43 | |
| *** vnogin has joined #openstack-ansible | 10:45 | |
| *** thorst_afk has quit IRC | 10:48 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: scripts: scripts-library.sh: Hide 'which' command output https://review.openstack.org/499501 | 10:55 |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: scripts: scripts-library.sh: Do not use the '-m' option on tracepath https://review.openstack.org/499502 | 10:55 |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: scripts: scripts-library.sh: Add support for openSUSE https://review.openstack.org/499503 | 10:55 |
| admin0 | baffled: openstack image list as admin works, as tenant, from horizon, i see the list .. from the cli, openstack image list gives me a 404 :D | 10:58 |
| *** mrch has joined #openstack-ansible | 11:00 | |
| *** drifterza has joined #openstack-ansible | 11:00 | |
| admin0 | if i disable keepalive on say a third infra node (only keepalive) .. that controller will act as a silent partner right ? .. due to haproxy, it will still serve all .. but it will not be a part of the failover IPs .. | 11:01 |
| admin0 | is there an easy way to get ELK {elasticsearch/logstash/kibaba} on the logs .. the easiest i am thinking of is to install ELK on the rsyslog container directly | 11:02 |
| admin0 | maybe someone else already has playbooks that take care of this | 11:03 |
| *** ashak has joined #openstack-ansible | 11:08 | |
| *** thorst_afk has joined #openstack-ansible | 11:09 | |
| ashak | Hi, i've been trying to follow the Openstack Ansible 16.0.0 documentation, but I get as far as trying to run: openstack-ansible setup-infrastructure.yml but I very quickly end up with the following error on my infra and compute nodes: | 11:10 |
| ashak | fatal: [infra1]: FAILED! => {"failed": true, "msg": "No file was found when using with_first_found. Use the 'skip: true' option to allow this task to be skipped if no files are found"} | 11:10 |
| *** fxpester has quit IRC | 11:10 | |
| ashak | This is for task: [resolvconf : Gather variables for each operating system] in play: Install unbound DNS resolver client configurations | 11:10 |
| *** thorst_afk has quit IRC | 11:11 | |
| *** fxpester has joined #openstack-ansible | 11:12 | |
| *** jvidal has quit IRC | 11:12 | |
| odyssey4me | ashak we merged a patch for that just yesterday - 16.0.0 is still a release candidate, so I'd suggest using the head of stable/pike for your testing | 11:15 |
| ashak | odyssey4me: ah, ok. I thought it was slightly odd that the docs said to clone 16.0.0.0rc1 :) | 11:16 |
| odyssey4me | yep, as the deployment projects rely on the upstream projects releasing - we get a two week period to stabilise before we release | 11:17 |
| *** esberglu has joined #openstack-ansible | 11:26 | |
| *** esberglu has quit IRC | 11:26 | |
| openstackgerrit | Merged openstack/openstack-ansible-rsyslog_server master: Updated from OpenStack Ansible Tests https://review.openstack.org/499677 | 11:29 |
| ashak | odyssey4me: Cool, it gets much further now. Thanks. | 11:32 |
| odyssey4me | ashak don't forget to bootstrap-ansible any time you change the git source :) | 11:32 |
| *** hw_wutianwei has quit IRC | 11:39 | |
| *** huxinhui_ has quit IRC | 11:41 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_barbican master: Updated from OpenStack Ansible Tests https://review.openstack.org/499650 | 11:51 |
| openstackgerrit | Merged openstack/openstack-ansible-os_trove master: Updated from OpenStack Ansible Tests https://review.openstack.org/499668 | 11:51 |
| openstackgerrit | Merged openstack/openstack-ansible-os_zaqar master: Updated from OpenStack Ansible Tests https://review.openstack.org/499670 | 11:51 |
| openstackgerrit | Merged openstack/openstack-ansible-os_watcher master: Updated from OpenStack Ansible Tests https://review.openstack.org/499669 | 11:52 |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara master: Updated from OpenStack Ansible Tests https://review.openstack.org/499666 | 11:59 |
| *** thorst_afk has joined #openstack-ansible | 12:04 | |
| openstackgerrit | Merged openstack/openstack-ansible-galera_server master: Updated from OpenStack Ansible Tests https://review.openstack.org/499641 | 12:11 |
| openstackgerrit | Merged openstack/openstack-ansible-os_tempest master: Updated from OpenStack Ansible Tests https://review.openstack.org/499667 | 12:12 |
| *** askb has quit IRC | 12:13 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_aodh master: Updated from OpenStack Ansible Tests https://review.openstack.org/499649 | 12:13 |
| openstackgerrit | Merged openstack/openstack-ansible-rabbitmq_server master: Updated from OpenStack Ansible Tests https://review.openstack.org/499673 | 12:13 |
| openstackgerrit | Merged openstack/openstack-ansible-os_horizon master: Updated from OpenStack Ansible Tests https://review.openstack.org/499658 | 12:14 |
| openstackgerrit | Merged openstack/openstack-ansible-os_glance master: Updated from OpenStack Ansible Tests https://review.openstack.org/499655 | 12:15 |
| openstackgerrit | Merged openstack/openstack-ansible-os_ironic master: Updated from OpenStack Ansible Tests https://review.openstack.org/499659 | 12:17 |
| openstackgerrit | Merged openstack/openstack-ansible-os_designate master: Updated from OpenStack Ansible Tests https://review.openstack.org/499654 | 12:17 |
| openstackgerrit | Merged openstack/openstack-ansible-os_cinder master: Updated from OpenStack Ansible Tests https://review.openstack.org/499652 | 12:17 |
| *** esberglu has joined #openstack-ansible | 12:19 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_ceilometer master: Updated from OpenStack Ansible Tests https://review.openstack.org/499651 | 12:22 |
| *** smatzek has joined #openstack-ansible | 12:22 | |
| *** esberglu has quit IRC | 12:23 | |
| *** gkadam_ has quit IRC | 12:27 | |
| *** udesale has joined #openstack-ansible | 12:29 | |
| andymccr | mhayden: mgariepy: do you remember seeing this issue?: http://paste.openstack.org/show/620205/ - i vaguely remember that happened before but im not sure what the fix was | 12:33 |
| * mhayden looks | 12:34 | |
| andymccr | appreciate that :) | 12:34 |
| mhayden | andymccr: IIRC, this happened when libselinux-python wasn't available in the virtualenv | 12:34 |
| andymccr | ah | 12:34 |
| andymccr | did we have to do some magic to get that to work? | 12:34 |
| mhayden | i am not honestly sure | 12:34 |
| mhayden | that was on my list of to-dos | 12:35 |
| andymccr | i thought we had this before. hmm | 12:35 |
| *** woodard has quit IRC | 12:36 | |
| odyssey4me | andymccr for the task in question you need ot ensure that it's using the right python interpreter - not the venv but the host | 12:37 |
| andymccr | odyssey4me: ahh ok | 12:37 |
| openstackgerrit | Merged openstack/openstack-ansible stable/newton: Fix LXC container start order https://review.openstack.org/499891 | 12:37 |
| *** woodard has joined #openstack-ansible | 12:37 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_nova master: Updated from OpenStack Ansible Tests https://review.openstack.org/499664 | 12:37 |
| andymccr | odyssey4me: like the bits we set in the test repos? | 12:37 |
| andymccr | odyssey4me: will give that a go - thanks for the tip | 12:37 |
| odyssey4me | the selinux lib is on the host, but when you're doing things like an aio ansible tries to be clever and use its own venv (if it's executed from a venv), so you have to force the interpreter to the host one | 12:37 |
| odyssey4me | like this for a play: https://github.com/openstack/openstack-ansible/blob/9c33562e4b63a178d108f5dcd6d3429f2c5ad890/tests/bootstrap-aio.yml#L25 | 12:38 |
| odyssey4me | but you can do it for a single task too | 12:38 |
| odyssey4me | eg: https://github.com/openstack/openstack-ansible-openstack_hosts/blob/393175577c3d4a8024fab2563b683dece10d46eb/tasks/openstack_update_hosts_file.yml#L24 | 12:39 |
| odyssey4me | we're doing it there due to the localhost delegation | 12:39 |
| mhayden | ah nice trick, odyssey4me | 12:39 |
| odyssey4me | mhayden wasn't me, it was mgariepy :) | 12:39 |
| odyssey4me | well - he figure it out and did some patches, I found some more issues and did some more... it's a team effort ;) | 12:40 |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron master: Updated from OpenStack Ansible Tests https://review.openstack.org/497275 | 12:42 |
| andymccr | sweet :) thanks mgariepy odyssey4me - will put a patch in for centos shortly | 12:43 |
| andymccr | centos-ceph rather | 12:43 |
| mgariepy | morning | 12:52 |
| openstackgerrit | Merged openstack/openstack-ansible-os_magnum master: Updated from OpenStack Ansible Tests https://review.openstack.org/499661 | 12:58 |
| *** thegreenhundred has quit IRC | 13:02 | |
| *** mohit048 has quit IRC | 13:03 | |
| *** SerenaFeng has joined #openstack-ansible | 13:04 | |
| *** esberglu has joined #openstack-ansible | 13:07 | |
| *** esberglu has quit IRC | 13:07 | |
| *** markvoelker has joined #openstack-ansible | 13:11 | |
| *** exodusftw has quit IRC | 13:11 | |
| *** drifterza has quit IRC | 13:14 | |
| openstackgerrit | Merged openstack/openstack-ansible-rsyslog_client master: Updated from OpenStack Ansible Tests https://review.openstack.org/499676 | 13:15 |
| openstackgerrit | Merged openstack/openstack-ansible-plugins master: Updated from OpenStack Ansible Tests https://review.openstack.org/499672 | 13:16 |
| *** exodusftw has joined #openstack-ansible | 13:17 | |
| openstackgerrit | Merged openstack/openstack-ansible-pip_install master: Updated from OpenStack Ansible Tests https://review.openstack.org/499671 | 13:18 |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Updated from OpenStack Ansible Tests https://review.openstack.org/499647 | 13:18 |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: Fix LXC container start order https://review.openstack.org/499888 | 13:19 |
| openstackgerrit | Merged openstack/openstack-ansible stable/ocata: Fix LXC container start order https://review.openstack.org/499890 | 13:20 |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_openrc master: Updated from OpenStack Ansible Tests https://review.openstack.org/499646 | 13:20 |
| openstackgerrit | Merged openstack/openstack-ansible-apt_package_pinning master: Updated from OpenStack Ansible Tests https://review.openstack.org/499638 | 13:21 |
| openstackgerrit | Merged openstack/openstack-ansible-memcached_server master: Updated from OpenStack Ansible Tests https://review.openstack.org/499644 | 13:21 |
| openstackgerrit | Merged openstack/openstack-ansible-ceph_client master: Updated from OpenStack Ansible Tests https://review.openstack.org/499639 | 13:21 |
| openstackgerrit | Merged openstack/openstack-ansible-haproxy_server master: Updated from OpenStack Ansible Tests https://review.openstack.org/499642 | 13:22 |
| openstackgerrit | Merged openstack/openstack-ansible-openstack_hosts master: Updated from OpenStack Ansible Tests https://review.openstack.org/499645 | 13:23 |
| openstackgerrit | Merged openstack/openstack-ansible-galera_client master: Updated from OpenStack Ansible Tests https://review.openstack.org/499640 | 13:24 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts master: Updated from OpenStack Ansible Tests https://review.openstack.org/499643 | 13:25 |
| openstackgerrit | Merged openstack/openstack-ansible-repo_server master: Updated from OpenStack Ansible Tests https://review.openstack.org/499675 | 13:27 |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts master: Retry lxc-net service start on CentOS7 https://review.openstack.org/499962 | 13:27 |
| *** vnogin has quit IRC | 13:28 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_monasca-agent master: Updated from OpenStack Ansible Tests https://review.openstack.org/499663 | 13:28 |
| *** vnogin has joined #openstack-ansible | 13:29 | |
| openstackgerrit | Merged openstack/openstack-ansible master: Enable verbose logging when fetching roles via git-clone https://review.openstack.org/498787 | 13:29 |
| *** galstrom_zzz is now known as galstrom | 13:30 | |
| *** vnogin has quit IRC | 13:30 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_almanach master: Updated from OpenStack Ansible Tests https://review.openstack.org/499648 | 13:31 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_hosts master: SUSE: Use the OBS Virtualization repository for lxc-2.X.X https://review.openstack.org/500050 | 13:31 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_container_create master: [DNM] - Testing openSUSE lxc_container_create role https://review.openstack.org/500051 | 13:32 |
| *** dave-mccowan has joined #openstack-ansible | 13:32 | |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_container_create master: Updated from OpenStack Ansible Tests https://review.openstack.org/497252 | 13:33 |
| openstackgerrit | Merged openstack/openstack-ansible-os_rally master: Updated from OpenStack Ansible Tests https://review.openstack.org/499665 | 13:35 |
| *** dxiri has joined #openstack-ansible | 13:35 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_cloudkitty master: Updated from OpenStack Ansible Tests https://review.openstack.org/499653 | 13:36 |
| *** esberglu has joined #openstack-ansible | 13:36 | |
| *** esberglu has quit IRC | 13:37 | |
| *** esberglu has joined #openstack-ansible | 13:39 | |
| *** dxiri has quit IRC | 13:40 | |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-rabbitmq_server master: Increase RabbitMQ's ulimit to 65536 https://review.openstack.org/500054 | 13:41 |
| openstackgerrit | Merged openstack/ansible-hardening master: Add release note for F26 support https://review.openstack.org/498847 | 13:41 |
| openstackgerrit | Major Hayden proposed openstack/ansible-hardening stable/pike: Add release note for F26 support https://review.openstack.org/500055 | 13:42 |
| *** udesale has quit IRC | 13:42 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_heat master: Updated from OpenStack Ansible Tests https://review.openstack.org/499657 | 13:47 |
| *** vnogin has joined #openstack-ansible | 13:49 | |
| *** shardy has quit IRC | 13:49 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_tempest master: Fix tempest log folder state when on metal https://review.openstack.org/500058 | 13:50 |
| openstackgerrit | Merged openstack/ansible-hardening master: Fix AppArmor idempotency https://review.openstack.org/498468 | 13:50 |
| *** woodard has quit IRC | 13:51 | |
| *** shardy has joined #openstack-ansible | 13:56 | |
| *** gouthamr has joined #openstack-ansible | 13:56 | |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: [WIP] Further hyper-converge our container usage https://review.openstack.org/499396 | 13:58 |
| *** mrch has quit IRC | 14:04 | |
| *** SerenaFe_ has joined #openstack-ansible | 14:04 | |
| *** vnogin_ has joined #openstack-ansible | 14:05 | |
| *** vnogin has quit IRC | 14:05 | |
| *** SerenaFeng has quit IRC | 14:05 | |
| *** dxiri has joined #openstack-ansible | 14:06 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_neutron master: Update links in README https://review.openstack.org/499421 | 14:09 |
| openstackgerrit | Merged openstack/openstack-ansible-os_tacker master: Revert "Update roles & vars for stable/pike" https://review.openstack.org/499958 | 14:13 |
| openstackgerrit | Merged openstack/openstack-ansible-os_tacker stable/pike: Update roles & vars for stable/pike https://review.openstack.org/499955 | 14:14 |
| openstackgerrit | Merged openstack/openstack-ansible-os_keystone stable/ocata: Remove Developer Tools package group https://review.openstack.org/496273 | 14:15 |
| *** cshen has quit IRC | 14:15 | |
| *** vakuznet has joined #openstack-ansible | 14:16 | |
| *** vnogin_ has quit IRC | 14:19 | |
| *** vnogin has joined #openstack-ansible | 14:19 | |
| *** woodard has joined #openstack-ansible | 14:24 | |
| *** SerenaFe_ has quit IRC | 14:29 | |
| *** marst has joined #openstack-ansible | 14:29 | |
| openstackgerrit | Merged openstack/openstack-ansible-repo_build stable/pike: Reduce package list for CentOS https://review.openstack.org/496741 | 14:33 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible master: Suppress curl warning w/shell module https://review.openstack.org/500075 | 14:37 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-os_swift master: Suppress rsync warnings w/command https://review.openstack.org/500076 | 14:39 |
| hwoarang | evrardjp: odyssey4me i believe this https://review.openstack.org/#/c/500050/ will make you happy :) | 14:40 |
| mgariepy | cool :D | 14:41 |
| openstackgerrit | Andy McCrae proposed openstack/openstack-ansible master: Add ansible_python_interpreter to Ceph playbook https://review.openstack.org/500082 | 14:45 |
| evrardjp | woot | 14:47 |
| evrardjp | andymccr: good forward thinking on the ceph thing | 14:48 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-os_tempest master: Fix tempest log folder state when on metal https://review.openstack.org/500058 | 14:50 |
| *** iahmed has quit IRC | 14:51 | |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts master: SUSE: Use the OBS Virtualization repository for lxc-2.X.X https://review.openstack.org/500050 | 14:51 |
| *** yifei has joined #openstack-ansible | 14:52 | |
| andymccr | evrardjp: was failing for me, trying to get all the periodicals going. hoping to get that done before release so we can say its all green! | 14:53 |
| * ashak grumbles. in play Install glance services, task os_glance : Ensure glance service is now failing for me with Unable to establish connection to http://172.29.236.3:35357/v3/auth/tokens: ('Connection aborted.', BadStatusLine("''",)) | 14:54 | |
| ashak | I tried using the openstack client from the glance container and it suffered a similar error. I eventually solved that by pip uninstall urllib3 and installing python-urllib3 using yum. | 14:55 |
| ashak | But the ansible task still fails with the error and i've no idea what it's actually telling me | 14:56 |
| ashak | Connectivity is fine. 172,29.236.3 is a vrpp vip hosted on one of the infra hosts using keepalived. haproxy is looking after getting 35357 to the correct container. | 14:57 |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: [DOC} Add openSUSE to Software requirements https://review.openstack.org/499913 | 14:58 |
| evrardjp | ashak: what does haproxy admin socket tell you ? | 14:58 |
| evrardjp | ashak: how did you configure this? | 14:59 |
| evrardjp | pip uninstall and install packages are generally not a good idea, we put all the proper versions within the venv | 14:59 |
| odyssey4me | ashak are you using centos? I think mhayden hit that yesterday and was trying to work out a solution | 14:59 |
| ashak | I am using Centos yeah | 15:00 |
| mgariepy | ashak, which tag ? | 15:00 |
| ashak | evrardjp: haproxy stats (if that's what you mean) suggets all is well. haproxy and keepalived configured by ansible with a manual addition to the file to enable stats on port 9000. | 15:01 |
| ashak | mgariepy: erm... stable/pike | 15:01 |
| ashak | HEAD | 15:01 |
| *** DanyC has quit IRC | 15:02 | |
| evrardjp | ashak: we support stats in our haproxy role, it's just one var to set IIRC. | 15:02 |
| ashak | evrardjp: ah right. I have missed that somewhere. | 15:02 |
| ashak | thanks | 15:03 |
| evrardjp | head of stable/pike... Interesting | 15:04 |
| evrardjp | you sure you are on head of stable/pike? | 15:05 |
| ashak | I was using 1.6.0.0rc1 this morning as per the docs. But I already ran into something that odyssey4me suggested had been fixed since that tag. | 15:05 |
| evrardjp | If that's the case, maybe we just merged something that broke it, or, alternatively, your configuration could be wrong | 15:05 |
| evrardjp | oh, so not head | 15:06 |
| evrardjp | when you pulled up latest version, you should re-run all your playbooks | 15:06 |
| evrardjp | including the repo build, that will generate new requirements and all. | 15:07 |
| ashak | Wel after asking about my previous problem, odyssey4me's suggestion was to switch to head of stable/pike for the moment to obtain the fix to the problem I was having | 15:07 |
| ashak | I did that I think. | 15:07 |
| evrardjp | that was a good idea, but did you re-run the repo build etc? | 15:07 |
| mhayden | ashak / odyssey4me: reviews please! :) https://review.openstack.org/499601 | 15:07 |
| mhayden | i'll add hwoarang's extras in there now | 15:07 |
| *** woodard has quit IRC | 15:08 | |
| evrardjp | typo! | 15:08 |
| ashak | evrardjp: I ran all the playbooks again that were listed in the documentation I was following. | 15:08 |
| mhayden | oopsies? | 15:08 |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible-pip_install master: Use OS pkgs, not pip https://review.openstack.org/499601 | 15:08 |
| evrardjp | got you covered bro | 15:08 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-pip_install master: Use OS pkgs, not pip https://review.openstack.org/499601 | 15:09 |
| ashak | damn, you guys are hardcore, i'm having trouble keeping up ;) | 15:09 |
| mhayden | evrardjp: haha, i trampled your edit | 15:09 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-pip_install master: Use OS pkgs, not pip https://review.openstack.org/499601 | 15:10 |
| evrardjp | mhayden: see my review too | 15:10 |
| openstackgerrit | Major Hayden proposed openstack/ansible-hardening stable/pike: Add release note for F26 support https://review.openstack.org/500055 | 15:11 |
| *** vikrant has joined #openstack-ansible | 15:12 | |
| *** EmilienM has quit IRC | 15:12 | |
| *** EmilienM has joined #openstack-ansible | 15:12 | |
| mhayden | evrardjp: leaving those variables undefined/blank should be fine since they exist in defaults/main.yml | 15:12 |
| evrardjp | if they do that's fine | 15:12 |
| mhayden | ... if you liked it, then you should have put a deuce on it ... | 15:14 |
| mhayden | all the lonely reviews | 15:14 |
| mhayden | now put your hands up | 15:14 |
| *** fxpester has quit IRC | 15:16 | |
| mgariepy | mhayden, see my comment in 499601 | 15:16 |
| mhayden | mgariepy: lookin | 15:16 |
| mhayden | oh good point | 15:17 |
| mhayden | any feedback from anyone else on whether we should *remove* those pip packages we installed? | 15:17 |
| mhayden | that probably wouldn't uninstall dependencies, though :( | 15:17 |
| mgariepy | or we add release note that containers need to be destroyed first... haha | 15:18 |
| *** dmsimard has quit IRC | 15:19 | |
| *** dmsimard has joined #openstack-ansible | 15:19 | |
| *** thegreenhundred has joined #openstack-ansible | 15:22 | |
| spotz | mgariepy: I know I just added something about detroying and creating containers somewhere | 15:22 |
| mgariepy | maybe the pip pkg removal should go in Pike instead of master. | 15:23 |
| *** vnogin has quit IRC | 15:26 | |
| *** yifei has quit IRC | 15:26 | |
| *** vnogin has joined #openstack-ansible | 15:26 | |
| *** pcaruana has quit IRC | 15:30 | |
| *** Oku_OS is now known as Oku_OS-away | 15:32 | |
| mhayden | hwoarang: was your suggestion to remove those pip packages with pip uninstall? | 15:34 |
| mhayden | or just omit them from the yml | 15:34 |
| mhayden | i get ancy about this removal of pip packages | 15:34 |
| mhayden | but then again, most of the time, the venv contains the modules | 15:34 |
| mhayden | not the system itself | 15:34 |
| hwoarang | mhayden: with pip uninstall :/ | 15:34 |
| *** chyka has joined #openstack-ansible | 15:36 | |
| *** kenperkins has joined #openstack-ansible | 15:39 | |
| kenperkins | I can't find a more specific channel for galera, so I'll ask here: | 15:39 |
| kenperkins | I'm trying to understand why the haproxy config for galera in openstack-ansible doesn't treat all of the galera nodes as equals instead of only a single backend with 2 backup nodes | 15:40 |
| mhayden | hwoarang: got it | 15:40 |
| mhayden | kenperkins: well, galera recommends sending reads to all nodes | 15:43 |
| mhayden | but writes to one primary one | 15:43 |
| mhayden | and use the others as backups | 15:43 |
| kenperkins | it does neither of those things | 15:43 |
| mhayden | kenperkins: can you link me to what you're seeing? | 15:43 |
| kenperkins | it sends all writes and reads to one node | 15:43 |
| mhayden | so we've talked about making a writing vip and reading vip in the past | 15:44 |
| mhayden | the openstack services don't understand how to split reads and writes, though | 15:44 |
| mhayden | there was code tested in mitaka/newton for that, but the nova folks gave up | 15:44 |
| kenperkins | https://github.com/openstack/openstack-ansible/blob/f026f81b8c862f053e915933b434bc8d1c63c80a/group_vars/all/haproxy.yml#L18-L19 | 15:44 |
| mhayden | since galera recommends writes go to a single server, we just went with that | 15:44 |
| kenperkins | where is that recommendation? I'd like to read it | 15:45 |
| mhayden | https://mariadb.com/resources/blog/setup-mariadb-enterprise-cluster-part-3-setup-ha-proxy-load-balancer-read-and-write-pools | 15:45 |
| mhayden | ideally we'd have a read VIP with all three and a write VIP with one (plus backups) | 15:45 |
| mhayden | kenperkins: there's always https://mariadb.com/products/technology/maxscale ;) | 15:46 |
| kenperkins | yea I'm not challenging it | 15:46 |
| kenperkins | just trying to understand it | 15:46 |
| kenperkins | remind me which version of galera we're deploying? is it the maria 10.x one? | 15:47 |
| mhayden | depends on the OSA release | 15:47 |
| mhayden | usually 10.0/10.1 | 15:47 |
| mhayden | i believe ocata/pike have 10.1 and newton has 10.0 | 15:47 |
| mhayden | but i'd have to go check | 15:47 |
| kenperkins | yep, 10.x is good enough | 15:48 |
| kenperkins | so basically we're using galera as an easier way to fail-over to a replica :D | 15:48 |
| mhayden | kenperkins: further bedtime reading http://lists.openstack.org/pipermail/openstack-dev/2014-August/042528.html | 15:50 |
| mhayden | kenperkins: at the moment, yes | 15:50 |
| mhayden | working around deadlocks is our priority | 15:50 |
| mhayden | hence the single reader/writer | 15:50 |
| mhayden | but if we could split ... ;) | 15:50 |
| *** mugsie has joined #openstack-ansible | 15:51 | |
| kenperkins | yea that's a messy situation. thanks for helping me understand | 15:54 |
| kenperkins | also read https://ghostaldev.com/2016/05/22/galera-gotcha-mysql-users/ | 15:54 |
| kenperkins | thanks @mhayden @mugsie | 15:54 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-pip_install master: Use OS pkgs, not pip https://review.openstack.org/499601 | 15:58 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-pip_install master: Use OS pkgs, not pip https://review.openstack.org/499601 | 15:59 |
| openstackgerrit | Major Hayden proposed openstack/openstack-ansible-pip_install master: Use OS pkgs, not pip https://review.openstack.org/499601 | 16:00 |
| mhayden | sheesh, third times a charm | 16:00 |
| mhayden | kenperkins: de nada, sir! | 16:00 |
| openstackgerrit | Major Hayden proposed openstack/ansible-hardening stable/pike: Fix AppArmor idempotency https://review.openstack.org/500117 | 16:03 |
| openstackgerrit | Major Hayden proposed openstack/ansible-hardening stable/pike: Fix AppArmor idempotency https://review.openstack.org/500117 | 16:04 |
| *** vnogin has quit IRC | 16:12 | |
| *** woodard has joined #openstack-ansible | 16:14 | |
| *** dave-mccowan has quit IRC | 16:20 | |
| *** esberglu has quit IRC | 16:24 | |
| *** DanyC has joined #openstack-ansible | 16:24 | |
| *** DanyC has quit IRC | 16:29 | |
| *** galstrom is now known as galstrom_zzz | 16:37 | |
| *** galstrom_zzz is now known as galstrom | 16:43 | |
| openstackgerrit | Major Hayden proposed openstack/ansible-hardening master: Updated from OpenStack Ansible Tests https://review.openstack.org/497293 | 16:47 |
| *** esberglu has joined #openstack-ansible | 16:47 | |
| *** esberglu has quit IRC | 16:51 | |
| *** galstrom is now known as galstrom_zzz | 16:53 | |
| *** rstarmer has joined #openstack-ansible | 17:00 | |
| *** germs has joined #openstack-ansible | 17:01 | |
| *** electrofelix has quit IRC | 17:01 | |
| *** germs1 has joined #openstack-ansible | 17:04 | |
| openstackgerrit | Merged openstack/ansible-hardening stable/pike: Fedora 26 support https://review.openstack.org/498846 | 17:04 |
| *** germs has quit IRC | 17:05 | |
| openstackgerrit | Merged openstack/ansible-hardening stable/pike: Updated from OpenStack Ansible Tests https://review.openstack.org/496915 | 17:07 |
| *** dxiri has quit IRC | 17:14 | |
| *** maybebuggy has quit IRC | 17:19 | |
| *** esberglu has joined #openstack-ansible | 17:20 | |
| *** DanyC has joined #openstack-ansible | 17:21 | |
| *** thegreenhundred has quit IRC | 17:24 | |
| *** galstrom_zzz is now known as galstrom | 17:28 | |
| *** stuartgr has left #openstack-ansible | 17:30 | |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: scripts: bootstrap-ansible: Add required packages on openSUSE Leap https://review.openstack.org/499504 | 17:39 |
| openstackgerrit | Merged openstack/openstack-ansible stable/pike: tests: bootstrap-host: Use btrfs tools to determine available disk space https://review.openstack.org/499505 | 17:51 |
| *** galstrom is now known as galstrom_zzz | 17:56 | |
| *** poopcat has joined #openstack-ansible | 18:01 | |
| *** dxiri has joined #openstack-ansible | 18:02 | |
| *** albertcard1 has joined #openstack-ansible | 18:05 | |
| *** rstarmer has quit IRC | 18:06 | |
| *** poopcat has quit IRC | 18:07 | |
| *** rstarmer has joined #openstack-ansible | 18:16 | |
| *** albertcard1 is now known as poopcat | 18:23 | |
| *** galstrom_zzz is now known as galstrom | 18:23 | |
| admin0 | openstack image list fails ( cli ) .. but works on Horizon . anyone has seen this before .. stable/ocata | 18:27 |
| mgariepy | cloudnull, testign leap upgrade | 18:39 |
| mgariepy | cloudnull, it's causing some issues :) https://github.com/rcbops/rpc-maas/blob/master/tests/aio-create.sh#L100 | 18:39 |
| *** vikrant has quit IRC | 18:45 | |
| tasker | what is "AIO"? | 18:51 |
| mattt | "all-in-one" | 18:53 |
| mattt | tasker: where you basically shoehorn a bunch of stuff onto a single instance, resulting in many tears and much crying | 18:54 |
| admin0 | tasker: its one click install of openstack // the tears and crying are of joy | 18:57 |
| *** jwitko has quit IRC | 19:00 | |
| tasker | hehe. this description is getting better and better. | 19:03 |
| *** esberglu has quit IRC | 19:04 | |
| *** esberglu has joined #openstack-ansible | 19:05 | |
| *** galstrom is now known as galstrom_zzz | 19:21 | |
| *** galstrom_zzz is now known as galstrom | 19:21 | |
| cloudnull | hahaha | 19:22 |
| cloudnull | mgariepy: what are you seeing? | 19:23 |
| cloudnull | something specific to that galera pin? | 19:23 |
| mgariepy | yeah it needs to be removed before running the upgrade. | 19:23 |
| cloudnull | ah yes | 19:24 |
| cloudnull | that pin was there just to get the AIO up | 19:24 |
| mgariepy | yeah | 19:24 |
| mgariepy | maybe we could check to see if it's there and remove the config ? | 19:24 |
| cloudnull | ++ | 19:24 |
| cloudnull | its likley others in real prod envs have something similar | 19:25 |
| cloudnull | especially if they've been running kilo all this time | 19:25 |
| mgariepy | yeah | 19:25 |
| mgariepy | testing in a vm right now, i'll need to replicate my prod in hw then test on a ""real"" env. | 19:26 |
| cloudnull | we'd not want to remove a specific URL set | 19:26 |
| cloudnull | maybe folks are using a frozen repo behond a firewall | 19:26 |
| cloudnull | so maybe we just add a couple break points | 19:27 |
| cloudnull | if that var is detected. | 19:27 |
| mgariepy | yeah | 19:27 |
| *** mbuil has quit IRC | 19:31 | |
| mgariepy | cloudnull, not quite sure where the check should be for this tho .. | 19:34 |
| cloudnull | we could probably use a playbook in the prep phase | 19:35 |
| cloudnull | and load the var files using the with_glob lookup | 19:35 |
| openstackgerrit | Merged openstack/openstack-ansible master: Add ansible_python_interpreter to Ceph playbook https://review.openstack.org/500082 | 19:35 |
| cloudnull | and then asser / fail when found and some option is not set. | 19:36 |
| * cloudnull spit balling | 19:36 | |
| odyssey4me | cloudnull mgariepy IIRc our glaera client/server roles try to detect another mariadb repo configured and remove it | 19:38 |
| odyssey4me | it's perhaps not perfect | 19:38 |
| mgariepy | if your vars overwrite it it will just use the same repo ;) | 19:40 |
| cloudnull | odyssey4me: ah good point I forgot about that. | 19:42 |
| cloudnull | maybe we should see about trying to shore that up ? | 19:42 |
| *** zcourts has joined #openstack-ansible | 19:43 | |
| odyssey4me | mgariepy quite, ah that may be the issue - because from some time on we set the filename | 19:46 |
| odyssey4me | I think Ocata. | 19:46 |
| odyssey4me | so you may end up with two files, same repo | 19:46 |
| odyssey4me | but perhaps the apt pinning is not cleaning up - not sure | 19:47 |
| mgariepy | it's hard to keep a system clean ;) | 19:47 |
| mgariepy | have a nice weekend all. | 19:51 |
| *** gouthamr has quit IRC | 19:51 | |
| cloudnull | take care mgariepy | 19:52 |
| admin0 | hi all .. i have an issue of openstack image list returning a 404 while in the same user login from horizon, returing the images OK | 19:54 |
| admin0 | GET call to None for https://my.turnkey.cloud:9292/v2/images used request id req-xxx => Request returned failure status: 404 … Not Found (HTTP 404) (Request-ID: req-xxx) | 19:55 |
| tasker | cloudnull: it looks like my problem is the URL for the file was looking for ubuntu-16.04-x86_64. the file I saw was under unbuntu-14.04-x86-64. | 19:57 |
| tasker | I think that's because the new container is looking to be built at xenial, but no currently active repo container has those files under that directory. | 19:58 |
| tasker | I'm following along with https://etherpad.openstack.org/p/osa-newton-xenial-upgrade | 19:58 |
| tasker | do I need to force a xenial repo container to be built prior to running "setup-everything"? | 19:58 |
| *** vnogin has joined #openstack-ansible | 19:59 | |
| cloudnull | admin0: https://my.turnkey.cloud:9292/ seems to work. is there anything on the horizon container that might be redirecting that traffic to some other address? | 20:01 |
| admin0 | horizon works, API does not | 20:02 |
| admin0 | its a standad install .. not sure if it installs anything else | 20:02 |
| cloudnull | api from the utility container ? | 20:02 |
| cloudnull | tasker: you running both 14 and 16.04 ? | 20:03 |
| tasker | I'm trying to migrate from 14 to 16 | 20:03 |
| cloudnull | ok. | 20:03 |
| cloudnull | and you have done one infra host? | 20:03 |
| cloudnull | at least one | 20:03 |
| cloudnull | if you did an inplace upgrade of an infra host, you will need to delete the repo container on that host | 20:04 |
| cloudnull | and then recreate it | 20:04 |
| cloudnull | if you re-kicked it, it should do the right thing out of the box | 20:04 |
| tasker | yeah, I've got two hosts at xenial. one with containers but nothing in them (built during "setup-everything") | 20:05 |
| tasker | but when the galera book plays, it gets to the 404 error trying to pull a file from any existing repo container that dosen't exist. | 20:05 |
| admin0 | cloudnull: i tried from utility ( as admin it works ) but as tenant its a virtualenv pip install python-openstackclient | 20:05 |
| tasker | I'm working on forcing out one xenial repo container into the new xenial host. | 20:07 |
| tasker | then some haproxy magic to redirect repo lookups there. | 20:07 |
| tasker | (thanks for pointing out hatop, btw) | 20:07 |
| *** smatzek has quit IRC | 20:16 | |
| *** gouthamr has joined #openstack-ansible | 20:29 | |
| *** marst has quit IRC | 20:37 | |
| *** galstrom is now known as galstrom_zzz | 20:37 | |
| tasker | great. I've tried setup-everything on controller-2 and -3 and neither are putting that file down. it's like neither one wants to host the repo container. i mean, it's got the container, but it doesn't have any of the built wheels. | 20:43 |
| tasker | when repo-install is run with --limit, neithen -2 nor -3 want to be a part of repo-masters. | 20:46 |
| *** thorst_afk has quit IRC | 20:46 | |
| tasker | since there's no host in that group, repo-build doesn't run and build stuff. | 20:46 |
| tasker | oh wise and powerful OSA devs, what have i done to offend? why do you forsake me? it's friday and I want to go home! <G> | 20:48 |
| tasker | I'm going to try it without a limit and see which it picks. | 20:48 |
| *** esberglu has quit IRC | 20:49 | |
| *** esberglu has joined #openstack-ansible | 20:50 | |
| tasker | looks like it's proceeding. | 20:50 |
| tasker | this explains some of the warnings in the etherpad about not using limit. | 20:50 |
| *** esberglu has quit IRC | 20:54 | |
| cloudnull | tasker: sorry | 21:08 |
| cloudnull | its friday before a holiday I suspect most are on their way out of town | 21:08 |
| tasker | it's okay. | 21:08 |
| cloudnull | did you make it go? | 21:08 |
| tasker | most of my commentary is just "stream of conciousness" rhetoric; not much response needed. | 21:09 |
| tasker | sort of. my last attempt ended in "'dict_object' has no attribute 'repo_build'" | 21:09 |
| tasker | running it again to see if it happens again. | 21:10 |
| tasker | then it's into the code to see if I can see what's going on. | 21:10 |
| cloudnull | running `openstack-ansible repo-build.yml` | 21:10 |
| cloudnull | ? | 21:10 |
| tasker | repo-install | 21:10 |
| cloudnull | ok | 21:10 |
| tasker | and it looks like it's gone on? not sure. repo-build can take a while. I'll know in a little bit. | 21:11 |
| cloudnull | I'll be here for the next 45. | 21:12 |
| cloudnull | let me know | 21:12 |
| tasker | don't hang around on my account. . ) | 21:12 |
| cloudnull | i've got some email to write before I can sign off. | 21:13 |
| *** gouthamr has quit IRC | 21:16 | |
| tasker | oh -- in case it was misunderstood; the "forsaken me" comment was a joke about why OSA isn't working today. not that y'all were busy doing other things and not answering. | 21:16 |
| tasker | ok. the error appeared again. | 21:18 |
| tasker | OSA collects two groups for the repo_master, 14.04 and 16.04. it appears to build 14.04 on repo-1 no problem. | 21:19 |
| tasker | when it goes to run repo_build for repo-2 (16.04) it comes back with 'dict object' has no attribute 'repo_build' for the task "Ensure that temporary folders from previous build failures are absent" | 21:19 |
| *** vnogin has quit IRC | 21:19 | |
| tasker | (repo-1 is 14.04) | 21:19 |
| *** vnogin has joined #openstack-ansible | 21:20 | |
| tasker | and, I meant to make that run with increased verbosity so that I could see where the error is coming from. | 21:21 |
| tasker | the message mentions "/etc/ansible/roles/repo_build/tasks/main.yml", but I think it might be deeper. | 21:21 |
| tasker | running it again with more verbosity. | 21:21 |
| cloudnull | moar! | 21:26 |
| odyssey4me | you may need to force a rebuild of wheels/venvs | 21:27 |
| odyssey4me | use -e 'repo_build_git_reclone =yes,repo_build_wheel_rebuild=yes, repo_build_venv_rebuild=yes' | 21:28 |
| tasker | ok. giving that a shot. | 21:29 |
| odyssey4me | it's an unfortunate side-effect of making the repo build idempotent - we don't currently have something that tells whether one of the arch/distro sets is missing | 21:29 |
| odyssey4me | if that works, please register a bug - hopefully we can fix that | 21:30 |
| cloudnull | that's a great note... neith ^ | 21:30 |
| cloudnull | I totally forgot about that | 21:30 |
| cloudnull | odyssey4me: for prez! | 21:31 |
| odyssey4me | we should maybe document that as a tip or something for now | 21:31 |
| odyssey4me | but a bug would help make that right | 21:32 |
| *** thorst_afk has joined #openstack-ansible | 21:32 | |
| *** galstrom_zzz is now known as galstrom | 21:32 | |
| * cloudnull creating a bug | 21:32 | |
| odyssey4me | my plan is to actually turn the repo build into more of a cache in this cycle for queens | 21:33 |
| odyssey4me | but older branches will still need a fix | 21:33 |
| odyssey4me | I'll be working on a prototype for the PTG, probably tomorrow. | 21:34 |
| *** thorst_afk has quit IRC | 21:36 | |
| *** galstrom is now known as galstrom_zzz | 21:39 | |
| tasker | new error, second time now: IOError: [Errno 2] No such file or directory: '/var/www/repo/pools/ubuntu-14.04-x86_64/gnocchiclient/gnocchiclient-2.8.2-py2.py3-none-any.whl | 21:39 |
| tasker | happened with and without the variable delcaration. | 21:40 |
| tasker | I'm going to flush the /var/www/repo directory and try again. | 21:40 |
| odyssey4me | hmm, sounds like an rsync fail | 21:40 |
| odyssey4me | basically it thinks it's there, then it isn't | 21:40 |
| odyssey4me | or it might be a symlink fail, so yeah - maybe a flush and re-do | 21:41 |
| cloudnull | odyssey4me tasker: https://bugs.launchpad.net/openstack-ansible/+bug/1714597 | 21:41 |
| openstack | Launchpad bug 1714597 in openstack-ansible "multi OS repo-build needs to be more intuitive " [Undecided,New] | 21:41 |
| odyssey4me | cloudnull the biggest hurdle there is not having a good way to test properly | 21:42 |
| odyssey4me | I tried going down the rabbit hole of adding a multi-os and multi-arch test in the repo build role, but it got complicated | 21:42 |
| cloudnull | yup. I wonder if we could override the container base OS to set this up in the gate. | 21:42 |
| cloudnull | ah. | 21:42 |
| odyssey4me | yeah, tried that - but we do quite a few things to keep things simple | 21:43 |
| odyssey4me | so the overrides just grew and grew and got very, very messy | 21:43 |
| odyssey4me | this is part of why I've decided to rethink it entirely | 21:43 |
| cloudnull | i could see that being an issue | 21:43 |
| odyssey4me | so with the new idea I have, I'll try to make it test all of that from the get-go | 21:43 |
| cloudnull | cool | 21:44 |
| tasker | marking myself as "affected" | 21:45 |
| tasker | thanks cloudnull! | 21:45 |
| cloudnull | i just did the clerical work, odyssey4me ++ for the possible fix | 21:45 |
| *** rstarmer has quit IRC | 21:47 | |
| *** thorst_afk has joined #openstack-ansible | 21:48 | |
| *** thorst_afk has quit IRC | 21:50 | |
| odyssey4me | tasker if you can add your own thoughts/experience/workarounds/solutions that'd be awesome | 21:50 |
| tasker | absolutely. | 21:50 |
| odyssey4me | thanks for putting that together cloudnull | 21:50 |
| tasker | now it's having a problem with repo-1! | 21:53 |
| tasker | "file not found: /var/www/repo/openstackgit/requirements/upper-constraints.txt" | 21:53 |
| tasker | what is up with me today? | 21:53 |
| odyssey4me | which one did you clear? | 21:53 |
| tasker | repo-2 | 21:53 |
| odyssey4me | hmm, odd | 21:54 |
| tasker | quite. | 21:54 |
| odyssey4me | I might be inclined to setup a lab for this and try the whole migration thing myself to see what happens | 21:54 |
| tasker | task path: /etc/ansible/roles/repo_build/tasks/repo_build_prepare.yml:67 | 21:54 |
| odyssey4me | I think it works well if you're doing a fresh env, but not so much when you're transitioning | 21:54 |
| odyssey4me | I wonder if it'd be worth just destroying all the repo containers, rebuilding them from scratch... if you're prepared to lose your previous data | 21:55 |
| tasker | and that only started showing up after I "rm -fr /var/www/repo/*" on repo-2 | 21:55 |
| odyssey4me | is this a test env, or prod env? | 21:55 |
| tasker | test | 21:55 |
| odyssey4me | oh - are you running repo-build, repo-server or repo-install ? | 21:55 |
| tasker | install | 21:56 |
| odyssey4me | hmm :/ | 21:56 |
| odyssey4me | I think the LB might be returning data, but then it balances to a node which doesn't have data | 21:56 |
| odyssey4me | disable the back-end for the one with no data in haproxy | 21:56 |
| tasker | but it showed up on a run of repo-build, so I regressed back to -install to catch the -server in there in case it did something important. | 21:56 |
| odyssey4me | then retry | 21:57 |
| tasker | all disabled. | 21:57 |
| odyssey4me | oh | 21:57 |
| tasker | well, -2 and -3 are. only -1 is active. | 21:57 |
| odyssey4me | and does one have data? | 21:57 |
| odyssey4me | are there any bad links on it? | 21:57 |
| odyssey4me | ie symlinks with no file to back them up? | 21:57 |
| tasker | data's there. just not that file it's looking for. | 21:58 |
| odyssey4me | oh wait - that's the git repositories | 21:58 |
| odyssey4me | the git repositories should be cloned, and the requirements repo is crucial to the first phase | 21:59 |
| tasker | the one thing I haven't done is run "setup-everything" without a limit. | 21:59 |
| odyssey4me | oh no - no need for that | 22:00 |
| odyssey4me | if you wipe directories, then repo-server is important | 22:00 |
| odyssey4me | otherwise just repo-build should do | 22:00 |
| tasker | ok. the notes at https://etherpad.openstack.org/p/osa-newton-xenial-upgrade are conflicting in some spots. | 22:00 |
| tasker | s/conflicting/confusing | 22:00 |
| cloudnull | ok. i have to run , folks are now at my house now. take care all. | 22:00 |
| odyssey4me | I need to call it a night :/ | 22:00 |
| tasker | have a great weekend! | 22:00 |
| tasker | no worries. me, too. | 22:00 |
| cloudnull | cheers | 22:00 |
| odyssey4me | cheers! | 22:01 |
| tasker | this can wait until tuesday. | 22:01 |
| tasker | thanks very much for your help. | 22:02 |
| *** DanyC has quit IRC | 22:05 | |
| *** DanyC has joined #openstack-ansible | 22:06 | |
| *** DanyC has quit IRC | 22:16 | |
| *** thorst_afk has joined #openstack-ansible | 22:29 | |
| *** thorst_afk has quit IRC | 22:32 | |
| *** rstarmer has joined #openstack-ansible | 22:34 | |
| *** rstarmer has quit IRC | 22:47 | |
| *** rstarmer has joined #openstack-ansible | 22:48 | |
| *** rstarmer has quit IRC | 23:02 | |
| *** rstarmer has joined #openstack-ansible | 23:07 | |
| *** gouthamr has joined #openstack-ansible | 23:13 | |
| *** thegreenhundred has joined #openstack-ansible | 23:16 | |
| *** zcourts has quit IRC | 23:20 | |
| *** thegreenhundred has quit IRC | 23:24 | |
| *** markvoelker has quit IRC | 23:36 | |
| *** chyka has quit IRC | 23:44 | |
| *** dxiri has quit IRC | 23:47 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
