| *** acormier has joined #openstack-ansible | 00:00 | |
| *** acormier has quit IRC | 00:05 | |
| *** acormier has joined #openstack-ansible | 00:11 | |
| *** acormier has quit IRC | 00:16 | |
| *** acormier has joined #openstack-ansible | 00:17 | |
| *** acormier has quit IRC | 00:22 | |
| *** yifei has joined #openstack-ansible | 00:22 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 00:26 |
|---|---|---|
| *** mrtenio has quit IRC | 00:26 | |
| *** kenperkins has quit IRC | 00:36 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 00:41 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_gnocchi master: Updated from global requirements https://review.openstack.org/502805 | 00:41 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 00:41 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 00:42 |
| prometheanfire | just wait til tomorrow, the olso stuff has been very slow in gate | 00:54 |
| *** acormier has joined #openstack-ansible | 00:54 | |
| *** hw_wutianwei1 has joined #openstack-ansible | 00:55 | |
| *** acormier has quit IRC | 00:58 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 01:06 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_gnocchi master: Updated from global requirements https://review.openstack.org/502805 | 01:06 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 01:06 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 01:06 |
| *** acormier has joined #openstack-ansible | 01:06 | |
| *** acormier has quit IRC | 01:07 | |
| *** acormier has joined #openstack-ansible | 01:08 | |
| *** hw_wutianwei1 has quit IRC | 01:09 | |
| *** hw_wutianwei1 has joined #openstack-ansible | 01:09 | |
| *** woodard_ has joined #openstack-ansible | 01:10 | |
| *** woodard has quit IRC | 01:10 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 01:22 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_gnocchi master: Updated from global requirements https://review.openstack.org/502805 | 01:22 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 01:22 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 01:22 |
| *** esberglu has quit IRC | 01:33 | |
| *** hybridpollo has joined #openstack-ansible | 02:19 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 02:36 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_gnocchi master: Updated from global requirements https://review.openstack.org/502805 | 02:36 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 02:36 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 02:36 |
| *** thorst has quit IRC | 02:51 | |
| *** thorst has joined #openstack-ansible | 02:51 | |
| *** Dinesh_Bhor has joined #openstack-ansible | 02:54 | |
| *** thorst has quit IRC | 02:56 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 02:56 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_gnocchi master: Updated from global requirements https://review.openstack.org/502805 | 02:56 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 02:56 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 02:57 |
| *** Dinesh_Bhor has quit IRC | 02:57 | |
| *** Dinesh_Bhor has joined #openstack-ansible | 02:57 | |
| *** dave-mcc_ has quit IRC | 03:09 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 03:12 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible stable/ocata: [DNM] Testing gate https://review.openstack.org/507336 | 03:13 |
| *** Dinesh_Bhor has quit IRC | 03:24 | |
| *** Dinesh_Bhor has joined #openstack-ansible | 03:28 | |
| *** acormier has quit IRC | 03:29 | |
| *** acormier has joined #openstack-ansible | 03:30 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 03:32 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_gnocchi master: Updated from global requirements https://review.openstack.org/502805 | 03:33 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 03:33 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 03:33 |
| *** acormier has quit IRC | 03:34 | |
| *** Dinesh_Bhor has quit IRC | 03:34 | |
| *** gouthamr has quit IRC | 03:35 | |
| *** chhavi has joined #openstack-ansible | 03:38 | |
| *** acormier has joined #openstack-ansible | 03:39 | |
| *** Dinesh_Bhor has joined #openstack-ansible | 03:46 | |
| *** thorst has joined #openstack-ansible | 03:53 | |
| logan- | hey all -- be on the lookout for gate failures w/ mariadb 10.1.27 | 03:56 |
| logan- | my ocata branch just started failing today when 10.1.27 hit my mirrors | 03:57 |
| logan- | mariadb segfaulting when keystone db_manage --expand runs | 03:57 |
| logan- | my last successful test (this morning) had 10.1.26 | 03:58 |
| logan- | http://paste.openstack.org/show/621892/ | 03:59 |
| *** udesale has joined #openstack-ansible | 03:59 | |
| *** acormier has quit IRC | 03:59 | |
| *** gkadam has joined #openstack-ansible | 04:02 | |
| *** poopcat has quit IRC | 04:03 | |
| logan- | limestone gate logs show it on https://review.openstack.org/#/c/454450/ | 04:09 |
| *** hybridpollo has quit IRC | 04:23 | |
| *** adreznec has quit IRC | 05:13 | |
| *** ecelik has joined #openstack-ansible | 05:17 | |
| *** chhavi has quit IRC | 05:40 | |
| *** chhavi_ has joined #openstack-ansible | 05:40 | |
| *** drifterza has joined #openstack-ansible | 05:45 | |
| *** adreznec has joined #openstack-ansible | 06:01 | |
| *** thorst has quit IRC | 06:03 | |
| *** Oku_OS-away is now known as Oku_OS | 06:06 | |
| *** pcaruana has joined #openstack-ansible | 06:20 | |
| *** chyka has quit IRC | 06:24 | |
| *** drifterza has quit IRC | 06:28 | |
| *** jvidal has joined #openstack-ansible | 06:28 | |
| *** drifterza has joined #openstack-ansible | 06:29 | |
| openstackgerrit | Christian Zunker proposed openstack/openstack-ansible-os_ceilometer stable/ocata: Remove python unicode encoding from policy.json file https://review.openstack.org/507368 | 06:34 |
| *** Oku_OS is now known as Oku_OS-away | 06:40 | |
| *** thorst has joined #openstack-ansible | 06:59 | |
| *** Oku_OS-away is now known as Oku_OS | 06:59 | |
| *** gkadam has quit IRC | 07:02 | |
| *** gkadam has joined #openstack-ansible | 07:02 | |
| *** hw_wutianwei1 has quit IRC | 07:24 | |
| *** mbuil has joined #openstack-ansible | 07:28 | |
| *** mrch has joined #openstack-ansible | 07:35 | |
| openstackgerrit | Merged openstack/openstack-ansible-lxc_hosts master: SUSE: Fix repo setup on container cache https://review.openstack.org/507106 | 08:05 |
| *** markvoelker has quit IRC | 08:06 | |
| *** ansibler has joined #openstack-ansible | 08:19 | |
| *** electrofelix has joined #openstack-ansible | 08:26 | |
| ansibler | Hi, trying to upgrade from ocata to pike. Upgrade did not succeed, trying to reinstall from scratch. openstack-ansible setup-hosts.yml fails on task TASK [lxc_hosts : Ensure image has been pre-staged]. Problem is it tries to retrieve non existent lxc container: https://us.images.linuxcontainers.org/images/ubuntu/xenial/amd64/default/20170921_03:49/rootfs.tar.xz, where do I reset that date(seems those are regenerated dai | 08:30 |
| evrardjp | that seems truncated | 08:32 |
| evrardjp | https://us.images.linuxcontainers.org/images/ubuntu/xenial/amd64/default/20170926_03:49/ seems indeed more accurate | 08:33 |
| evrardjp | did you override that? | 08:33 |
| *** bhujay has joined #openstack-ansible | 08:33 | |
| ansibler | no, I did try upgrade/install several days ago and now I cannot proceed because somewhere this old date is cached. Have tried to recreate /opt/openstack-ansible from git, does not help | 08:35 |
| evrardjp | are your roles up to date? | 08:36 |
| evrardjp | ansibler: could you tell me which systemd version do you run? | 08:37 |
| evrardjp | and which os? | 08:37 |
| ansibler | you mean roles somewhere under /etc/openstack_deploy/ ? | 08:38 |
| evrardjp | yes | 08:39 |
| evrardjp | no | 08:39 |
| evrardjp | sorry | 08:39 |
| evrardjp | under /etc/ansible | 08:39 |
| evrardjp | when you git clone/git pull + checkout to a new branch, you need to update not only osa code, but also the dependent roles | 08:40 |
| evrardjp | if you rebootstrap ansible that should do the trick | 08:40 |
| evrardjp | scripts/bootstrap-ansible.sh is part of the process. | 08:40 |
| evrardjp | did you do that? | 08:40 |
| ansibler | yes, several times. All hosts are Ubuntu 16.04.2 LTS | 08:41 |
| evrardjp | ok | 08:41 |
| evrardjp | any special backend/configuration? | 08:41 |
| ansibler | I have ceph as storage backend | 08:42 |
| evrardjp | ok that's important | 08:43 |
| ansibler | trying to find file where that "20170921_03:49" pattern hides, could delete/recreate | 08:44 |
| evrardjp | let me compare with today's upgrade job | 08:44 |
| evrardjp | Job complete, result: SUCCESS | 08:44 |
| evrardjp | ok so it's something in your environment or our jobs are wrong, let me dig deeper | 08:44 |
| evrardjp | could you check your systemd version? | 08:45 |
| evrardjp | that would simplify my life, I don't know it by heart | 08:45 |
| *** udesale has quit IRC | 08:45 | |
| ansibler | all hosts have systemd 229 | 08:47 |
| evrardjp | thanks let me check | 08:48 |
| *** udesale has joined #openstack-ansible | 08:51 | |
| evrardjp | ok this task is using machinectl | 08:53 |
| evrardjp | https://github.com/openstack/openstack-ansible-lxc_hosts/blob/stable/pike/tasks/lxc_cache_preparation.yml#L45 | 08:53 |
| ansibler | this really should be somewhere in my environment as I have tested newton then ocata here and now trying to clean all up and install pike, but still some remnants exist, cannot find where that pattern "20170921" is. | 08:54 |
| evrardjp | could you show me a dump of your openstack-ansible lxc-hosts.yml (I am insterested in tasks starting from https://github.com/openstack/openstack-ansible-lxc_hosts/blob/stable/pike/tasks/lxc_cache_preparation_systemd_new.yml#L19 ) | 08:54 |
| evrardjp | lxc-hosts-setup.yml sorry | 08:55 |
| odyssey4me | ansibler are you accessing through a proxy perhaps | 08:57 |
| *** hachi_ has joined #openstack-ansible | 09:01 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-specs master: Spec: Python Build/Install Process Simplification https://review.openstack.org/501296 | 09:04 |
| *** pbandark has joined #openstack-ansible | 09:05 | |
| odyssey4me | hmm, it looks like https://us.images.linuxcontainers.org/meta/1.0/index-system does not have that date any more, which means that perhaps the process to download that in lxc-hosts is not refreshing it properly | 09:07 |
| odyssey4me | ansibler do you see /tmp/index-system on the lxc hosts? | 09:08 |
| *** udesale has quit IRC | 09:11 | |
| ansibler | yes the problem is my scripts reference wrong url and yes I see /tmp/index-system file on lxc hosts | 09:11 |
| *** udesale has joined #openstack-ansible | 09:11 | |
| evrardjp | odyssey4me: it's using machinectl | 09:12 |
| ansibler | those are full of "20170921" pattern. Will delete those /tmp/index-system files and try again. | 09:12 |
| ansibler | not familiar with machinectl, seems related to systemd. Should I rather run some systemd/machinectl command to remove those indexes or simply delete? | 09:14 |
| evrardjp | odyssey4me: is that /tmp/index-system something of lxc download template? | 09:14 |
| evrardjp | or is that something from machinectl? Because it doesn't look like a machinectl thing | 09:15 |
| evrardjp | and due to the lxc_hosts role version of pike, we should have a machinectl usage for ansibler 's systems (running systemd 229( | 09:15 |
| evrardjp | so I am first thinking of poor fact cache | 09:16 |
| evrardjp | but I am maybe wrong, and I want to understand the /tmp/index-system you're talking about | 09:16 |
| ansibler | here is 1 line from /tmp/index-system: ubuntu;xenial;arm64;default;20170921_03:49;/images/ubuntu/xenial/arm64/default/20170921_03:49/ | 09:16 |
| ansibler | nothing about "lxc" in that file | 09:16 |
| evrardjp | that looks like the lxc standard default thing | 09:18 |
| evrardjp | metadata file | 09:18 |
| evrardjp | ok | 09:18 |
| ansibler | Here is the full error from failed task, that contains those paths from /tmp/index-system: | 09:18 |
| ansibler | TASK [lxc_hosts : Ensure image has been pre-staged] 2017-09-26 11:59:50,213 p=11781 u=root | failed: [infra1] (item={'_ansible_parsed': True, 'changed': True, '_ansible_no_log': False, u'ansible_job_id': u'77512949089.15635', u'started': 1, '_ansible_item_result': True, 'item': u'ubuntu;xenial;amd64;default;20170921_03:49;/images/ubuntu/xenial/amd64/default/20170921_03:49/', u'finished': 0, u'results_file': u'/root/.a | 09:19 |
| evrardjp | could you check your ansible facts /etc/openstack_deploy/ | 09:19 |
| evrardjp | could you show the full log? | 09:19 |
| evrardjp | I need to see the tasks before this task | 09:19 |
| ansibler | all tasks before succeed, here is a dump(formatting truncated): | 09:21 |
| ansibler | TASK [lxc_hosts : Remove old image cache] ok: [infra1] ok: [infra2] ok: [infra3] TASK [lxc_hosts : Set volume size] ok: [infra2] ok: [infra1] ok: [infra3] TASK [lxc_hosts : Ensure image has been pre-staged] 2017-09-26 11:59:50,213 p=11781 u=root | failed: [infra1] (item={'_ansible_parsed': True, 'changed': True, '_ansible_no_log': False, u'ansible_job_id': u'77512949089.15635', u'started': 1, '_ansible_item_result': | 09:21 |
| ansibler | not sure what to look for in /etc/openstack_deploy/ansible_facts/ There are files for all instances. | 09:25 |
| openstackgerrit | Merged openstack/openstack-ansible master: adding opendaylight services into ha proxy https://review.openstack.org/496583 | 09:26 |
| ansibler | evrardjp - thanks a lot! I will delete those ansible_facts too and try once more | 09:32 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_hosts master: vars: Drop development packages https://review.openstack.org/507449 | 09:32 |
| evrardjp | ansibler: before doing that | 09:32 |
| evrardjp | could you please paste somewhere the run? | 09:32 |
| evrardjp | to ensure it's that? | 09:32 |
| evrardjp | I'd rather keep facts, it's very useful | 09:33 |
| evrardjp | and also I'd prefer to verify that it's the cause | 09:33 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible master: [DNM] - Testing reduced packages https://review.openstack.org/507451 | 09:33 |
| ansibler | let me know what exactly is needed to retain - I can copy ansible_facts and /tmp/index-system somewhere away and then run playbooks again. Anything else needs to be kept? | 09:35 |
| evrardjp | yeah you could do that | 09:36 |
| evrardjp | but I thought you had a log to show before doing all that :) | 09:36 |
| ansibler | well, I have /openstack/log/ansible-logging/ansible.log and from there I pasted several task statuses above starting with TASK [lxc_hosts : Remove old image cache] and then up to TASK [lxc_hosts : Ensure image has been pre-staged] which failed. | 09:39 |
| ansibler | is that enough as you mentioned you need tasks from from ...lxc_cache_preparation_systemd_new.yml#L19 ? | 09:48 |
| admin0 | morning \o | 09:50 |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_hosts master: vars: Drop development packages https://review.openstack.org/507449 | 09:53 |
| *** yifei has quit IRC | 09:54 | |
| evrardjp | ansibler: yes if you could show me the log that would be great. The 10 tasks before your failure for example | 09:56 |
| *** markvoelker has joined #openstack-ansible | 10:07 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible-lxc_hosts master: vars: Drop development packages https://review.openstack.org/507449 | 10:13 |
| *** ecelik has quit IRC | 10:35 | |
| *** udesale has quit IRC | 10:37 | |
| *** markvoelker has quit IRC | 10:41 | |
| *** udesale has joined #openstack-ansible | 10:53 | |
| logan- | upstream bug for the mariadb 10.1.27 segfaults: https://jira.mariadb.org/browse/MDEV-13908 | 11:01 |
| *** sulo has left #openstack-ansible | 11:04 | |
| *** hachi_ has quit IRC | 11:05 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-specs master: Spec: Python Build/Install Process Simplification https://review.openstack.org/501296 | 11:06 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-specs master: Spec: Python Build/Install Process Simplification https://review.openstack.org/501296 | 11:07 |
| odyssey4me | evrardjp ^ HTH clarifying | 11:08 |
| evrardjp | good for me :) | 11:10 |
| evrardjp | others please vote ^ :p | 11:10 |
| ansibler | evrardjp, I ran openstack-ansible lxc-hosts-setup.yml. Gathered log, here https://pastebin.com/GnRJZuTp is the tail of it including 10 tasks before failing one. All tasks before mentioned ones complete successfully. | 11:11 |
| evrardjp | thanks | 11:14 |
| evrardjp | that helps | 11:14 |
| evrardjp | ansibler: ok I understand our process better now. I need more logs though :p | 11:23 |
| evrardjp | I need the logs for the lxc_cache_prestage steps | 11:25 |
| *** smatzek has joined #openstack-ansible | 11:26 | |
| *** yifei has joined #openstack-ansible | 11:27 | |
| evrardjp | in this file, we should redownload from all mirrors. If that task is not marked as changed -> we have a bug and/or you have a caching infrastructure that always give you the same result | 11:28 |
| *** fellypefca has joined #openstack-ansible | 11:29 | |
| ansibler | where those can be found? I greped for similar pattern in /openstack/log/ansible-logging/ansible.log but see no match | 11:30 |
| evrardjp | the slurp then gets this file, and uses the [-1] part of the appropriate line to get your result | 11:30 |
| evrardjp | in your ansible.log you should find | 11:30 |
| evrardjp | Fetch LXC image-index | 11:30 |
| evrardjp | you can dump what's starting at Set LXC cache path fact | 11:30 |
| *** yifei has quit IRC | 11:31 | |
| *** thorst has quit IRC | 11:31 | |
| fellypefca | cloudnull o lost the logs about the problem with "Create haproxy service config files" :/ But in our playbook, we are manually applying this patch https://review.openstack.org/#/c/493739 and since it's in merge conflict, it's broken | 11:31 |
| fellypefca | I lost* sorry | 11:32 |
| gokhan | hi team, Firstly thanks for your help. I achieved installing OSA in prod. I learned that we must pay attention about MTU. Management bridge MTU should be equal or greater than container management interface MTU(thanks to mwe1 about this situaton). Secondly, I had developed with my teammate horizon angularjs plugin. For activating it on horizon, I connected to horizon containers and follow these steps http://paste.openstack.org/show/621910/ . B | 11:32 |
| gokhan | ut then I got wsgi error which is can not load panel. I can't understand where I made wrong thing. can anyone help me ? this is my horizon angular plugin github link https://github.com/b3lab/safir_monitor_dashboard | 11:32 |
| evrardjp | gokhan: I am no specialist but I heard recently a few discussion about cors | 11:34 |
| evrardjp | gokhan: not sure if linked or not | 11:35 |
| evrardjp | gokhan: I think you're gonna find more help on horizon chan | 11:36 |
| evrardjp | :p | 11:36 |
| fellypefca | So, we already have the idea of this patch in another patch? https://review.openstack.org/#/c/493739 | 11:36 |
| fellypefca | or it's already merged on master? | 11:36 |
| gokhan | evrardjp: ok thanks . try to get help from horizon channel :) | 11:37 |
| evrardjp | sorry can't help you more there :p | 11:38 |
| *** markvoelker has joined #openstack-ansible | 11:39 | |
| gokhan | evrardjp : ok it is not problem :P I thing this team is very hepful. Without this team help, really I can't install OSA in prod. Thanks again :) | 11:43 |
| *** acormier has joined #openstack-ansible | 11:43 | |
| *** acormier has quit IRC | 11:47 | |
| ansibler | evrardjp, here is the log: https://pastebin.com/ME7JQf4W | 11:49 |
| *** bauruine has quit IRC | 11:52 | |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible-ops master: [WIP] Updates for Trusty VMs https://review.openstack.org/507503 | 11:53 |
| evrardjp | ansibler: great | 11:53 |
| evrardjp | so | 11:53 |
| evrardjp | the behavior seems ok | 11:54 |
| *** bauruine has joined #openstack-ansible | 11:54 | |
| evrardjp | could you curl https://uk.images.linuxcontainers.org/meta/1.0/index-system | 11:54 |
| evrardjp | on your machines ? | 11:54 |
| evrardjp | or even curl ... | grep ubuntu;xenial;amd64;default | 11:54 |
| ansibler | it works | 11:55 |
| evrardjp | what's the output ? | 11:55 |
| ansibler | sample line retrieved: ubuntu;zesty;s390x;default;20170926_03:49;/images/ubuntu/zesty/s390x/default/20170926_03:49/ | 11:55 |
| ansibler | dates are ok | 11:55 |
| evrardjp | wait | 11:56 |
| evrardjp | that's not possible | 11:56 |
| evrardjp | could you curl that with the grep? | 11:56 |
| evrardjp | just to make sure | 11:56 |
| ansibler | yes it works "ubuntu;xenial;amd64;default;20170926_03:49;/images/ubuntu/xenial/amd64/default/20170926_03:49/" but I just noticed that in second log I pasted those are from yesterday. | 11:59 |
| admin0 | curl works fine | 11:59 |
| evrardjp | and aria with both servers ? | 12:02 |
| evrardjp | these are fine ... yesterday is fine | 12:03 |
| ansibler | evrardjp, sorry must leave now. I will try to run playbooks later again and come back if anything interesting pops up. Also just to note previuosly I have had some remnants with dnsmasq config that pointed to some local container, had to remove those manually, but that could be different story. Let me know here if anything else could be helpful to gather. | 12:06 |
| evrardjp | the aria part ;) | 12:07 |
| *** dave-mcc_ has joined #openstack-ansible | 12:09 | |
| evrardjp | aria2c --max-connection-per-server=4 --continue --allow-overwrite=true --dir=/tmp --out=index-system-new https://uk.images.linuxcontainers.org/meta/1.0/index-system https://us.images.linuxcontainers.org/meta/1.0/index-system | 12:09 |
| evrardjp | and show the content of index-system-new | 12:09 |
| evrardjp | ansibler: with that we should understand | 12:10 |
| *** thorst has joined #openstack-ansible | 12:11 | |
| *** markvoelker has quit IRC | 12:12 | |
| evrardjp | logan-: did you got any issues with ceph lately? Many of our periodic jobs are failing, but I just got the result of one I did manually, and got no issue | 12:12 |
| mhayden | buenos dias | 12:21 |
| *** acormier has joined #openstack-ansible | 12:27 | |
| *** udesale has quit IRC | 12:29 | |
| *** acormier has quit IRC | 12:31 | |
| *** markvoelker has joined #openstack-ansible | 12:32 | |
| *** gparaskevas has joined #openstack-ansible | 12:42 | |
| gparaskevas | Hello folks, I have filed a bug that seems to be legit at least for centos 7 https://bugs.launchpad.net/nova/+bug/1717915 do you have any issues with rabbitmq/vhost and transport_url in ocata? | 12:43 |
| openstack | Launchpad bug 1717915 in oslo.messaging "nova services and transport_url, cannot connect to vhost if specified" [Undecided,New] | 12:43 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 12:45 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_gnocchi master: Updated from global requirements https://review.openstack.org/502805 | 12:45 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 12:45 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 12:46 |
| *** gkadam has quit IRC | 12:48 | |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible-ops master: Updates for Trusty VMs https://review.openstack.org/507503 | 12:49 |
| *** ricardoas has joined #openstack-ansible | 12:51 | |
| openstackgerrit | Matt Thompson proposed openstack/openstack-ansible-ops master: Updates for Trusty VMs https://review.openstack.org/507503 | 12:53 |
| Tahvok | I have found some missing configurations in designate role template (that our organization needs). I would like to add it and submit a review for the change. Show I open a bug for it or a blueprint? | 12:55 |
| *** esberglu has joined #openstack-ansible | 12:57 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible master: [DNM] - Testing reduced packages https://review.openstack.org/507451 | 12:58 |
| fellypefca | cloudnull: this are the logs of haproxy issue there I was talking about https://pastebin.com/Lfvm79k5 | 12:59 |
| *** Dinesh_Bhor has quit IRC | 12:59 | |
| *** woodard_ has quit IRC | 13:03 | |
| *** woodard has joined #openstack-ansible | 13:03 | |
| *** ejb1123_ is now known as ejb1123 | 13:04 | |
| openstackgerrit | Merged openstack/openstack-ansible-pip_install stable/pike: Optimize pip_install for CentOS https://review.openstack.org/507271 | 13:04 |
| ejb1123 | Good morning! I'm back, trying to deploy openstack yet again. | 13:04 |
| *** lucasxu has joined #openstack-ansible | 13:07 | |
| *** japestinho has quit IRC | 13:08 | |
| *** Dinesh_Bhor has joined #openstack-ansible | 13:08 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 13:15 |
| logan- | morning ejb1123 | 13:16 |
| logan- | evrardjp nope I have not seen any issues | 13:16 |
| logan- | which branch are you seeing this on? | 13:16 |
| evrardjp | many | 13:16 |
| evrardjp | http://logs.openstack.org/periodic/periodic-openstack-ansible-deploy-ceph-master-ubuntu-xenial/2741d4a/testr_results.html.gz | 13:16 |
| evrardjp | http://logs.openstack.org/periodic/periodic-openstack-ansible-deploy-ceph-ocata-ubuntu-xenial/1b3c24f/testr_results.html.gz | 13:16 |
| ejb1123 | I'm trying to figure out how to setup the hosts network interfaces with NetworkManager. | 13:16 |
| evrardjp | http://logs.openstack.org/periodic/periodic-openstack-ansible-upgrade-ceph-pike-ubuntu-xenial/a333184/console.html | 13:16 |
| ejb1123 | I feel like a total noob | 13:17 |
| evrardjp | wow network manager | 13:17 |
| evrardjp | I can't help you there. | 13:17 |
| logan- | so it is the upgrade jobs that are failing | 13:18 |
| ejb1123 | so I guess I just need a bridge to bridge the physical nic to the container network | 13:19 |
| pylot | Hi, trying to install tempest with os-tempest-install playbook... it runs till ensure tempest image and fails with 'Broken pipe' on the infra1_utility_container ... when i connect manually with lxc-attach and do a glance image-list no error occurs | 13:19 |
| pylot | trying 15.1.9, ocata, ubuntu 16.04.3 | 13:20 |
| pylot | it seems to come from: Error in store configuration. Adding images to store is disabled. | 13:25 |
| pylot | all deployed with openstack-ansible and adopted the production examle with nfs | 13:26 |
| *** smatzek has quit IRC | 13:27 | |
| *** udesale has joined #openstack-ansible | 13:27 | |
| evrardjp | logan-: the first two are deploys | 13:30 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 13:30 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_gnocchi master: Updated from global requirements https://review.openstack.org/502805 | 13:30 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 13:30 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 13:31 |
| logan- | evrardjp: lots of vif plugging errors in all of the nova compute logs for those failures http://logs.openstack.org/periodic/periodic-openstack-ansible-deploy-ceph-master-ubuntu-xenial/2741d4a/logs/openstack/aio1-nova/nova-compute.log.txt.gz | 13:31 |
| *** jwitko has joined #openstack-ansible | 13:31 | |
| evrardjp | yeah | 13:32 |
| evrardjp | lots of timeouts | 13:32 |
| evrardjp | i didn't check to be honest | 13:32 |
| logan- | well now that i think about it, things have been moving smoothly except https://review.openstack.org/#/c/470323/ has been stuck in the gate for a few days | 13:32 |
| logan- | and i was looking last night and saw vif plugging errors on that one | 13:33 |
| logan- | will do some more digging | 13:35 |
| *** dmsimard|off is now known as dmsimard | 13:39 | |
| logan- | here we go evrardjp | 13:39 |
| logan- | http://logs.openstack.org/23/470323/8/gate/gate-openstack-ansible-openstack-ansible-ceph-ubuntu-xenial/e26031b/logs/host/libvirt/qemu/instance-00000001.log.txt.gz | 13:39 |
| logan- | http://logs.openstack.org/23/470323/8/gate/gate-openstack-ansible-openstack-ansible-ceph-ubuntu-xenial/e26031b/logs/host/libvirt/qemu/instance-00000002.log.txt.gz | 13:40 |
| logan- | vif plugging fails because the vm is dying | 13:40 |
| evrardjp | darn how should we fix that | 13:40 |
| evrardjp | it looks like a kernel panic at the end right? | 13:41 |
| fellypefca | Hey logan- Since you review this patch (https://review.openstack.org/#/c/493739), can you help me to find if there is another patch with this idea of deploy unbound? Since there is this comment "a different direction was already applied to the code base." | 13:41 |
| fellypefca | We really need this patch to run our OSA script, but since it is in merge conflict, our script is broken. :/ | 13:42 |
| logan- | fellypefca: https://review.openstack.org/#/c/504612/ is the patch that ended up merging instead | 13:42 |
| evrardjp | thanks for digging | 13:43 |
| evrardjp | logan-: yes I saw this as issue, same issue everywhere basically | 13:44 |
| logan- | stack trace here evrardjp http://logs.openstack.org/23/470323/8/gate/gate-openstack-ansible-openstack-ansible-ceph-ubuntu-xenial/e26031b/logs/host/kern.log.txt.gz | 13:44 |
| evrardjp | for the gate job it makes sense to be killed at a certain timing, but our periodics should be alright | 13:44 |
| *** bhujay has quit IRC | 13:44 | |
| mgariepy | Found this issue on my controller : https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1715609 | 13:44 |
| openstack | Launchpad bug 1715609 in linux (Ubuntu Zesty) "kernel warning: skb_warn_bad_offload" [Medium,In progress] - Assigned to Joseph Salisbury (jsalisbury) | 13:44 |
| mgariepy | I think it was a cause of my rabbitmq cluster issues after the upgrade. | 13:45 |
| mgariepy | Also when I boot a controller with this, my lxcbr0 doesn't come up and all my containers don't have network. | 13:45 |
| logan- | :/ | 13:45 |
| fellypefca | Oh, since it's already on master, my problem with haproxy it's not related to this :/ | 13:46 |
| logan- | mgariepy interesting i guess maybe that is causing our gate issues based on the dmesg log there | 13:48 |
| mgariepy | yep. | 13:48 |
| *** drifterza has quit IRC | 13:49 | |
| evrardjp | so I checked the bug, basically updating the kernel | 13:49 |
| evrardjp | that sounds bad | 13:50 |
| evrardjp | I wonder why devstack is not catching it then | 13:50 |
| logan- | yeah between that and broken galera this is a rough week :( | 13:50 |
| evrardjp | broken galera? darn I missed something again? | 13:51 |
| evrardjp | the jobs didn't seem failing? | 13:51 |
| evrardjp | periodics | 13:51 |
| logan- | openstack ci hasnt pulled in the updated version yet | 13:51 |
| evrardjp | oh it's the .2 ? | 13:51 |
| logan- | all of my jobs using galera have had 100% failure since yesterday | 13:51 |
| logan- | no 10.1.27 | 13:52 |
| evrardjp | darn a minor. | 13:52 |
| logan- | https://jira.mariadb.org/browse/MDEV-13908 | 13:52 |
| logan- | my mirror pulled in the 10.1.27 packages yesterday afternoon | 13:52 |
| logan- | everything has been failing since then | 13:52 |
| evrardjp | sounds fun | 13:53 |
| logan- | lots of other ppl in #maria and various bugs reporting the same issue :/ | 13:53 |
| evrardjp | yeah I am having a look at your link, fun stuff. | 13:53 |
| *** taseer has joined #openstack-ansible | 13:54 | |
| *** acormier has joined #openstack-ansible | 13:57 | |
| *** yifei has joined #openstack-ansible | 13:58 | |
| *** smatzek has joined #openstack-ansible | 14:01 | |
| *** pbandark1 has joined #openstack-ansible | 14:03 | |
| *** gparaskevas has quit IRC | 14:04 | |
| *** pbandark has quit IRC | 14:04 | |
| *** pbandark1 is now known as pbandark | 14:04 | |
| *** yifei has quit IRC | 14:07 | |
| *** yifei has joined #openstack-ansible | 14:08 | |
| *** mrch has quit IRC | 14:08 | |
| ejb1123 | can OSA automate the creation of network devices? | 14:09 |
| cloudnull | mornings | 14:16 |
| ejb1123 | morning cloudnull | 14:19 |
| cloudnull | o/ | 14:19 |
| fellypefca | Morning cloudnull | 14:19 |
| ejb1123 | finally got setup-host.yml to run successfully. | 14:23 |
| ejb1123 | Is container Network ip the only ips that are used externally to the lxc container host? | 14:25 |
| *** galstrom_zzz is now known as galstrom | 14:28 | |
| spotz | morning | 14:28 |
| admin0 | \0 | 14:29 |
| cloudnull | ejb1123: container network aka management network | 14:30 |
| ejb1123 | yes | 14:30 |
| cloudnull | the lxcbr0 interface is used so containers can get access externally through the host | 14:30 |
| tasker | good morning, OSA | 14:30 |
| cloudnull | o/ | 14:30 |
| tasker | I'm going over the "newton/upgrade-guide/manual-upgrade" guide and wondered if the order of the upgrade scripts matters. must I do them in the order written in that document or can I alter the steps some? | 14:30 |
| *** lucasxu has quit IRC | 14:30 | |
| cloudnull | You can alter them however the ordering is what we've generally tested. | 14:32 |
| tasker | thought as much. thanks! | 14:32 |
| admin0 | logan-, ping .. were you able to look into the key distributions thing ? | 14:32 |
| logan- | no i have not admin0. these are the 2 files that handle the key fetch and distribution: | 14:33 |
| logan- | https://github.com/openstack/openstack-ansible-os_nova/blob/stable/ocata/tasks/nova_compute_key_populate.yml fetches the keys from the nova_computes | 14:33 |
| logan- | https://github.com/openstack/openstack-ansible-os_nova/blob/stable/ocata/tasks/nova_compute_key_distribute.yml generates the authorized_keys file in your deployment host and sends all of the keys out to the compute_hosts | 14:33 |
| *** weezS has joined #openstack-ansible | 14:34 | |
| admin0 | is there a way to tell osa only to run these 2 playbooks ? or only related tasks from these ? | 14:35 |
| logan- | all of the tasks appear to be tagged with nova-key | 14:35 |
| logan- | so os-nova-install.yml --tags nova-key | 14:35 |
| evrardjp | dear cloudnull, DimGR, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae, odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, mhayden, scarlisle, luckyinva, ntt, javeriak, spotz, vdo, jmccrory, alextricity25, jasondotstar, admin0, michaelgugino, ametts, v1k0d3n, severion, bgmccollum, darrenc, JRobinson__, asettle, colinm | 14:39 |
| evrardjp | cnamara, thorst, adreznec, eil397, qwang,nishpatwa_, cathrichardson, drifterza, hwoarang, cshen: | 14:39 |
| evrardjp | the osa bug triage will start in 1h20’. Please have a look at the bug list before starting: https://etherpad.openstack.org/p/osa-bugtriage | 14:39 |
| evrardjp | Thank you | 14:39 |
| spotz | hey evrardjp how'd the move go? | 14:40 |
| evrardjp | I have back pain. | 14:40 |
| evrardjp | :( | 14:40 |
| evrardjp | except that, almost done :) | 14:40 |
| admin0 | when are we switching to nspawn ? | 14:41 |
| admin0 | is that still the plan ? | 14:41 |
| spotz | evrardjp: :) | 14:42 |
| evrardjp | still planned | 14:42 |
| evrardjp | we're gonna keep two in parallel | 14:42 |
| admin0 | like a variable in user_variables to make a switch ? | 14:44 |
| *** lbragstad has quit IRC | 14:44 | |
| *** jwitko has quit IRC | 14:44 | |
| *** jwitko has joined #openstack-ansible | 14:45 | |
| ejb1123 | What do I run to tear down an environment? | 14:53 |
| *** lbragstad has joined #openstack-ansible | 14:54 | |
| admin0 | rm -rf ? | 14:58 |
| *** jvidal has quit IRC | 14:58 | |
| *** Oku_OS is now known as Oku_OS-away | 15:00 | |
| ivve | anyone know where i can find this, link seems broken: http://docs.openstack.org/developer/openstack-ansible/install-guide/configure-fwaas.html | 15:04 |
| ivve | (for pike) | 15:04 |
| ivve | neutron_plugin_base: with - firewall doesn't seem to do it | 15:06 |
| ivve | (any more) | 15:06 |
| ivve | which i can find here https://docs.openstack.org/openstack-ansible-os_neutron/latest/configure-network-services.html#firewall-service-optional | 15:07 |
| *** yifei has quit IRC | 15:09 | |
| *** yifei has joined #openstack-ansible | 15:10 | |
| openstackgerrit | Markos Chandras (hwoarang) proposed openstack/openstack-ansible master: [DNM] - Testing reduced packages https://review.openstack.org/507451 | 15:12 |
| *** lucasxu has joined #openstack-ansible | 15:13 | |
| *** chyka has joined #openstack-ansible | 15:21 | |
| Tahvok | I'm having rabbitmq issue during test run. Is it known issue? http://logs.openstack.org/25/504125/4/check/gate-openstack-ansible-os_designate-ansible-func-centos-7/d218428/console.html | 15:22 |
| *** yifei has quit IRC | 15:23 | |
| prometheanfire | Tahvok: me too :D | 15:24 |
| *** kristian__ has joined #openstack-ansible | 15:45 | |
| *** kristian__ has quit IRC | 15:48 | |
| *** kristian__ has joined #openstack-ansible | 15:48 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_freezer master: Updated from global requirements https://review.openstack.org/484120 | 15:53 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_gnocchi master: Updated from global requirements https://review.openstack.org/502805 | 15:53 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_horizon master: Updated from global requirements https://review.openstack.org/504683 | 15:53 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_searchlight master: Updated from global requirements https://review.openstack.org/495238 | 15:53 |
| *** fellypefca has left #openstack-ansible | 16:00 | |
| evrardjp | Bug triage cloudnull, DimGR, andymccr, d34dh0r53, hughsaunders, b3rnard0, palendae, odyssey4me, serverascode, rromans, erikmwilson, mancdaz, _shaps_, BjoernT, claco, echiu, dstanek, jwagner, ayoung, prometheanfire, evrardjp, arbrandes, mhayden, scarlisle, luckyinva, ntt, javeriak, spotz, vdo, jmccrory, alextricity25, jasondotstar, admin0, michaelgugino, ametts, v1k0d3n, severion, bgmccollum, darrenc, JRobinson__, asettle, | 16:01 |
| evrardjp | colinmcnamara, thorst, adreznec, eil397, qwang,nishpatwa_, cathrichardson, drifterza, hwoarang, cshen | 16:01 |
| evrardjp | Here is our bug list for today https://etherpad.openstack.org/p/osa-bugtriage | 16:01 |
| evrardjp | #startmeeting openstack_ansible_meeting | 16:01 |
| openstack | Meeting started Tue Sep 26 16:01:32 2017 UTC and is due to finish in 60 minutes. The chair is evrardjp. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:01 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:01 |
| *** openstack changes topic to " (Meeting topic: openstack_ansible_meeting)" | 16:01 | |
| openstack | The meeting name has been set to 'openstack_ansible_meeting' | 16:01 |
| evrardjp | #topic new bugs | 16:01 |
| *** openstack changes topic to "new bugs (Meeting topic: openstack_ansible_meeting)" | 16:01 | |
| evrardjp | https://bugs.launchpad.net/openstack-ansible/+bug/1719517 Ceilometer policy.json includes python unicode encoding | 16:01 |
| openstack | Launchpad bug 1719517 in openstack-ansible "Ceilometer policy.json includes python unicode encoding" [Undecided,New] | 16:02 |
| evrardjp | Is Christian Zunker here? | 16:02 |
| evrardjp | I suggest we move ahead, as ceilometer is not getting real traction, so we have no way to find out | 16:03 |
| evrardjp | next? | 16:03 |
| spotz | evrardjp: ping if needed need to get some work done:( | 16:03 |
| prometheanfire | o/ | 16:04 |
| evrardjp | ok let's move on | 16:05 |
| evrardjp | https://bugs.launchpad.net/openstack-ansible/+bug/1719344 defined lxc.pool if lxd_storage_pool is set | 16:05 |
| openstack | Launchpad bug 1719344 in openstack-ansible "defined lxc.pool if lxd_storage_pool is set" [Undecided,New] | 16:05 |
| evrardjp | we do | 16:06 |
| evrardjp | seems straightforward and makes sense | 16:06 |
| evrardjp | not sure why it worked before and doesn't seem to work now because it passes tempest | 16:06 |
| evrardjp | but still, it makes sense to me to configure it since we have that in the vars already | 16:06 |
| evrardjp | who wants an easy patch? | 16:07 |
| evrardjp | confirmed medium low hanging fruit ok? | 16:08 |
| hwoarang | fine by me | 16:08 |
| *** udesale has quit IRC | 16:08 | |
| evrardjp | new contributors, you should rush on these ones! :p | 16:08 |
| evrardjp | ;) | 16:08 |
| evrardjp | anyway | 16:09 |
| evrardjp | next | 16:09 |
| evrardjp | https://bugs.launchpad.net/openstack-ansible/+bug/1718979 lxc- host role failed due to absense of unzip and lack of cleanup of old /tmp/rootfs.tar.xz, Release Pike | 16:09 |
| openstack | Launchpad bug 1718979 in openstack-ansible "lxc- host role failed due to absense of unzip and lack of cleanup of old /tmp/rootfs.tar.xz, Release Pike" [Undecided,New] | 16:09 |
| evrardjp | again another problem with the prestage of images | 16:10 |
| evrardjp | I think we should raise the severity on this one | 16:10 |
| evrardjp | it's not critical because it doesn't touch gates, but it's worth fixing ASAP | 16:11 |
| evrardjp | confirmed high ? | 16:11 |
| hwoarang | ok | 16:11 |
| evrardjp | well we do have xz and the cleanup should be done automatically, am I right odyssey4me? | 16:14 |
| evrardjp | .... | 16:15 |
| evrardjp | next | 16:15 |
| evrardjp | https://bugs.launchpad.net/openstack-ansible/+bug/1718356 | 16:15 |
| openstack | Launchpad bug 1718356 in Zun "Include default config files in python wheel" [Undecided,In progress] - Assigned to Rajat Sharma (tajar29) | 16:15 |
| evrardjp | a few items are assigned to Jesse on this one, and it will completely depends on upstream. I will mark it as confirmed this way it gets out of the triage | 16:17 |
| evrardjp | marking it in progress even... when launchpad will be my friend | 16:19 |
| evrardjp | next | 16:19 |
| evrardjp | https://bugs.launchpad.net/openstack-ansible/+bug/1716908 | 16:19 |
| openstack | Launchpad bug 1716908 in openstack-ansible "Unable to create new instances after upgrade." [Undecided,New] | 16:19 |
| evrardjp | logan-: ? | 16:19 |
| evrardjp | I remember we talked about that, but I don't remember the outcome, should we hard list all those dependencies? | 16:20 |
| ejb1123 | Now I got an error with the repo-server | 16:21 |
| ejb1123 | https://usercontent.irccloud-cdn.com/file/SuKpmBoR/image.png | 16:21 |
| evrardjp | ejb1123: we are doing triage, could you wait a little? | 16:23 |
| ejb1123 | yea sure | 16:23 |
| evrardjp | I know it doesn't look like it, but we are technically doing it right :p | 16:23 |
| evrardjp | right now* | 16:23 |
| ejb1123 | sorry I'm new to this | 16:23 |
| evrardjp | so according to what I see, it looks like there is something wrong with the upgrade of packages, according to logan- 's output | 16:24 |
| evrardjp | ansible should have handled this with package latest | 16:24 |
| evrardjp | 5 | 16:25 |
| evrardjp | 4 | 16:25 |
| evrardjp | 3 | 16:25 |
| evrardjp | 2 | 16:25 |
| evrardjp | 1 | 16:25 |
| evrardjp | #topic previous week's bugs | 16:25 |
| *** openstack changes topic to "previous week's bugs (Meeting topic: openstack_ansible_meeting)" | 16:25 | |
| evrardjp | anyone got the chance to work on those bugs listed in the etherpad? | 16:25 |
| evrardjp | Has anyone the time to do it now? | 16:26 |
| Tahvok | What's etherpad ? | 16:26 |
| evrardjp | https://etherpad.openstack.org/p/osa-bugtriage | 16:26 |
| evrardjp | etherpad is collaborative tool to write text together. Here in openstack we are using it for many reasons | 16:27 |
| *** kristian__ has quit IRC | 16:27 | |
| evrardjp | I am using it as a tool to help the triage of bugs | 16:27 |
| *** kristian__ has joined #openstack-ansible | 16:28 | |
| evrardjp | but nobody is here to triage more, so I'll stop the meeting for today | 16:28 |
| evrardjp | except if anyone has something to raise | 16:28 |
| evrardjp | 5 | 16:28 |
| evrardjp | 4 | 16:28 |
| evrardjp | 3 | 16:28 |
| evrardjp | 2 | 16:28 |
| evrardjp | 1 | 16:28 |
| evrardjp | #endmeeting | 16:28 |
| *** openstack changes topic to "Launchpad: https://launchpad.net/openstack-ansible || Weekly Meetings: https://wiki.openstack.org/wiki/Meetings/openstack-ansible || Review Dashboard: https://goo.gl/tTmdgs" | 16:28 | |
| openstack | Meeting ended Tue Sep 26 16:28:45 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:28 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2017/openstack_ansible_meeting.2017-09-26-16.01.html | 16:28 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2017/openstack_ansible_meeting.2017-09-26-16.01.txt | 16:28 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2017/openstack_ansible_meeting.2017-09-26-16.01.log.html | 16:28 |
| logan- | sorry was away from desk evrardjp but yeah i guess whats happening there is our package list does not specifically call those out, so apt does not upgrade them because it is not being asked to install/upgrade them, and they are already "present" so according to apt that is a satisfied dependency | 16:32 |
| logan- | so maybe we need to call all of those packages out in the package list | 16:32 |
| *** kristian__ has quit IRC | 16:32 | |
| evrardjp | logan-: ok that's what I thought. | 16:33 |
| *** galstrom is now known as galstrom_zzz | 16:41 | |
| *** mrtenio has joined #openstack-ansible | 16:47 | |
| openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-ansible master: [DNM] Inventory migrate CLI https://review.openstack.org/504508 | 16:48 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible stable/ocata: [DNM] Testing gate https://review.openstack.org/507336 | 16:49 |
| admin0 | what happened to the https://bugs.launchpad.net/openstack-ansible/+bug/1715416 libvirtd override :D | 16:53 |
| openstack | Launchpad bug 1715416 in openstack-ansible "libvirtd override" [Undecided,New] | 16:53 |
| admin0 | waiting for ages | 16:53 |
| *** drifterza has joined #openstack-ansible | 16:53 | |
| admin0 | sorry was driving | 16:53 |
| spotz | admin0: don't IRC and drive! | 16:54 |
| admin0 | yes whch is why i could not bring up this topic .. if new topics go on top all the time, the other topics will be down there forever | 16:55 |
| *** galstrom_zzz is now known as galstrom | 16:55 | |
| evrardjp | admin0: yeah, but if nobody wants to bring it up and nobody triages it, that's the risk | 17:03 |
| evrardjp | we can magically fix all bugs :p | 17:03 |
| *** mbuil has quit IRC | 17:04 | |
| ivve | hey guys, shouldnt /etc/libvirt/libvirtd.conf ; listen_tcp = 1 ? | 17:21 |
| ivve | deployed pretty standard pike, is set to 0 | 17:21 |
| *** pbandark1 has joined #openstack-ansible | 17:22 | |
| *** pbandark has quit IRC | 17:23 | |
| *** pbandark1 is now known as pbandark | 17:23 | |
| *** kristian__ has joined #openstack-ansible | 17:29 | |
| *** kristian__ has quit IRC | 17:34 | |
| *** pbandark has quit IRC | 17:34 | |
| *** pbandark has joined #openstack-ansible | 17:36 | |
| *** galstrom is now known as galstrom_zzz | 17:40 | |
| *** taseer has quit IRC | 17:40 | |
| *** kristian__ has joined #openstack-ansible | 17:41 | |
| *** kristia__ has joined #openstack-ansible | 17:42 | |
| *** galstrom_zzz is now known as galstrom | 17:44 | |
| cloudnull | I think that is set automatically | 17:44 |
| cloudnull | it depends on your config | 17:44 |
| cloudnull | let me look | 17:44 |
| DimGR | look who is back | 17:45 |
| *** kristian__ has quit IRC | 17:45 | |
| cloudnull | o/ | 17:47 |
| *** taseer has joined #openstack-ansible | 17:47 | |
| *** lbragstad has quit IRC | 17:47 | |
| *** taseer is now known as Taseer | 17:47 | |
| cloudnull | ivve: nova_libvirtd_listen_tcp thats the op | 17:47 |
| cloudnull | you can set that to nova_libvirtd_listen_tcp=1 | 17:48 |
| ivve | hmm but should it be 1 or not? | 17:48 |
| ivve | tcp @ 16509 is failing (during live mig) | 17:48 |
| ivve | just trying to figure out the *correct* conf for libvirt(d) and nova :) | 17:49 |
| cloudnull | we take a very conservative approach. | 17:50 |
| cloudnull | there's no correct or not setup | 17:50 |
| ivve | ah | 17:50 |
| cloudnull | you can set that option if you need | 17:50 |
| cloudnull | https://github.com/openstack/openstack-ansible-os_nova/blob/90af968a682e8d316fcff7dec73b6eeb3fb7d579/tasks/main.yml#L36-L54 | 17:50 |
| ivve | did it change recently? | 17:50 |
| cloudnull | we have guards in place to ensure a correct config | 17:50 |
| ivve | i understand.. doesn't work with both enabled | 17:51 |
| ivve | so how about ssh? | 17:51 |
| *** lbragstad has joined #openstack-ansible | 17:51 | |
| logan- | ivve: the base configuration is ssh live migration, so that's why libvirt listen_tcp is not used | 17:52 |
| ivve | hmm that is wierd | 17:52 |
| ivve | i have a fresh pike and its trying to use tcp@16509 | 17:52 |
| ivve | just assumed it was a chosen approach | 17:52 |
| logan- | checking | 17:53 |
| *** Taseer has quit IRC | 17:53 | |
| *** kristia__ has quit IRC | 17:54 | |
| logan- | ivve: are you on stable/pike? or a tag? can you check ansible-role-requirements for the os_nova sha you've got? | 17:54 |
| logan- | openstack-ansible/ansible-role-requirements.yml is the file* | 17:54 |
| DimGR | is pike stable | 17:55 |
| ivve | logan-: stable/pike (16.0.?) | 17:55 |
| logan- | https://github.com/openstack/openstack-ansible-os_nova/commit/b04ae4fb290378ba8dc17007d10ba3c921b60369 is the magic SHA for you :) | 17:55 |
| logan- | https://github.com/openstack/openstack-ansible-os_nova/commits/stable/pike | 17:55 |
| ivve | checking | 17:55 |
| ivve | 1c125daf381659f4d1dba05ff833a5a52fd033ca | 17:56 |
| logan- | yea the 16.0 release tag does not have i t | 17:56 |
| ivve | jikes | 17:56 |
| logan- | it is 2 commits later | 17:56 |
| ivve | ok, ill get the latest and just rerun nova playbook | 17:56 |
| odyssey4me | once the zuulv3 migration is done, and stabilised, we should be able to release again | 17:57 |
| logan- | let me see if stable/pike has the fix | 17:57 |
| odyssey4me | is that SHA included in the current integrated a-r-r though? | 17:57 |
| logan- | it does not | 17:57 |
| ivve | oh dang | 17:57 |
| odyssey4me | ah, we should bump that then to make sure it's in 16.0.1 | 17:57 |
| logan- | you could test by modifying the sha to b04ae4fb290378ba8dc17007d10ba3c921b60369, re-run bootstrap-ansible, and run the os-nova-install.yml playbook again | 17:57 |
| ivve | ok | 17:58 |
| ivve | testing | 17:58 |
| odyssey4me | DimGR it has been stable in testing, but obviously it's new so we don't have many production environments using it yet... so YMMV | 17:58 |
| ivve | bootstrap started | 17:58 |
| ivve | also did notice some wierd stuff going on with ubuntu images not properly getting nics | 17:59 |
| ivve | ill investigate further to make sure | 17:59 |
| *** Taseer has joined #openstack-ansible | 17:59 | |
| openstackgerrit | Marc Gariépy (mgariepy) proposed openstack/openstack-ansible-lxc_hosts master: Add unzip to Centos pkg list https://review.openstack.org/507639 | 18:06 |
| *** goldenfri has joined #openstack-ansible | 18:07 | |
| *** poopcat has joined #openstack-ansible | 18:07 | |
| *** kristian__ has joined #openstack-ansible | 18:08 | |
| *** kristia__ has joined #openstack-ansible | 18:08 | |
| admin0 | anyone else upgraded to pike from ocata yet ? | 18:10 |
| *** kristi___ has joined #openstack-ansible | 18:11 | |
| *** lucasxu has quit IRC | 18:11 | |
| ivve | admin0: not upgraded but fresh deploy | 18:11 |
| ivve | if that counts | 18:11 |
| *** dave-mcc_ is now known as dave-mccowan | 18:12 | |
| *** albertcard1 has joined #openstack-ansible | 18:12 | |
| *** kristian__ has quit IRC | 18:12 | |
| *** poopcat has quit IRC | 18:14 | |
| *** kristia__ has quit IRC | 18:14 | |
| *** albertcard1 has quit IRC | 18:19 | |
| mhayden | admin0: i'll be doing that sometime soon | 18:20 |
| ivve | i guess 9gbit between compute is ok when using jumboframes | 18:20 |
| *** albertcard1 has joined #openstack-ansible | 18:21 | |
| ivve | so far im very happy with everything, hopefully this last novadeploy fixes live migrations :) | 18:22 |
| -openstackstatus- NOTICE: The infra team is continuing work to bring Zuul v3 online; expect service disruptions and please see https://docs.openstack.org/infra/manual/zuulv3.html for more information. | 18:23 | |
| *** albertcard1 has quit IRC | 18:23 | |
| *** albertcard1 has joined #openstack-ansible | 18:24 | |
| openstackgerrit | Merged openstack/openstack-ansible-ops master: Updates for Trusty VMs https://review.openstack.org/507503 | 18:25 |
| *** pbandark has quit IRC | 18:25 | |
| *** electrofelix has quit IRC | 18:25 | |
| ivve | logan-: nope, it failed at "Failed to update apt cache" | 18:33 |
| *** albertcard1 is now known as poopcat | 18:37 | |
| ivve | i guess somehow the isntallation screwed up the repos | 18:37 |
| openstackgerrit | Antony Messerli proposed openstack/openstack-ansible-ops master: WIP - Allow for Leapfrogs from other Releases https://review.openstack.org/507645 | 18:50 |
| *** gouthamr has joined #openstack-ansible | 18:52 | |
| *** lucasxu has joined #openstack-ansible | 18:52 | |
| ivve | fixed apt, trying again :P | 18:53 |
| *** gouthamr has quit IRC | 18:57 | |
| *** ianw|pto is now known as ianw | 19:01 | |
| *** hybridpollo has joined #openstack-ansible | 19:08 | |
| *** drifterza has quit IRC | 19:09 | |
| openstackgerrit | Antony Messerli proposed openstack/openstack-ansible-ops master: WIP - Allow for Leapfrogs from other Releases https://review.openstack.org/507645 | 19:10 |
| *** hybridpollo has quit IRC | 19:15 | |
| *** chhavi_ has quit IRC | 19:25 | |
| *** kristi___ has quit IRC | 19:35 | |
| ivve | logan-: b04ae4fb290378ba8dc17007d10ba3c921b60369 worked like a charm, thanks a bunch! | 19:48 |
| logan- | awesome great to hear | 19:49 |
| ivve | btw | 19:50 |
| *** aludwar has quit IRC | 19:50 | |
| ivve | if i want to use a certain interfaces, would i use overrides or are there options in user_variables i can use? | 19:50 |
| ivve | s/interfaces/interface | 19:51 |
| ivve | for migrations that is | 19:51 |
| logan- | it can get tricky.. basically with your libvirt migration URI setting you are telling each nova-compute what it should advertise as its URI for other hosts to migrate to. the other hosts will use whatever their routing/forwarding table says is the best nexthop to reach the target host | 19:52 |
| ivve | ah yea | 19:52 |
| ivve | well the heavy stuff is in ceph so i guess i can live with it, its a home lab anyway | 19:53 |
| openstackgerrit | Logan V proposed openstack/openstack-ansible stable/pike: Bump nova SHA for bug fixes https://review.openstack.org/507664 | 19:53 |
| logan- | would appreciate if you could review that ^ since you've tested it :) | 19:54 |
| ivve | will do | 19:54 |
| logan- | ty | 19:55 |
| *** lucasxu has quit IRC | 19:57 | |
| ivve | anonymous coward apparently lol :) | 20:04 |
| logan- | haha | 20:05 |
| *** pcaruana has quit IRC | 20:05 | |
| ivve | btw that seemed to have solved the issue i had with ubuntu images not rasing network interfaces | 20:08 |
| ivve | sweet | 20:08 |
| *** dave-mcc_ has joined #openstack-ansible | 20:09 | |
| *** dave-mccowan has quit IRC | 20:12 | |
| *** dave-mccowan has joined #openstack-ansible | 20:23 | |
| *** x0ff has joined #openstack-ansible | 20:24 | |
| x0ff | hello | 20:24 |
| *** dave-mcc_ has quit IRC | 20:25 | |
| x0ff | we've made a mistake of using l2population and l3 HA with openstack ocata | 20:25 |
| x0ff | would you guys consider adding sanity check so nobody does the same mistake as we did? | 20:26 |
| *** smatzek has quit IRC | 20:29 | |
| *** x0ff has quit IRC | 20:52 | |
| *** smatzek has joined #openstack-ansible | 21:04 | |
| *** gouthamr has joined #openstack-ansible | 21:12 | |
| *** thorst has quit IRC | 21:16 | |
| *** thorst has joined #openstack-ansible | 21:16 | |
| *** gouthamr has quit IRC | 21:16 | |
| *** thorst has quit IRC | 21:21 | |
| mhayden | oof, didn't make it in time to talk to x0ff | 21:33 |
| mhayden | https://bugs.launchpad.net/openstack-ansible/+bug/1719743 | 21:34 |
| openstack | Launchpad bug 1719743 in openstack-ansible "Fail when l2pop + L3 HA are enabled" [Undecided,New] | 21:34 |
| mhayden | went ahead and logged that one ;) | 21:34 |
| *** dxiri has joined #openstack-ansible | 21:40 | |
| *** pabelanger has quit IRC | 21:50 | |
| *** pabelanger_ has joined #openstack-ansible | 21:51 | |
| *** esberglu has quit IRC | 21:58 | |
| *** tasker has quit IRC | 22:12 | |
| *** esberglu has joined #openstack-ansible | 22:12 | |
| *** esberglu has quit IRC | 22:16 | |
| *** galstrom is now known as galstrom_zzz | 22:18 | |
| *** acormier has quit IRC | 22:20 | |
| *** smatzek has quit IRC | 22:27 | |
| *** dave-mccowan has quit IRC | 22:36 | |
| *** lbragstad has quit IRC | 22:40 | |
| openstackgerrit | Logan V proposed openstack/openstack-ansible-galera_server master: Implement workaround for 10.1.27 segfault https://review.openstack.org/507701 | 22:46 |
| *** threestrands has joined #openstack-ansible | 22:54 | |
| *** threestrands has quit IRC | 22:54 | |
| *** threestrands has joined #openstack-ansible | 22:54 | |
| *** chyka has quit IRC | 23:01 | |
| *** smatzek has joined #openstack-ansible | 23:04 | |
| *** thorst has joined #openstack-ansible | 23:04 | |
| *** thorst has quit IRC | 23:08 | |
| *** chyka has joined #openstack-ansible | 23:28 | |
| *** chyka has quit IRC | 23:32 | |
| *** smatzek has quit IRC | 23:32 | |
| *** acormier has joined #openstack-ansible | 23:36 | |
| *** acormier has quit IRC | 23:40 | |
| *** smatzek has joined #openstack-ansible | 23:41 | |
| *** smatzek_ has joined #openstack-ansible | 23:43 | |
| *** smatzek has quit IRC | 23:47 | |
| *** chyka has joined #openstack-ansible | 23:54 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
