16:01:21 <evrardjp> #startmeeting openstack_ansible_meeting
16:01:23 <openstack> Meeting started Tue Mar  7 16:01:21 2017 UTC and is due to finish in 60 minutes.  The chair is evrardjp. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:24 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:26 <asettle> Heyo
16:01:26 <openstack> The meeting name has been set to 'openstack_ansible_meeting'
16:01:50 <spotz> \o/
16:02:11 <evrardjp> Does someone want to prioritize work?
16:02:35 <evrardjp> if not, we'll take the order planned
16:02:51 <logan-> evrardjp: order planned is fine with me (just seeing your ping earlier)
16:03:13 <andymccr> order seems fine!
16:03:17 <evrardjp> ok I'll wait for a few seconds more to let the chance to ppl to join
16:03:55 <evrardjp> well I'm not patient enough I guess.
16:03:57 <evrardjp> let's go
16:03:59 <evrardjp> https://bugs.launchpad.net/openstack-ansible/+bug/1670632
16:03:59 <openstack> Launchpad bug 1670632 in openstack-ansible "ceilometer error because gnocchiclient > 3.0 for stable/newton " [Undecided,New]
16:04:10 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1670632
16:04:21 <evrardjp> alextricity25: stevelle ?
16:04:49 <openstackgerrit> Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-lxc_hosts stable/newton: Allow the Trusty backport repo addition to be disabled  https://review.openstack.org/442592
16:04:55 <stevelle> dont know much about that one yet evrardjp
16:04:55 <evrardjp> it looks like an upstream bug that needs a bump, maybe we can't do that in that branch?
16:05:10 <evrardjp> gnocchi is using a different branching, right?
16:05:33 <stevelle> close but not necessarily in sync w/ OS
16:05:47 <odyssey4me> this is likely due to the fact that telemetry doesn't abide by the g-r process
16:06:09 <evrardjp> odyssey4me: I would agree
16:06:31 <evrardjp> do we have news from the bug reporter if everything is working on latest newton?
16:06:43 <odyssey4me> that later gnocchi client must be coming from somewhere though
16:06:49 <evrardjp> yes
16:07:21 <andymccr> i believe fabg filed the bug today
16:07:30 <odyssey4me> ok, let me try and build an environment to see if I can confirm it
16:07:51 <odyssey4me> I've self-assigned it
16:07:56 <andymccr> ok cool
16:07:56 <evrardjp> ok
16:07:59 <andymccr> thanks odyssey4me! next
16:08:00 <evrardjp> let's continue
16:08:12 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1670580
16:08:13 <openstack> Launchpad bug 1670580 in openstack-ansible "No bridges on Xenial AIO reboot" [Undecided,New]
16:08:50 <odyssey4me> this one has been coming up a lot lately
16:08:51 <evrardjp> I'd say incomplete
16:09:06 <odyssey4me> there was much discussion earlier for the same issue on multinode deployments
16:09:06 <evrardjp> We have no /etc/network/interfaces(.d)
16:09:19 <xdfil_> I had that issue yesterday
16:09:27 <xdfil_> but it was because of my interfaces file
16:09:48 <mhayden> odyssey4me / mgariepy: https://review.openstack.org/442484 <-- keystone gate unblocker
16:09:50 <andymccr> seems to be a bug with our suggested network interface files then?
16:09:55 <xdfil_> its not an OSA issue, its a "i dont use ubuntu usually" problem
16:09:58 <evrardjp> xdfil_: could you put your diagnostic on the bug?
16:10:31 <andymccr> i think the bug relates specifically to the AIO though - so that's hardcoded in - and potentially relates to our docs too
16:10:49 <evrardjp> oh it's an AIO
16:10:51 <palendae> xdfil_: Well, the reporter uses it more or less daily, so I'm not sure that it's just an unfamiliarity problem
16:11:02 <evrardjp> andymccr: yes
16:11:12 <xdfil_> oh, they might be having a seperate issue
16:11:21 <xdfil_> thats what it was in my case
16:11:28 <andymccr> id say thats kinda high - if people are configuring networking based on our docs/use cases and its wrong then we should fix that
16:11:48 <evrardjp> xdfil_: if you could have a look at our docs to see if there is something wrong there, that would be nice
16:11:54 <andymccr> ^ yeah that
16:11:58 <xdfil_> k
16:12:04 <andymccr> if its just the AIO its less critical - still a bug potentially
16:12:12 <mhayden> xgerman: finally got time to gander at your patches -- 422062 and 428979, right?
16:12:20 <evrardjp> xdfil_: sections for configuring host networking and AIO :D
16:12:52 <evrardjp> andymccr: let's classify this as confirmed and medium. It doesn't impact gates, let's fix it when convenient
16:13:08 <andymccr> evrardjp: yeah that seems fine for triage
16:13:12 <xgerman> mhayden correct
16:13:16 <spotz> andymccr: Do we want it hard coded in and then docs explaining what to do after to fix?
16:13:19 <mhayden> xgerman: lookin'
16:13:31 <xgerman> thx
16:13:54 <andymccr> spotz: at the moment i think we just want to figure out what the issue is - perhaps our docs are fine and that would be less "high" priority then
16:14:00 <evrardjp> mhayden: xgerman could you talk about that later, it's harder to triage when there is noise :p
16:14:02 <odyssey4me> spotz evrardjp andymccr not sure it's confirmed at this point, personally
16:14:09 <andymccr> odyssey4me: im testing now
16:14:10 <odyssey4me> has anyone else rebooted an AIO to confirm?
16:14:12 <mhayden> evrardjp: sorry!
16:14:14 <odyssey4me> alright
16:14:15 <andymccr> i think it is - i just rebooted an AIO and its taking a while :P
16:14:27 <andymccr> but i'll mark as confirmed if/when i get access to the AIO again
16:14:32 <andymccr> or incomplete as appropriate
16:14:35 <odyssey4me> whatever the issue we need to fix it - it may be a canary for multi-node deployments
16:14:37 <evrardjp> I think I may have got that in the past
16:14:48 <evrardjp> let's continue and we can get back to it
16:14:56 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1670556
16:14:56 <openstack> Launchpad bug 1670556 in openstack-ansible "Glance needs ceph client when rdb in use" [Undecided,New]
16:14:59 <spotz> odyssey4me: andymccr evrardjp - well write notes if its docs and assign to me, I'll ask my usual silly questions if needed:)
16:15:01 <evrardjp> mhayden: np :p
16:15:12 <evrardjp> spotz: thanks!
16:15:16 <andymccr> spotz: excellent, appreciate that
16:15:27 <evrardjp> yeah that's great to hear :)
16:16:08 <odyssey4me> ah that bug is referring to https://review.openstack.org/#/c/438671/9/playbooks/os-glance-install.yml
16:16:18 <odyssey4me> ok, I'll patch that up real quick
16:16:38 <odyssey4me> marking medium, confirmed if that's ok with everyone
16:16:41 <xdfil_> looking at the example network configs in 15rc1 they are all correct in the sense of inet manual vs inet static
16:17:02 <evrardjp> ok odyssey4me
16:17:23 <evrardjp> odyssey4me: if you are working on it I can even bump it to in progress.
16:17:35 <odyssey4me> evrardjp once the patch is submitted it'll get done
16:17:46 <evrardjp> yup yup!
16:17:48 <evrardjp> ok next
16:17:54 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1670357
16:17:54 <openstack> Launchpad bug 1670357 in openstack-ansible ""Worker" not defined in ceilometer config template for ocata" [Undecided,New]
16:18:09 <andymccr> i think thats fixed already
16:18:17 <odyssey4me> yep, patch went in to fix that yesterday
16:18:35 <andymccr> i'll find and mark released
16:18:37 <andymccr> lets go to next!
16:19:08 <evrardjp> ok
16:19:16 <odyssey4me> https://review.openstack.org/442047
16:19:20 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1670021
16:19:20 <openstack> Launchpad bug 1670021 in openstack-ansible "user_external_repo_*_list undefined errors" [Undecided,New]
16:19:59 <andymccr> ive also seen those errors so confirmed
16:20:25 <odyssey4me> yeah, me too
16:20:37 <evrardjp> Yes, it's my laziness.
16:20:47 <odyssey4me> it happens when bootstrap-host runs the pip_install role
16:20:48 <evrardjp> oh wait.
16:21:01 <odyssey4me> so we can't have that var undefined, it needs to be defined, but empty
16:21:10 <evrardjp> well I can define default to empty and it's gonna be fine
16:21:13 <evrardjp> I'll do it
16:21:16 <andymccr> yeah seems fine
16:21:18 <andymccr> ok cool
16:21:18 <evrardjp> COnfirmed low?
16:21:19 <odyssey4me> okie dokey :)
16:21:28 <odyssey4me> yeah, that's fine
16:21:40 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1669897
16:21:40 <openstack> Launchpad bug 1669897 in openstack-ansible "galera_server apt related vars difficult to override" [Undecided,New]
16:21:49 <nea1> is it normal that interfaces added later are ignored by cloud init? they don't get an IP
16:22:00 <openstackgerrit> Merged openstack/openstack-ansible-haproxy_server master: Add http-check options to haproxy service template  https://review.openstack.org/441690
16:22:22 <spotz> nea1: If I remember cloud-init right, it's only one initial startup
16:22:45 <evrardjp> is the related fix good enough?
16:23:09 <xdfil_> before you go AFK remind me I have a question about enabling iscsi multipath
16:23:15 <evrardjp> it looks good enough
16:24:12 <evrardjp> I'll move to fix commited and we'll see
16:24:34 <evrardjp> if the reporter tells us it's not enough it's gonna swap the bug status to new I think.
16:25:16 <evrardjp> so for the record andymccr just confirmed https://bugs.launchpad.net/openstack-ansible/+bug/1670580
16:25:17 <openstack> Launchpad bug 1670580 in openstack-ansible "No bridges on Xenial AIO reboot" [Medium,Confirmed]
16:25:36 <andymccr> i also see the fix - so i'll take and fix
16:25:48 <evrardjp> let's keep up the good work
16:25:50 <evrardjp> next bug
16:25:52 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1669691
16:25:52 <openstack> Launchpad bug 1669691 in openstack-ansible "repo-install.yml failed in tasks of repo_pre_build.yml " [Undecided,New]
16:26:57 <evrardjp> change in requirements/mysql_python recently?
16:27:41 <evrardjp> not sure if it's newton or not, the tag was added and removed :p
16:28:15 <alextricity25> sorry I'm late. I'm here now o/
16:28:32 <openstackgerrit> Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-pip_install stable/newton: Revert "Remove EPEL and use RDO instead"  https://review.openstack.org/442607
16:28:33 <evrardjp> cool fresh blood!
16:28:37 <evrardjp> :)
16:28:42 <andymccr> so its 14.1.0
16:28:57 <openstackgerrit> Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-pip_install stable/newton: Revert "Remove EPEL and use RDO instead"  https://review.openstack.org/442607
16:28:59 <evrardjp> so the uca change
16:29:03 <andymccr> hmm maybe]
16:29:37 <evrardjp> what was the other reason why we bumped to a higher sub version?
16:29:49 <andymccr> i believe the UCA was the reason
16:29:58 <andymccr> because we setup UCA everywhere
16:30:42 <evrardjp> yes I think so too
16:30:49 <evrardjp> ok so
16:31:12 <evrardjp> odyssey4me: could you have a look?
16:31:21 <odyssey4me> yeah I can take a look
16:31:32 <odyssey4me> not sure if this is repeatable
16:31:33 <evrardjp> it's a repo build requirements thing, I know you love it ;)
16:31:36 <odyssey4me> will see
16:31:40 <evrardjp> ok
16:31:50 <evrardjp> let's leave it as is if nobody reproduced it
16:31:56 <evrardjp> let's see how it goes next week
16:32:09 <evrardjp> next
16:32:11 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1669436
16:32:11 <openstack> Launchpad bug 1669436 in openstack-ansible "nova_libvirtd_listen_tcp won't actually work" [Undecided,New]
16:32:37 <evrardjp> andymccr: what's the status of this?
16:32:45 <andymccr> ahh yeah so that setting will cause libvirtd to not start on xenial - other than that i havnt had a chance to look into it in more detail
16:32:47 <xdfil_> is listen tcp the default?
16:32:50 <andymccr> nope
16:33:01 <andymccr> its not - so i didn't look too deep into it
16:33:02 <evrardjp> I knew ppl using this :(
16:33:03 <andymccr> i just know the service fails
16:33:16 <evrardjp> know*
16:33:25 <andymccr> evrardjp: on OSA?
16:33:27 <evrardjp> did some have a look at this?
16:33:31 <odyssey4me> evrardjp andymccr it may be worth having a chat with jamespage or looking at what the charms guys do... we both use the same platform and systems
16:33:32 <evrardjp> yup with osa
16:33:38 <evrardjp> well with other systems too
16:33:42 <andymccr> odyssey4me: yeah good point
16:33:43 <xdfil_> messing with libvirt within nova.conf has never produced the same results as messing with libvirtd directly in my experience
16:33:59 <xdfil_> so this might be outside the scope of OSA
16:34:00 <andymccr> xdfil_: its mostly the service defaults not so much the nova conf
16:34:10 <xdfil_> if the default config works then ... idk
16:34:27 <andymccr> we set like "-l" in the libvirt startup opts when using tcp and the service then doesnt start
16:34:34 <evrardjp> ok
16:34:37 <evrardjp> that's a systemd thing
16:34:39 <andymccr> but yeah odyssey4me has the best idea there i think
16:34:40 <evrardjp> we should remove that
16:34:51 <evrardjp> or I think we should I'm not really sure anymore
16:35:11 <evrardjp> oh wait
16:35:18 <evrardjp> I'm confused with another option
16:35:33 <evrardjp> forget what I just said
16:35:45 <evrardjp> well I'll take it
16:36:03 <evrardjp> In the meantime, I'd say confirmed and medium
16:36:17 <andymccr> agreed
16:36:18 <evrardjp> I guess we can trust the bug reporter
16:36:40 <evrardjp> next
16:36:44 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1669125
16:36:45 <openstack> Launchpad bug 1669125 in openstack-ansible "rabbitmq-server fails so start with ubuntu16.04" [Undecided,New]
16:37:01 <xdfil_> I'm having a hard time understanding the use case for tcp
16:37:05 <evrardjp> ok confirmed, and we'll wait for an upstream patch
16:37:13 <evrardjp> xdfil_: pure speed
16:37:17 <xdfil_> I've used it with SASL
16:37:27 <xdfil_> in non openstack
16:37:55 <spotz> evrardjp: did you still want a note added to the doc or no?
16:38:05 <xdfil_> but for openstack to me it seems like TLS is required as SASL wont work for service initiated stuff
16:38:31 <xdfil_> oh so the user is accepting the less secure scenerio... i get it now
16:38:43 <evrardjp> spotz: for the rabbitmq?
16:38:46 <evrardjp> I think it's fine
16:38:56 <evrardjp> well the patch to the doc is definitely welcomed
16:39:09 <evrardjp> to say: "hey there is a bug there if you have hostname in your bashrc"
16:39:16 <evrardjp> but I don't know where to put it.
16:39:42 <evrardjp> anyway, I've marked the bug as won't fix
16:40:08 <spotz> ev yeah
16:40:15 <evrardjp> and expressed it was a deployer/ansible issue, not really an openstack-ansible one
16:40:44 <evrardjp> Could you target the bug in your docs commit spotz?
16:40:48 <evrardjp> That would be great.
16:41:02 <spotz> evrardjp: I'll look to see if there's a good place and add if there is. Yeah do we have a link in there?
16:41:26 <evrardjp> A link for the bug in ansible or the bug link?
16:41:40 <spotz> evrardjp: I'm assuming the ansible bug not our bug:)
16:41:55 <evrardjp> Well I don't think I filed a bug, I think I submitted a PR directly
16:41:59 <evrardjp> let me have a look
16:42:20 <evrardjp> #link https://github.com/ansible/ansible/pull/22246
16:42:30 <spotz> thanks
16:42:39 <evrardjp> so not fixed until 2.3
16:43:06 <spotz> adding it to the comments so I'll remember:)
16:43:13 <evrardjp> Perfect.
16:43:19 <evrardjp> Next osa bug then!
16:43:23 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1668949
16:43:23 <openstack> Launchpad bug 1668949 in openstack-ansible "dnsmasq-dhcp log spam from lxbr0" [Undecided,New]
16:43:47 <evrardjp> andymccr: want to change to using proper dns instead?
16:43:49 <evrardjp> :D
16:44:12 <andymccr> evrardjp: haha yeah ok go make it happen ;P
16:44:22 <andymccr> that was from the ML and is confirmed - but its low i'd say
16:44:26 <andymccr> im not sure if there is a simple fix - there probably is
16:44:28 <openstackgerrit> Major Hayden proposed openstack/openstack-ansible-lxc_hosts master: Set higher priority for COPR repo  https://review.openstack.org/442613
16:44:35 <evrardjp> I think unbound can act as a basic authoritative server for some things on top of being recursive
16:45:07 <evrardjp> maybe logan- is already doing it the proper way :p
16:45:18 <evrardjp> ok I'll mark it as confirmed low
16:45:34 <evrardjp> next
16:45:38 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1667130
16:45:38 <openstack> Launchpad bug 1667130 in openstack-ansible "During N->O upgrade, nova-api-os-compute service won't start" [Undecided,Fix released]
16:45:40 <evrardjp> forget it
16:45:41 <evrardjp> :p
16:46:00 <evrardjp> next
16:46:05 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1666235
16:46:05 <openstack> Launchpad bug 1666235 in openstack-ansible "Update Zaqar to use tempest for func tests" [Undecided,New]
16:46:05 <spotz> heheh
16:46:36 <evrardjp> for me this means our role is broken if it's not properly tested :/
16:46:51 <andymccr> evrardjp: depends on your definition of "properly tested"
16:46:55 <andymccr> but yes it'd be ideal to use the tempest tests
16:47:00 <evrardjp> it's not like wishlist "that would be great if our role do the right thing"
16:47:03 <evrardjp> :p
16:47:27 <evrardjp> Well I guess what's in my head doesn't translate properly ...
16:47:34 <evrardjp> confirmed and... ?
16:47:44 <evrardjp> medium?
16:48:18 <evrardjp> it's to match the community goals of having better tests that I say that :D
16:48:22 <andymccr> evrardjp: yeah agreed
16:48:24 <andymccr> lets set it to that
16:48:34 <evrardjp> ok
16:48:52 <evrardjp> next
16:48:55 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1665667
16:48:55 <openstack> Launchpad bug 1665667 in openstack-ansible "Slow failover Recover on primary node restart" [Undecided,New]
16:49:22 <openstackgerrit> Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible master: Check for rbd as a default & optional glance back-end  https://review.openstack.org/442615
16:50:03 <palendae> Hm wonder if that's just HAproxy or something else
16:50:20 <openstackgerrit> Merged openstack/openstack-ansible-pip_install stable/ocata: Use yum priorities for RDO  https://review.openstack.org/442562
16:50:54 <evrardjp> palendae: it depends on the load balancing behavior
16:51:06 <palendae> Yeah, that too
16:51:13 <evrardjp> I think for some of the upgrades we force shutting down connections
16:51:14 <andymccr> well that interval is 12000
16:51:29 <evrardjp> So I'd say we may need some tweaking
16:51:35 <evrardjp> it's technically possible to improve this
16:51:44 <palendae> andymccr: Seconds?
16:51:47 <evrardjp> not sure how we can improve that with our current haproxy role
16:51:54 <evrardjp> palendae: ms
16:52:08 <evrardjp> haproxy are always ms except explicitly metioned
16:52:11 <palendae> Ah
16:52:12 <evrardjp> mentioned*
16:52:20 <evrardjp> I think the interval is 5s though
16:52:32 <evrardjp> yeah 5s
16:52:38 <palendae> Still, much shorter than 5m
16:52:39 <andymccr> oh thats not so bad then.
16:52:54 <evrardjp> wait
16:53:01 <evrardjp> you mean the check interval
16:53:09 <evrardjp> well I don't think it's relevant to that
16:53:13 <palendae> "If that primary node goes down we see hard failures in applications for 2-5 minutes and connection failures in the haproxy logs going for up to 20 minutes or so."
16:53:16 <evrardjp> but let me verify if the check interval is in ms
16:54:18 <evrardjp> yes it's ms
16:54:45 <evrardjp> I think it's connection termination that we should enforce for faster reconnect
16:54:56 <evrardjp> IIRC
16:54:56 <palendae> On which side?
16:55:05 <palendae> This might be relevant to RPC's minor upgrade efforts
16:55:08 <evrardjp> haproxy config
16:55:11 <palendae> Ah
16:55:15 <palendae> Well then it wouldn't be :)
16:55:35 <evrardjp> well it's good to know it, this way we can adapt
16:55:42 <palendae> Yeah
16:55:58 <evrardjp> f5 would have the same kind of issue, maybe they know it already and did it smartly "by default"
16:55:59 <evrardjp> :D
16:56:02 <evrardjp> anyway
16:56:35 <evrardjp> I'm not suprised of this, can someone technically confirm?
16:57:05 <evrardjp> I will have a look
16:57:20 <logan-> posted a few comments on the galera one
16:57:33 <evrardjp> I think the haproxy one we could have a look together :)
16:57:45 <evrardjp> ok last one for today
16:57:49 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1665568
16:57:49 <openstack> Launchpad bug 1665568 in openstack-ansible "Check for .shosts or shosts.equiv files is very slow" [Undecided,New]
16:57:54 <evrardjp> mhayden: ?
16:58:33 <evrardjp> could we use a locatedb or something like that for security role?
16:58:48 <evrardjp> is this confirmed?
17:00:04 <evrardjp> ok no answer, it's gonna be for next week!
17:00:11 <andymccr> yeah
17:00:19 <evrardjp> thanks everyone!
17:00:21 <odyssey4me> that may as well be confirmed, although I don't know if there's anything we can do about it
17:00:22 <odyssey4me> https://github.com/openstack/openstack-ansible-security/blob/1ff2d1b4aa2f866fafac10dd42907bfe0c720664/tasks/rhel7stig/auth.yml#L514
17:00:42 <odyssey4me> maybe some paths could be excluded?
17:00:54 <evrardjp> maybe use another way to locate, or be more restrictive
17:00:55 <evrardjp> yes
17:01:07 <odyssey4me> I'd not use locate because it may not be up to date
17:01:16 <odyssey4me> but being more restrictive in the find should be better
17:01:16 <evrardjp> I think we'll have to wait for mhayden answer I guess...
17:02:12 <evrardjp> for the locate an update could be used at the beginning of the role, async, and then use it at the end
17:02:27 <odyssey4me> then locate will take an age :p
17:02:31 <evrardjp> well I guess we can mark it as confirmed and wishlist
17:02:42 <odyssey4me> the reporter could also just skip the task
17:02:52 <evrardjp> fine for everyone and we wrap it up?
17:03:08 <odyssey4me> need another review for https://review.openstack.org/442607 to unblock newton patches please
17:03:51 <evrardjp> ok thanks everyone
17:03:55 <evrardjp> #endmeeting