16:02:40 <evrardjp> #startmeeting openstack_ansible_meeting
16:02:40 <openstack> Meeting started Tue Feb 13 16:02:40 2018 UTC and is due to finish in 60 minutes.  The chair is evrardjp. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:02:41 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:02:43 <openstack> The meeting name has been set to 'openstack_ansible_meeting'
16:02:47 <evrardjp> #topic rollcall
16:03:09 <evrardjp> is anyone here?
16:03:32 <evrardjp> if not we'll postpone on next week.
16:04:34 <bgmccollum> here not here
16:04:58 * admin0 is here
16:05:06 <jmccrory> o/
16:05:59 <evrardjp> ok let's do this
16:06:28 <evrardjp> #topic focus of the week
16:06:30 <evrardjp> releasing rc and branching
16:06:36 <spotz> o/
16:07:07 <evrardjp> yeah so the focus for this week is to have the rc out as soon as possible, so that we can branch stable/queens, and get rolling.
16:07:25 <evrardjp> let's move on to the bugs then :)
16:07:27 <evrardjp> #topic bugtriage
16:07:32 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1749083
16:07:33 <openstack> Launchpad bug 1749083 in openstack-ansible "Nova, Glance, Cinder, ... downtime during O to P upgrade" [Undecided,New]
16:08:01 <evrardjp> I have fixed something that is linked to that bug, but I am not sure we don't have a moving part, so I'll leave that open for now
16:08:06 <evrardjp> next
16:08:08 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1749037
16:08:08 <openstack> Launchpad bug 1749037 in openstack-ansible "Nova API downtime during Ocata upgrade" [Undecided,New]
16:08:45 <openstackgerrit> Major Hayden proposed openstack/openstack-ansible-openstack_hosts master: Install yum utilities in openstack_hosts  https://review.openstack.org/543983
16:08:52 <evrardjp> jmccrory: I think your point is valid
16:09:43 <evrardjp> please note that we didn't see something wrong in the gates, but we don't test the downtime in the integrated in O
16:09:45 <evrardjp> IIRC
16:10:00 <evrardjp> and also quite reduced in size
16:10:26 <cloudnull> o/
16:10:28 <evrardjp> jmccrory: do you plan to tackle that issue?
16:10:43 <jmccrory> yeah, kind of an edge case. the issue is with any external services that need to look up instance data during an upgrade will fail until a first compute is upgraded
16:10:46 <evrardjp> or should we just mark it as confirmed, and high.
16:10:47 <openstackgerrit> James E. Blair proposed openstack/openstack-ansible-haproxy_server stable/ocata: Zuul: Remove project name  https://review.openstack.org/543985
16:10:53 <openstackgerrit> Major Hayden proposed openstack/openstack-ansible-lxc_hosts master: Add RPM warm cache plugin  https://review.openstack.org/543986
16:11:10 <evrardjp> jmccrory: yeah but it's quite impacting :)
16:11:14 <jmccrory> i'll take it, think things can just be moved around to fix it
16:12:05 <evrardjp> maybe on a conditional basis? Now that we have facts in N to know if we are in upgrades or not
16:12:17 <evrardjp> ok I will mark it as confirmed and high.
16:12:36 <evrardjp> next
16:12:38 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1748951
16:12:39 <openstack> Launchpad bug 1748951 in openstack-ansible "Use default sysctl_file in openstack_hosts" [Undecided,New]
16:14:22 <evrardjp> we are using sysctl module which should be idempotent, and not be the cause of problems
16:14:22 <gillesMo> Hello ! I can't find documentation about ceph rgw integration with openstack-ansible. Is it possible to have Openstack Object storage handled by ceph rgw configured by openstack-ansible ? And Horizon integration too ?
16:14:47 <evrardjp> gillesMo: we are in the bug triage meeting, could you come a little later, please? Like 45 minutes?
16:15:02 <gillesMo> Ah ! Sorry, yes
16:15:21 <evrardjp> I'd mark this as Invalid or maybe, at best, confirmed wishlist.
16:15:45 <evrardjp> (because sysctl_file can be passed to the module)
16:17:10 <jmccrory> wishlist makes sense with sysctl_file
16:18:16 <evrardjp> asking for the use case, and we'll probably mark it as confirmed/wishlist next week
16:18:23 <evrardjp> next
16:18:25 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1748740
16:18:26 <openstack> Launchpad bug 1748740 in openstack-ansible "Documentation missing for adding compute host with ceph" [Undecided,New]
16:19:20 <evrardjp> I'd say confirmed and medium
16:20:26 <evrardjp> ok?
16:20:48 <evrardjp> 5
16:20:48 <jmccrory> yep
16:20:53 <evrardjp> ok next
16:21:01 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1748485
16:21:02 <openstack> Launchpad bug 1748485 in openstack-ansible "OpenStack-Ansible RabbitMQ server in OpenStack-Ansible: standalone module fails on CentOS7" [Undecided,New]
16:21:23 <evrardjp> I think there was a patch recently with rabbitmq
16:23:23 <mhayden> i haven't seen this problem yet
16:23:27 <mhayden> but i am a few weeks behind on master
16:24:16 <evrardjp> I think this is very close to what logan- is proposing: We are not using our roles externally enough
16:24:30 <evrardjp> this is a good example of someone that just tried to use our role, and went to a lot of pain.
16:24:44 <evrardjp> which prevents them from contributing
16:25:28 <evrardjp> I meant logan- has a spec up for being more "generic" in our roles
16:27:31 <evrardjp> I think we should be considering this failure as confirmed and low
16:28:18 <evrardjp> in the meantime, we'd just have to add some packages that are required in the package list
16:29:38 <evrardjp> ok moving on then
16:29:40 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1748302
16:29:41 <openstack> Launchpad bug 1748302 in openstack-ansible "Block Device Timers in Nova (Missing Variables)" [Undecided,New]
16:30:29 <evrardjp> I think this is a docs bug for operations guide.
16:30:38 <evrardjp> We should list what can be done for ops.
16:30:48 <evrardjp> this var could be a good fit.
16:30:55 <idlemind> ^^ 1748302 isn't for docs
16:31:30 <idlemind> (mine) - need the vars defined in the nova.conf template
16:32:17 <evrardjp> why?
16:32:22 <jmccrory> idlemind https://docs.openstack.org/project-deploy-guide/openstack-ansible/latest/app-advanced-config-override.html#overriding-conf-files
16:32:28 <evrardjp> I've updated the bug
16:32:34 <evrardjp> talking about config_template
16:33:00 <evrardjp> ok if I mark this as invalid?
16:33:22 <openstackgerrit> Merged openstack/openstack-ansible-memcached_server stable/ocata: Zuul: Remove project name  https://review.openstack.org/543956
16:34:14 <idlemind> jmccrory ya the override will work for my needs!
16:34:34 <evrardjp> ok marking it as invalid.
16:34:43 <evrardjp> idlemind: it's documented in the bug ;)
16:34:55 <idlemind> the nature of the bug is as a user it feels a little strange needing such an advanced feature to get 2 ubuntu instances to launch on a test lvm based cloud
16:35:45 <evrardjp> idlemind: that doesn't convene well on the bug description I'd say.
16:36:05 <idlemind> ya that piece should be added
16:36:16 <idlemind> :) i was also unaware of the ability to override at at all until a few ago
16:36:18 <evrardjp> I think another bug should be filed
16:36:34 <evrardjp> to say we are not using proper defaults, or, alternatively, nova is not using proper defaults.
16:37:13 <idlemind> k will do that!
16:37:21 <evrardjp> thanks idlemind
16:37:23 <evrardjp> next
16:37:27 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1747813
16:37:28 <openstack> Launchpad bug 1747813 in openstack-ansible "Lxc container restart has no attribute 'ansible_pkg_mgr'" [Undecided,New]
16:38:41 <evrardjp> It looks like the facts aren't gathered.
16:38:56 <jmccrory> yeah
16:39:00 <odyssey4me> yep
16:39:08 <evrardjp> I will say incomplete, we need to know how he arived to that state.
16:39:30 <odyssey4me> possibly due to the use of --limit?
16:39:56 <odyssey4me> any targeting of containers should always include lxc_hosts in the limit
16:40:45 <evrardjp> Maybe, but we don't have info. So incomplete :)
16:40:51 <evrardjp> next
16:40:53 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1747755
16:40:54 <openstack> Launchpad bug 1747755 in openstack-ansible "UCA repo included multiple times" [Undecided,New]
16:41:16 <evrardjp> mgariepy: on which node does this appear?
16:41:30 <evrardjp> If that's haproxy nodes, I know why, and we have fixed it :)
16:41:49 <openstackgerrit> Kevin Lefevre proposed openstack/openstack-ansible-os_horizon master: Add support for Horizon Octavia UI  https://review.openstack.org/543999
16:42:13 <evrardjp> also I think we have a remove of repos in our process of openstack_hosts, do we?
16:42:42 <evrardjp> I will ask mgariepy to complete the bug to see on which node this appears. Because if it happens on compute nodes, that is something we should be aware of.
16:42:55 <evrardjp> we have maybe a step missing.
16:44:07 <openstackgerrit> Major Hayden proposed openstack/openstack-ansible-os_neutron master: Add SELinux contexts for neutron log directory  https://review.openstack.org/543588
16:44:30 <evrardjp> should I mark this as incomplete?
16:45:00 <mgariepy> evrardjp, all my nodes
16:45:15 <evrardjp> oh great you're here.
16:45:21 <evrardjp> well that's weird then
16:45:31 <evrardjp> s/./!/
16:45:44 <mgariepy> the roles just ensure the new repo is added, but the old is not removed.
16:45:53 <evrardjp> but I thought we removed the old ones.
16:46:04 <mgariepy> only in queens
16:46:39 <evrardjp> yup
16:46:41 <evrardjp> https://github.com/openstack/openstack-ansible-pip_install/blob/stable/ocata/tasks/pre_install_apt.yml#L30-L43
16:46:46 <evrardjp> It seems valid and high
16:46:52 <evrardjp> medium I'd say
16:47:01 <evrardjp> not high
16:47:10 <mgariepy> it's not a huge issue since it still works, but still.. :)
16:47:11 <evrardjp> Yeah we need to add another task that removes the previous uca
16:47:20 <evrardjp> yeah
16:47:25 <evrardjp> could the cause of issues
16:47:46 <evrardjp> let's mark this as confirmed and medium or low?
16:47:59 <mgariepy> low i  guess.
16:48:31 <mgariepy> like I said it still works, if it wouldn't any deployer would have seen it on upgrade.
16:48:37 <evrardjp> yeah
16:48:43 <evrardjp> ok triaged!
16:48:45 <evrardjp> next
16:48:50 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1747754
16:48:51 <openstack> Launchpad bug 1747754 in openstack-ansible "container /var/backup mounted twice on upgrade to pike" [Undecided,New]
16:48:54 <evrardjp> wow
16:49:08 <mgariepy> yeah that one also haha
16:49:10 <evrardjp> isn't that in a remo ?
16:49:14 <evrardjp> reno*
16:49:49 <openstackgerrit> Merged openstack/openstack-ansible-os_watcher stable/ocata: Zuul: Remove project name  https://review.openstack.org/543947
16:50:19 <mgariepy> not sure i havent seen it
16:50:25 <evrardjp> doesn't seem so
16:50:39 <evrardjp> so yes, I'd say it's something we should improve in our upgrade scripts
16:50:54 <openstackgerrit> Kevin Lefevre proposed openstack/openstack-ansible master: Add supports for Horizon Octavia Dashboard  https://review.openstack.org/544002
16:50:59 <evrardjp> where does it come from?
16:51:03 <mgariepy> but the issue is only that the option changed, so the line is added and doesn't replace the old one.
16:51:35 <openstackgerrit> Kevin Lefevre proposed openstack/openstack-ansible master: Add support for Horizon Octavia Dashboard  https://review.openstack.org/544002
16:51:39 <odyssey4me> yeah, lxc allows multiple config values for the same options so we can't do full replacements
16:51:45 <odyssey4me> not as a general rule
16:51:50 <mgariepy> lxc_container_create/tasks/lxc_container_config.yml:
16:52:04 <evrardjp> that's logan- patch!
16:52:21 <evrardjp> https://github.com/openstack/openstack-ansible-lxc_container_create/commit/c82c7a60e9df2931ab0b456deb26b615d9ccd4e6
16:52:43 <evrardjp> that never happens, so I had to say it :D
16:53:03 <evrardjp> I also think we talked about it in a meeting.
16:53:18 <evrardjp> discussing what should we do...
16:53:28 <logan-> what'd i break? :D
16:53:37 <evrardjp> well
16:53:40 * logan- looks at bug
16:53:41 <evrardjp> you did what you had to
16:53:57 <evrardjp> the dual mount thing. It rings me a bell at least
16:54:03 <mgariepy> just make all the containers reboot ! haha
16:54:05 <evrardjp> should we add a reno ?
16:54:10 <evrardjp> mgariepy: haha
16:54:11 <evrardjp> yeah!
16:54:23 <evrardjp> why not, disposable, right? :p
16:54:38 <mgariepy> galera was filtered out at least.
16:54:41 <mgariepy> haha
16:54:53 <evrardjp> haha that's the only one that matters!
16:55:00 <mgariepy> mostly
16:55:03 <evrardjp> neutron agents? what?
16:55:05 <mgariepy> neutron-agent is bad
16:55:07 <evrardjp> never heard of him.
16:55:08 <mgariepy> haha
16:55:10 <openstackgerrit> Merged openstack/openstack-ansible-haproxy_server stable/ocata: Zuul: Remove project name  https://review.openstack.org/543985
16:55:18 <evrardjp> mgariepy: ;)
16:55:31 <evrardjp> ok anyway. I think it's a valid point, but I am not sure how we can fix this
16:55:39 <mgariepy> wasn't a big issue for me on that deploy (only 20 or so router)
16:55:45 <evrardjp> a line in file, to see if we have a matching line? Or a known issue
16:55:54 <mgariepy> need to regexp the mount pount.
16:55:57 <mgariepy> point**
16:55:57 <logan-> yeah we could just regexp the matching line to include both the old and new
16:56:30 <evrardjp> I think it's confirmed and low for now
16:56:33 <mgariepy> it would trigger container restart
16:57:27 <evrardjp> not if we do a lineinfile outside the process.
16:57:33 <evrardjp> like in the upgrade process.
16:57:53 <mgariepy> https://github.com/openstack/openstack-ansible-lxc_container_create/commit/c82c7a60e9df2931ab0b456deb26b615d9ccd4e6#diff-123a82ae3960af389bcb8b776bb79f32L194
16:57:53 <evrardjp> so we'd change the configs, but lxc wouldn't notice a change
16:58:06 <mgariepy> we notify of the change.
16:58:12 <logan-> i get confused about which bind mounts the lxc_container_create is responsible for vs which ones the common-tasks integrated repo handles
16:58:13 <logan-> https://review.openstack.org/#/c/408090/1/playbooks/common-tasks/os-lxc-container-setup.yml
16:58:43 <evrardjp> we should probably simplify all that too :D
16:58:45 <logan-> my commit only brought consistency to lxc_container_create to match an earlier change to the integrated repo
16:58:58 <evrardjp> I think you did good.
16:59:01 <logan-> but i'm not sure which piece is actually responsible for a dupe mount
16:59:10 <evrardjp> mmm
16:59:13 <mgariepy> hmm maybe that's where it's done.
17:00:19 <evrardjp> I think in any case, if we do a lineinfile during the upgrade process, the notify wouldn't trigger, so we'd be cleaner.
17:00:35 <evrardjp> I guess the create=dir doesn't change much on upgrades.
17:00:46 <evrardjp> so it would be fine
17:01:56 <evrardjp> no?
17:02:17 <evrardjp> let's end up for this bug, and wrap up for today
17:03:41 <evrardjp> ok, no consensus, so I'll just close up for today.
17:03:46 <evrardjp> Thanks everyone!
17:03:49 <evrardjp> #endmeeting