16:05:19 <mnaser> #startmeeting openstack_ansible_meeting
16:05:20 <openstack> Meeting started Tue Aug 21 16:05:19 2018 UTC and is due to finish in 60 minutes.  The chair is mnaser. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:05:21 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:05:22 <mnaser> sorry about that slight delay
16:05:24 <openstack> The meeting name has been set to 'openstack_ansible_meeting'
16:05:26 <mnaser> #topic rollcall
16:05:38 <mnaser> o/
16:05:51 <jrosser> o/
16:05:59 <openstackgerrit> Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Minimal(ist) network config for nspawn gating  https://review.openstack.org/593230
16:06:06 <guilhermesp> o/ o/
16:06:09 <ansmith> o/
16:06:10 <chandankumar> \o
16:06:34 <arxcruz> o/
16:07:04 <mnaser> cool so
16:07:09 <mnaser> #topic what happened since last meeting
16:07:14 <mnaser> mnaser: jrosser lead a lot of work on bionic and it's almost there
16:07:18 <openstackgerrit> Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Fix nspawn bind mount register process  https://review.openstack.org/594339
16:07:21 <mnaser> jrosser: wanna expand on that?
16:07:26 <jrosser> sure
16:07:55 <openstackgerrit> Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Minimal(ist) network config for nspawn gating  https://review.openstack.org/593230
16:07:55 <jrosser> i got the first integrated build to pass today https://review.openstack.org/#/c/591287/
16:08:05 <jrosser> so i'd appreciate reviews on that and it's depends-on
16:08:37 <jrosser> mostly a massive fight for a few days with networkd on bionic, it really doesnt like how the AIO was set up with dummy interfaces and bonds
16:09:17 <jrosser> i've simplified that, and ensured there are no race conditions when bringing up the network, and that seems to have fixed it
16:09:46 <jrosser> so bionic really is very close indeed - those patches to go in and we are good i think
16:10:21 <mnaser> yay
16:10:26 <mnaser> jrosser: do you wanna tag patches with a topic
16:10:29 <mnaser> so we can easily review them
16:11:09 <jrosser> i can, although the topics may be useful when backporting? i assume this lot wants to go back to rocky too?
16:11:12 <Pradiprwt> jamesdenton: neutron playbooks executed successfully, but getting error while creating networks... I am thinking to configure mannualy... Can you please give some expalination how OVS work with container on controller node.. I mean bridge and agent communication
16:11:36 <mnaser> jrosser: i think that might be a second useful thing too
16:11:39 <mnaser> so it will help us track the work
16:12:00 <jrosser> whichever way makes it easiest is ok, just let me know or fiddle the topics
16:12:08 <mnaser> cools
16:12:46 <mnaser> so quickly
16:12:47 <mnaser> mnaser: centos jobs are back, w00t
16:12:57 <mnaser> that was a ton of work but finally got it done woo, thanks to some help and some other patches
16:13:17 <mnaser> mnaser: cleaned up jobs that were failing for a long time, we need to slowly phase in the bad ones too
16:13:29 <mnaser> mnaser: removed leap15 jobs because they were always failing, let's bring them back in when they pass
16:13:34 <mnaser> that might be relevant to hwoarang and evrardjp
16:13:42 <mnaser> they jobs are welcome back once the core issues are resolved
16:13:49 <mnaser> mnaser: os_neutron for distro centos-7 is nv, but os_nova for distro centos-7 is voting, os_neutron change for smart variables broke it os_nova for distro
16:14:01 <mnaser> cc cloudnull ^ i dunno if you wanna figure out how do we fix it or if we revert that change
16:14:36 <cloudnull> im not sure.
16:14:42 <evrardjp> mnaser: that's fine, we can re-introduce them when needed.
16:14:50 <evrardjp> at least fine for me
16:15:18 <mnaser> cloudnull: the change that failed it was https://review.openstack.org/#/c/588647/
16:16:02 <cloudnull> I think distro w/ its tendency to put things conf strange places and rename service units is something we need to get a better handle on.
16:16:08 <cloudnull> so i'd be for revertuing that pr
16:16:18 <cloudnull> though I could also be over thinking things
16:16:36 <mnaser> cloudnull: it's just the api paste file that is in a different path on rdo i guess.. so maybe we can just make a patch to do the mappings
16:17:08 <cloudnull> like we had to create a blacklist of units in designate, I suspect we need to do that elsewhere too
16:17:37 <cloudnull> mnaser are there other basic config files in that path?
16:17:50 <mnaser> cloudnull: unfortunately no, just the api-paste is in /usr/share/whatever
16:18:17 <cloudnull> rdo used to put all the packaged config in that place and then tell the service to load that config first then /etc/$name
16:18:38 <mnaser> yeah i dont think they do that except for the api paste (and api paste is differnet given its loaded not from the service file but inside the service config)
16:18:43 <mnaser> aka api_paste_config = foo
16:18:50 <cloudnull> hum
16:19:16 <cloudnull> so maybe we just need to add that path to the task and use something like "with_first_round" ?
16:19:37 <evrardjp> with first round if for beers, but I understand :)
16:19:50 <mnaser> i defer to you for deciding what you think is best, but i think thats a good approach
16:20:23 <openstackgerrit> Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Minimal(ist) network config for nspawn gating  https://review.openstack.org/593230
16:20:46 <mnaser> not to take too much on this, if cloudnull can take care of this, then we can get over that hump soon :)
16:20:49 <mnaser> mnaser: please fill this etherpad https://etherpad.openstack.org/p/osa-stein-ptg
16:20:58 <mnaser> please if you're attending, make sure you add your name and add any topics that you'd like
16:21:01 <mnaser> and finally
16:21:03 <mnaser> evrardjp: Rocky RC2 is this week.
16:21:20 <evrardjp> yup will work on that releasing, but so you know it.
16:21:28 <evrardjp> y'all know it:)
16:21:38 <mnaser> thanks for that update, we'll keep an eye and hopefully it merges cleanly and easily ;)
16:21:47 <mnaser> anything else before we jump to bugs?
16:22:27 <openstackgerrit> Kevin Carter (cloudnull) proposed openstack/openstack-ansible master: Minimal(ist) network config for nspawn gating  https://review.openstack.org/593230
16:22:27 <jrosser> memory usage is down
16:22:37 <jrosser> i don't see swapping any more
16:22:54 <mnaser> cool
16:22:59 <mnaser> i guess we have a little amount of bugs to go through
16:23:02 <mnaser> #topic bug triage
16:23:06 <mnaser> #link https://bugs.launchpad.net/openstack-ansible/+bug/1787048
16:23:06 <openstack> Launchpad bug 1787048 in openstack-ansible "attempting to disable ctrl-alt-del reboot always skips" [Undecided,New]
16:23:31 <evrardjp> jrosser: swift is still the biggest consumer?
16:23:45 <evrardjp> maybe we should keep this conversation for open discussion
16:23:47 <evrardjp> sorry
16:24:49 <mnaser> is there not a systemd module
16:24:52 <mnaser> that we can use to mask
16:24:53 <mnaser> rather than shell
16:25:08 <mnaser> maybe we can summon mhayden
16:25:15 <mnaser> https://docs.ansible.com/ansible/2.4/systemd_module.html
16:25:16 <mnaser> oh we can use that
16:25:34 <evrardjp> yeah
16:25:38 <cloudnull> http://paste.openstack.org/show/728541/ <- swift is the largest offender on my system
16:25:46 <mnaser> anyone feel like hacking on that?
16:25:53 <mnaser> it should be a straight forward thing for ansible-hardening
16:25:57 <evrardjp> cloudnull: not surprised
16:26:19 <evrardjp> cloudnull: maybe pypy would help
16:26:27 <cloudnull> ++ would be an interesting test
16:26:52 <cloudnull> mnaser I can go hack on that
16:26:54 <evrardjp> mnaser: I don't have the cycles, but it should be a low-hanging-fruit
16:27:05 <mnaser> cloudnull: cool, thanks, it should be a pretty straight forward fix
16:27:11 <cloudnull> ++
16:27:41 <mnaser> confirmed/medium/assigned to cloudnull
16:27:45 <mnaser> #link https://bugs.launchpad.net/openstack-ansible/+bug/1783668
16:27:46 <openstack> Launchpad bug 1783668 in openstack-ansible "Playbook openstack-service-setup does not run on MNAIO" [Undecided,New]
16:28:19 <evrardjp> cloudnull: or antonym  ^
16:28:26 <mnaser> i think we forgot to change status of that to incomplete last cycle
16:28:31 <mnaser> or last meeting that is
16:28:38 <evrardjp> that's fine
16:28:41 <evrardjp> for next time :)
16:28:54 <evrardjp> I am doing it then
16:28:57 <evrardjp> :p
16:29:00 <odyssey4me> that bug is confirmed, quite honestly I don't think that playbook should even run
16:29:18 <evrardjp> odyssey4me: why is the bug confirmed?
16:29:24 <odyssey4me> the reason it doesn't work is because it's running against the wrong inventory
16:29:31 <evrardjp> we don't have data to show it's confirmed
16:29:41 <odyssey4me> evrardjp: I am confirming it.
16:29:46 <evrardjp> :)
16:29:47 <antonym> yeah, I think I tried running it one day and it is the wrong inventory
16:30:03 <evrardjp> that's confirmed for me :)
16:30:07 <antonym> i'll +1 that confirm ;)
16:30:23 <evrardjp> I counted you in antonym :)
16:30:24 * mnaser doesn't know anything about MNAIO
16:30:24 <odyssey4me> I'd like to see the MNAIO thing become something that sets up the VM's and stays away from trying to do everything else
16:30:26 <mnaser> so ill defer
16:30:44 <antonym> ++
16:30:45 <evrardjp> odyssey4me: that's fair
16:30:55 <evrardjp> so you think this should be removed?
16:31:04 <evrardjp> I am fine with that.
16:31:26 <evrardjp> I think if you're setting the things in MNAIO, you can then use a "flavor" of bootstrap role to do a MNAIO
16:31:53 <evrardjp> different network config, different user variables, and more likely be re-usable for multinode testing in gates should that ever happen
16:32:00 <odyssey4me> personally, yeah, I think it should be removed as it's just confusing anyway
16:32:39 <antonym> maybe it's good for a multi-node-aio-extras or something
16:32:57 <antonym> i haven't needed to use it personally yet tho
16:32:57 <odyssey4me> I'm doing some work on the MNAIO tooling anyway so I can pick that bug up
16:32:59 <openstackgerrit> Kevin Carter (cloudnull) proposed openstack/ansible-hardening master: Correct issue with ansible hardening and systemd  https://review.openstack.org/594366
16:33:14 <odyssey4me> antonym: well, yeah - perhaps just a general ops repo thing rather than an MNAIO thing
16:33:15 <mnaser> cool co
16:33:22 <mnaser> confirmed medium assigned to odyssey4me ?
16:33:30 <odyssey4me> I'd say low
16:33:35 <mnaser> your call :)
16:33:40 <odyssey4me> it doesn't affect anything, it's just confusing
16:33:45 <evrardjp> low
16:33:48 <evrardjp> I'd say low
16:33:50 <mnaser> voila
16:33:53 <evrardjp> :p
16:33:56 <mnaser> #link https://bugs.launchpad.net/openstack-ansible/+bug/1782388
16:33:56 <openstack> Launchpad bug 1782388 in openstack-ansible "Installing Multipath But Not Enabling In Nova Causes Volume Attachment Failures" [Undecided,New]
16:34:52 <mnaser> ok so bgmccollum any updates on that one?
16:35:25 <evrardjp> I think we discussed it in a bug triage lately, after july?
16:35:29 <evrardjp> is it possible?
16:35:47 <mnaser> yeah we keep leaving it status as is to get updats about it
16:36:07 <mnaser> not sure where to go next from here
16:36:51 <evrardjp> eading last week's log
16:36:53 <evrardjp> just a sec
16:38:18 <evrardjp> http://eavesdrop.openstack.org/meetings/openstack_ansible_meeting/2018/openstack_ansible_meeting.2018-08-14-16.01.log.txt
16:38:31 <evrardjp> I think with this text in mind, we can keep it open until someone shows up
16:38:46 <mnaser> cool
16:38:49 <mnaser> well in that case
16:38:54 <mnaser> we're wrapped up bug triage
16:38:57 <mnaser> #topic open discussion
16:39:23 <spotz> Sign up if you’re coming to the PTG!!!
16:39:39 <spotz> And remember Mon-Tues will be the OPS meetup
16:40:53 <jrosser> there still looks to be a good deal to be done on memory usage http://paste.openstack.org/show/728542/
16:41:27 <jrosser> thats a bionic AIO so very representative of thuse ubuntu gate jobs
16:42:03 <mnaser> seems a lot for galera
16:42:16 <jrosser> it's surprising, yes
16:43:06 <jrosser> all these tools count different things in different ways though
16:43:36 <evrardjp> yeah that's very different to what I used to see
16:43:44 <evrardjp> you can check with dstat
16:43:48 <cloudnull> sadly i gotta run. be back later.
16:44:08 <evrardjp> so that is everywhere the same
16:44:11 <evrardjp> cloudnull: cya later!
16:44:30 <spotz> Seeya cloudnull
16:44:57 <guilhermesp> take care cloudnull
16:47:53 <mnaser> I guess that wraps it up
16:48:00 <mnaser> #endmeeting