16:04:02 <evrardjp> #startmeeting openstack_ansible_meeting
16:04:03 <openstack> Meeting started Tue Aug  7 16:04:02 2018 UTC and is due to finish in 60 minutes.  The chair is evrardjp. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:04:04 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:04:07 <openstack> The meeting name has been set to 'openstack_ansible_meeting'
16:04:23 <evrardjp> spotz: seems I am still here.
16:04:35 <spotz> evrardjp: yep:)
16:04:38 <evrardjp> #topic rollcall
16:05:51 <evrardjp> o/
16:06:01 <evrardjp> #topic Last week highlights
16:06:30 <evrardjp> spotz: there seem to be something weird with the bot though.
16:06:42 <spotz> o/
16:06:52 <evrardjp> it doesn't seem to ack my commands
16:06:54 <evrardjp> weirdly
16:07:05 <evrardjp> let's continue and hope things are logged
16:07:13 <spotz> Yeah noticed that in the woo meeting
16:07:50 <evrardjp> evrardjp and odyssey4me thanked jrosser for the work done in Bionic
16:08:01 <evrardjp> odyssey4me: reminded that there is more work to be done
16:08:31 <evrardjp> hwoarang reminded that leap 15 work is blocked due to mariadb, lxc, and sudo
16:08:44 <evrardjp> evrardjp reminded that RC1 is this week
16:09:09 <evrardjp> and evrardjp asked to fill the etherpad https://etherpad.openstack.org/p/osa-stein-ptg
16:09:17 <bgmccollum> o/
16:09:20 <evrardjp> is there any other last week highlights?
16:09:27 <evrardjp> if not let's go to bugs
16:09:33 <evrardjp> #topic bugtriage
16:09:46 <evrardjp> please see our usual etherpad: https://etherpad.openstack.org/p/osa-bugtriage
16:09:53 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1785824
16:09:53 <openstack> Launchpad bug 1785824 in openstack-ansible "Octavia fails on octavia-housekeeping group binding" [Undecided,New]
16:10:55 <evrardjp> I am asking more details about the openstack_user_config, as this seems an inventory issue
16:11:19 <johnsom> This was just a mis-configured host networking I think...  Ping the user, I'm pretty sure they resolved that issue on their own
16:12:08 <evrardjp> johnsom: I wouldn't be surprised about a user configuration issue.
16:12:11 <evrardjp> let's move on
16:12:20 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1785651
16:12:20 <openstack> Launchpad bug 1785651 in openstack-ansible "P->Q Upgrade - Nova cert shows as down" [Undecided,New]
16:13:06 <evrardjp> I guess we need some cleanup of haproxy from P to Q as a service looks removed?
16:13:28 <evrardjp> I am not aware of what nova-cert does, and if it really was included in nova-api . Proves my internal knowledge of nova.
16:13:29 <evrardjp> :p
16:14:36 <openstackgerrit> Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/queens: Fix log compression  https://review.openstack.org/589536
16:14:46 <openstackgerrit> Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/pike: Fix log compression  https://review.openstack.org/589537
16:14:58 <evrardjp> https://github.com/openstack/nova/commit/d6d5c6be0cc0738bb7d67ca8391e5cc4f0a419f2
16:15:15 <evrardjp> I'd say confirmed and medium, as we need to remove things in haproxy, which we haven't done.
16:15:53 <bgmccollum> And the containers
16:16:06 <bgmccollum> And the service entry in Nova?
16:16:57 <evrardjp> well the thing is... I don't see anything in haproxy configuration
16:17:09 <evrardjp> so I guess it's probably only the containers and service entries
16:18:00 <bgmccollum> Maybe the inventory too? I went thought this myself, and forget all the places I had to clean up...
16:18:17 <evrardjp> we can mark it as confirmed then
16:18:25 <evrardjp> first hand experience there :p
16:19:22 <cloudnull> o/
16:19:29 <evrardjp> nova_cert is still in the inventory
16:19:42 <evrardjp> omg
16:19:46 <evrardjp> in rocky
16:19:47 <evrardjp> that's fun.
16:20:34 <evrardjp> confirmed medium?
16:21:04 <evrardjp> it doesn't hurt it's just ugly so I'd say "fix when convenient" is appropriate
16:21:19 <bgmccollum> its not hurting anything, its just the ocata-era nova-cert starts up, then fails because of object version mismatch or somethings...and quietly goes away
16:21:30 <evrardjp> ok
16:21:42 <evrardjp> let's move on
16:21:45 <evrardjp> https://bugs.launchpad.net/openstack-ansible/+bug/1785646
16:21:45 <openstack> Launchpad bug 1785646 in openstack-ansible "Cleanup of haproxy files could be more obvious" [Undecided,New]
16:21:47 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1785646
16:22:42 <evrardjp> confirmed wishlist?
16:22:51 <evrardjp> it's a bug pending to happen basically
16:23:16 <evrardjp> anyone agrees?
16:23:28 <bgmccollum> another doesn't hurt?
16:23:49 <evrardjp> bgmccollum: not sure what you mean
16:24:18 <bgmccollum> it doesn't break anything if the service is gone from the openstack side, but the haproxy config still exists...
16:24:36 <evrardjp> that's true
16:24:48 <evrardjp> so confirmed wishlist?
16:24:53 <bgmccollum> sure
16:25:10 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1785592
16:25:10 <openstack> Launchpad bug 1785592 in openstack-ansible "dynamic inventory doesn't handle is_metal: false->true change" [Undecided,New]
16:25:19 <evrardjp> (I have to go soon though)
16:25:44 <evrardjp> cloudnull: could you have a look at this ^ ?
16:25:55 <cloudnull> sure
16:25:59 <cloudnull> assign that one to me
16:26:19 <jrosser> that felt very very similar to one we patched previously
16:26:26 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1785517
16:26:26 <openstack> Launchpad bug 1785517 in openstack-ansible "iptables checksum-fill causing kernel warning error stack" [Undecided,New]
16:27:25 <evrardjp> that sounds like invalid
16:27:30 <evrardjp> cloudnull: can you confirm?
16:27:48 <bgmccollum> This is the kernel-deubg slowdown issue, right?
16:28:03 <cloudnull> evrardjp that kernel one seems valid
16:28:18 <evrardjp> yes but not in that sense I'd say
16:28:20 <cloudnull> cent7 seems to have debug logging enabled by default
16:28:33 <bgmccollum> Exacerbated by the checksumming?
16:28:40 <evrardjp> cloudnull: I have submitted a patch for that
16:28:53 <cloudnull> the checksumming is something that we should probably have a look at.
16:28:59 <cloudnull> we may not even need that anymore
16:29:25 <evrardjp> that's probably true
16:29:59 <evrardjp> but that can be an user configuration, and with my patch explaining the change for centos, we should be good, right?
16:30:04 <evrardjp> so I should mark this as invalid?
16:30:19 <evrardjp> or I keep this for the checksum investigation?
16:30:31 * cloudnull hasn't seen your patch
16:30:36 <cloudnull> however it sounds like it covers this
16:30:50 <cloudnull> or if i have seen it I dont remember
16:31:07 <bgmccollum> other distros with kernel-debug not enabled do not exhibit this problem
16:31:36 <cloudnull> or maybe they are and we're just not seeing it ?
16:31:45 <evrardjp> cloudnull: I think that's the case
16:31:47 <cloudnull> as in we're not seeing the logs
16:31:48 <evrardjp> we are not seeing it
16:31:51 <cloudnull> ++
16:31:55 <cloudnull> I would believe that
16:31:57 <evrardjp> and it doesn't hurt that much on iops
16:32:01 <bgmccollum> i though the slowdown was the massive amount of logging happening
16:32:08 <evrardjp> yes
16:32:09 * bgmccollum shrugs
16:32:11 <evrardjp> that's waht I'd say
16:32:17 <cloudnull> however I'd also believe its just a crusty old cent7 issue :)
16:32:21 <evrardjp> so if no logging, better behavior
16:32:26 <evrardjp> haha
16:32:44 <evrardjp> cloudnull: for fun we should set it to 7 1 4 7 everywhere
16:32:58 <evrardjp> in the meantime I mark this as fix released
16:32:58 <bgmccollum> for science
16:33:03 <cloudnull> ++
16:33:05 <evrardjp> bgmccollum: for science!
16:33:16 <evrardjp> fix comitted*
16:33:21 <evrardjp> I have to go
16:33:26 <evrardjp> can someone take over from here?
16:34:34 <spotz> Let me find the link evrardjp
16:34:42 <evrardjp> spotz: https://etherpad.openstack.org/p/osa-bugtriage
16:34:50 <evrardjp> we are there:
16:34:53 <evrardjp> #link https://bugs.launchpad.net/openstack-ansible/+bug/1785386
16:34:53 <openstack> Launchpad bug 1785386 in openstack-ansible "Integration of Swift and Manila with Openstack-ansible having Ceph backend?" [High,New]
16:34:58 <evrardjp> I have to run
16:35:00 <evrardjp> ttyl everyone
16:35:04 <evrardjp> #chair spotz
16:35:04 <spotz> go I got it
16:35:05 <openstack> Current chairs: evrardjp spotz
16:35:18 <evrardjp> thanks spotz :)
16:35:25 <cloudnull> later evrardjp
16:35:42 <spotz> Ok so looking at it what are the opinions?
16:36:03 <spotz> Sounds kinda wishlist to me
16:36:20 <bgmccollum> Not too familiar with Ceph, but shouldn't it "just work" if the rados gateway is deployed?
16:37:00 <spotz> I'm taking it as they'd like a config option?
16:37:23 <spotz> Now they shouldn't have changed it to high importance but another issue:)
16:37:56 <spotz> odyssey4me cloudnull jrosser?
16:38:20 <jrosser> well quite a few of these things seem to fall into the "user story" category maybe
16:38:34 <jrosser> as in its probably all possible given the right config
16:38:43 <bgmccollum> is manilla even an OSA deployable project?
16:38:47 <spotz> If it can be done already it's a docs issue. But not sure
16:38:59 <odyssey4me> it makes absolutely no sense to 'integrate swift' with ceph
16:39:08 <cloudnull> I think andymccr was working on that role?
16:39:08 <odyssey4me> and manila is not something we have a role for
16:39:23 <bgmccollum> well there you go...wish list?
16:39:34 <spotz> Or won't fix
16:39:41 <odyssey4me> well, as the question is posed there - it can be answered
16:39:57 <odyssey4me> There is no manila role in OSA. We would welcome a contribution.
16:40:51 <spotz> Ok changing to wishlist, and I'll mention the no manila role. Even if it doesn't make sense could they config ceph and swift?
16:40:54 <odyssey4me> And Swift is not something that you integrate on top of Ceph. Please look at Ceph RGW if you want a swift API with a ceph back-end.
16:41:13 <bgmccollum> I think that is the spirit of what he is asking...
16:41:35 <bgmccollum> But without a Manila role, there is nothing to more to provide.
16:42:09 <jrosser> bitrotted stuff here https://github.com/bkreitch/openstack-ansible-os_manila
16:42:30 <spotz> reply - There is not currently a Manila role within OpenStack-Ansible. We would readily accept a contribution of this role. Swift is not something that is integrated on top of Ceph. Please look at Ceph RGW if you want a swift API with a ceph back-end.
16:43:06 <bgmccollum> sounds good to me
16:43:15 <spotz> jrosser: But we can't support someone else's and I don't think we want to point folks to non-supported repos?
16:43:16 <odyssey4me> interesting, that role might be nice to import and iterate
16:43:33 <cloudnull> https://github.com/andymcc/openstack-ansible-os_manila
16:43:37 <jrosser> spotz: of course, looks like someones had a bash though
16:43:40 <cloudnull> oh sorry.
16:44:01 <spotz> Now Andy's is another story:), but anyone know the status?
16:44:17 <odyssey4me> andymccr's is clearly nothing more than scaffolding
16:44:31 <odyssey4me> and andymccr is unlikely to continue work on it given his work changes
16:44:53 <spotz> Ok I went with wish-list and what I posted in channel let me get the next one
16:45:06 <spotz> #link https://bugs.launchpad.net/openstack-ansible/+bug/1785365
16:45:06 <openstack> Launchpad bug 1785365 in openstack-ansible "LXC container network slow because of kernel debug mesg" [Undecided,New]
16:45:54 <spotz> 15 minute warning FYI
16:46:02 <spotz> cloudnull: you have any thoughts?
16:46:33 <cloudnull> that was the same issue evrardjp touched on a bit ago
16:46:35 <cloudnull> maybe duplicate
16:47:23 <spotz> I'll make a note in the etherpad for JP to take a look. Let me get the next one
16:47:50 <spotz> #link https://bugs.launchpad.net/openstack-ansible/+bug/1785068
16:47:51 <openstack> Launchpad bug 1785068 in openstack-ansible "os_glance : Ensure glance service" [Undecided,New]
16:48:47 <spotz> Looks like he's included info anout his setup and config settings
16:48:55 <jrosser> external and internal vip same -> bzzzzt
16:49:40 <jrosser> that never ends well
16:50:27 <spotz> ok told him that marked invalid and asked him to retry
16:50:37 <spotz> #link https://bugs.launchpad.net/openstack-ansible/+bug/1784949
16:50:37 <openstack> Launchpad bug 1784949 in openstack-ansible "Queen issue adding compute nodes " [Undecided,New]
16:50:40 <spotz> Next:)
16:55:15 <spotz> Has anyone done an upgrade to queens and run into this?
16:55:58 <odyssey4me> not me, but this issue is not unique to queens or an upgrade
16:56:16 <spotz> True bug or something we should document?
16:56:19 <bgmccollum> Sounds like he didn't run the setup-hosts.yml playbook?
16:56:32 <bgmccollum> Or not...
16:56:33 <bgmccollum> :.
16:56:44 <odyssey4me> the issue is that the tasks which do the linking seem to have not ran - there's not enough here to diagnose anything
16:57:19 <jrosser> if we can point to where the venv should get pulled in and ask for confirmation thats been run, and debug output if it doesnt do anything?
16:57:24 <odyssey4me> we pushed this up recently: https://review.openstack.org/#/q/Id962efe16c425424715409f071c4a304f8416001
16:58:28 <jrosser> "I found i don't have /openstack/venvs/nova-17.0.8 directory" <- the whole thing is missing though?
16:58:35 <odyssey4me> so all that stuff is done after the venv is downloaed and extracted
16:58:40 <spotz> Well May 31st, I would think he had that patch?
16:58:49 <odyssey4me> which means that somehow this person got to that task without the venv being there
16:59:00 <spotz> Ok let me ask him for his steps and maybe he missed a step?
16:59:35 <spotz> This is going to be the last one we get to today
16:59:51 <odyssey4me> so basically there should be a failed task somewhere around the venv deployment, or the task was skipped for whatever reason - either way, the reporter need to provide more info
17:00:54 <spotz> ok asked for steps and for them to look for failed tasks. And we're out of time
17:00:59 <spotz> #endmeeting