16:04:02 #startmeeting openstack_ansible_meeting 16:04:03 Meeting started Tue Aug 7 16:04:02 2018 UTC and is due to finish in 60 minutes. The chair is evrardjp. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:04:04 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:04:07 The meeting name has been set to 'openstack_ansible_meeting' 16:04:23 spotz: seems I am still here. 16:04:35 evrardjp: yep:) 16:04:38 #topic rollcall 16:05:51 o/ 16:06:01 #topic Last week highlights 16:06:30 spotz: there seem to be something weird with the bot though. 16:06:42 o/ 16:06:52 it doesn't seem to ack my commands 16:06:54 weirdly 16:07:05 let's continue and hope things are logged 16:07:13 Yeah noticed that in the woo meeting 16:07:50 evrardjp and odyssey4me thanked jrosser for the work done in Bionic 16:08:01 odyssey4me: reminded that there is more work to be done 16:08:31 hwoarang reminded that leap 15 work is blocked due to mariadb, lxc, and sudo 16:08:44 evrardjp reminded that RC1 is this week 16:09:09 and evrardjp asked to fill the etherpad https://etherpad.openstack.org/p/osa-stein-ptg 16:09:17 o/ 16:09:20 is there any other last week highlights? 16:09:27 if not let's go to bugs 16:09:33 #topic bugtriage 16:09:46 please see our usual etherpad: https://etherpad.openstack.org/p/osa-bugtriage 16:09:53 #link https://bugs.launchpad.net/openstack-ansible/+bug/1785824 16:09:53 Launchpad bug 1785824 in openstack-ansible "Octavia fails on octavia-housekeeping group binding" [Undecided,New] 16:10:55 I am asking more details about the openstack_user_config, as this seems an inventory issue 16:11:19 This was just a mis-configured host networking I think... Ping the user, I'm pretty sure they resolved that issue on their own 16:12:08 johnsom: I wouldn't be surprised about a user configuration issue. 16:12:11 let's move on 16:12:20 #link https://bugs.launchpad.net/openstack-ansible/+bug/1785651 16:12:20 Launchpad bug 1785651 in openstack-ansible "P->Q Upgrade - Nova cert shows as down" [Undecided,New] 16:13:06 I guess we need some cleanup of haproxy from P to Q as a service looks removed? 16:13:28 I am not aware of what nova-cert does, and if it really was included in nova-api . Proves my internal knowledge of nova. 16:13:29 :p 16:14:36 Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/queens: Fix log compression https://review.openstack.org/589536 16:14:46 Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible stable/pike: Fix log compression https://review.openstack.org/589537 16:14:58 https://github.com/openstack/nova/commit/d6d5c6be0cc0738bb7d67ca8391e5cc4f0a419f2 16:15:15 I'd say confirmed and medium, as we need to remove things in haproxy, which we haven't done. 16:15:53 And the containers 16:16:06 And the service entry in Nova? 16:16:57 well the thing is... I don't see anything in haproxy configuration 16:17:09 so I guess it's probably only the containers and service entries 16:18:00 Maybe the inventory too? I went thought this myself, and forget all the places I had to clean up... 16:18:17 we can mark it as confirmed then 16:18:25 first hand experience there :p 16:19:22 o/ 16:19:29 nova_cert is still in the inventory 16:19:42 omg 16:19:46 in rocky 16:19:47 that's fun. 16:20:34 confirmed medium? 16:21:04 it doesn't hurt it's just ugly so I'd say "fix when convenient" is appropriate 16:21:19 its not hurting anything, its just the ocata-era nova-cert starts up, then fails because of object version mismatch or somethings...and quietly goes away 16:21:30 ok 16:21:42 let's move on 16:21:45 https://bugs.launchpad.net/openstack-ansible/+bug/1785646 16:21:45 Launchpad bug 1785646 in openstack-ansible "Cleanup of haproxy files could be more obvious" [Undecided,New] 16:21:47 #link https://bugs.launchpad.net/openstack-ansible/+bug/1785646 16:22:42 confirmed wishlist? 16:22:51 it's a bug pending to happen basically 16:23:16 anyone agrees? 16:23:28 another doesn't hurt? 16:23:49 bgmccollum: not sure what you mean 16:24:18 it doesn't break anything if the service is gone from the openstack side, but the haproxy config still exists... 16:24:36 that's true 16:24:48 so confirmed wishlist? 16:24:53 sure 16:25:10 #link https://bugs.launchpad.net/openstack-ansible/+bug/1785592 16:25:10 Launchpad bug 1785592 in openstack-ansible "dynamic inventory doesn't handle is_metal: false->true change" [Undecided,New] 16:25:19 (I have to go soon though) 16:25:44 cloudnull: could you have a look at this ^ ? 16:25:55 sure 16:25:59 assign that one to me 16:26:19 that felt very very similar to one we patched previously 16:26:26 #link https://bugs.launchpad.net/openstack-ansible/+bug/1785517 16:26:26 Launchpad bug 1785517 in openstack-ansible "iptables checksum-fill causing kernel warning error stack" [Undecided,New] 16:27:25 that sounds like invalid 16:27:30 cloudnull: can you confirm? 16:27:48 This is the kernel-deubg slowdown issue, right? 16:28:03 evrardjp that kernel one seems valid 16:28:18 yes but not in that sense I'd say 16:28:20 cent7 seems to have debug logging enabled by default 16:28:33 Exacerbated by the checksumming? 16:28:40 cloudnull: I have submitted a patch for that 16:28:53 the checksumming is something that we should probably have a look at. 16:28:59 we may not even need that anymore 16:29:25 that's probably true 16:29:59 but that can be an user configuration, and with my patch explaining the change for centos, we should be good, right? 16:30:04 so I should mark this as invalid? 16:30:19 or I keep this for the checksum investigation? 16:30:31 * cloudnull hasn't seen your patch 16:30:36 however it sounds like it covers this 16:30:50 or if i have seen it I dont remember 16:31:07 other distros with kernel-debug not enabled do not exhibit this problem 16:31:36 or maybe they are and we're just not seeing it ? 16:31:45 cloudnull: I think that's the case 16:31:47 as in we're not seeing the logs 16:31:48 we are not seeing it 16:31:51 ++ 16:31:55 I would believe that 16:31:57 and it doesn't hurt that much on iops 16:32:01 i though the slowdown was the massive amount of logging happening 16:32:08 yes 16:32:09 * bgmccollum shrugs 16:32:11 that's waht I'd say 16:32:17 however I'd also believe its just a crusty old cent7 issue :) 16:32:21 so if no logging, better behavior 16:32:26 haha 16:32:44 cloudnull: for fun we should set it to 7 1 4 7 everywhere 16:32:58 in the meantime I mark this as fix released 16:32:58 for science 16:33:03 ++ 16:33:05 bgmccollum: for science! 16:33:16 fix comitted* 16:33:21 I have to go 16:33:26 can someone take over from here? 16:34:34 Let me find the link evrardjp 16:34:42 spotz: https://etherpad.openstack.org/p/osa-bugtriage 16:34:50 we are there: 16:34:53 #link https://bugs.launchpad.net/openstack-ansible/+bug/1785386 16:34:53 Launchpad bug 1785386 in openstack-ansible "Integration of Swift and Manila with Openstack-ansible having Ceph backend?" [High,New] 16:34:58 I have to run 16:35:00 ttyl everyone 16:35:04 #chair spotz 16:35:04 go I got it 16:35:05 Current chairs: evrardjp spotz 16:35:18 thanks spotz :) 16:35:25 later evrardjp 16:35:42 Ok so looking at it what are the opinions? 16:36:03 Sounds kinda wishlist to me 16:36:20 Not too familiar with Ceph, but shouldn't it "just work" if the rados gateway is deployed? 16:37:00 I'm taking it as they'd like a config option? 16:37:23 Now they shouldn't have changed it to high importance but another issue:) 16:37:56 odyssey4me cloudnull jrosser? 16:38:20 well quite a few of these things seem to fall into the "user story" category maybe 16:38:34 as in its probably all possible given the right config 16:38:43 is manilla even an OSA deployable project? 16:38:47 If it can be done already it's a docs issue. But not sure 16:38:59 it makes absolutely no sense to 'integrate swift' with ceph 16:39:08 I think andymccr was working on that role? 16:39:08 and manila is not something we have a role for 16:39:23 well there you go...wish list? 16:39:34 Or won't fix 16:39:41 well, as the question is posed there - it can be answered 16:39:57 There is no manila role in OSA. We would welcome a contribution. 16:40:51 Ok changing to wishlist, and I'll mention the no manila role. Even if it doesn't make sense could they config ceph and swift? 16:40:54 And Swift is not something that you integrate on top of Ceph. Please look at Ceph RGW if you want a swift API with a ceph back-end. 16:41:13 I think that is the spirit of what he is asking... 16:41:35 But without a Manila role, there is nothing to more to provide. 16:42:09 bitrotted stuff here https://github.com/bkreitch/openstack-ansible-os_manila 16:42:30 reply - There is not currently a Manila role within OpenStack-Ansible. We would readily accept a contribution of this role. Swift is not something that is integrated on top of Ceph. Please look at Ceph RGW if you want a swift API with a ceph back-end. 16:43:06 sounds good to me 16:43:15 jrosser: But we can't support someone else's and I don't think we want to point folks to non-supported repos? 16:43:16 interesting, that role might be nice to import and iterate 16:43:33 https://github.com/andymcc/openstack-ansible-os_manila 16:43:37 spotz: of course, looks like someones had a bash though 16:43:40 oh sorry. 16:44:01 Now Andy's is another story:), but anyone know the status? 16:44:17 andymccr's is clearly nothing more than scaffolding 16:44:31 and andymccr is unlikely to continue work on it given his work changes 16:44:53 Ok I went with wish-list and what I posted in channel let me get the next one 16:45:06 #link https://bugs.launchpad.net/openstack-ansible/+bug/1785365 16:45:06 Launchpad bug 1785365 in openstack-ansible "LXC container network slow because of kernel debug mesg" [Undecided,New] 16:45:54 15 minute warning FYI 16:46:02 cloudnull: you have any thoughts? 16:46:33 that was the same issue evrardjp touched on a bit ago 16:46:35 maybe duplicate 16:47:23 I'll make a note in the etherpad for JP to take a look. Let me get the next one 16:47:50 #link https://bugs.launchpad.net/openstack-ansible/+bug/1785068 16:47:51 Launchpad bug 1785068 in openstack-ansible "os_glance : Ensure glance service" [Undecided,New] 16:48:47 Looks like he's included info anout his setup and config settings 16:48:55 external and internal vip same -> bzzzzt 16:49:40 that never ends well 16:50:27 ok told him that marked invalid and asked him to retry 16:50:37 #link https://bugs.launchpad.net/openstack-ansible/+bug/1784949 16:50:37 Launchpad bug 1784949 in openstack-ansible "Queen issue adding compute nodes " [Undecided,New] 16:50:40 Next:) 16:55:15 Has anyone done an upgrade to queens and run into this? 16:55:58 not me, but this issue is not unique to queens or an upgrade 16:56:16 True bug or something we should document? 16:56:19 Sounds like he didn't run the setup-hosts.yml playbook? 16:56:32 Or not... 16:56:33 :. 16:56:44 the issue is that the tasks which do the linking seem to have not ran - there's not enough here to diagnose anything 16:57:19 if we can point to where the venv should get pulled in and ask for confirmation thats been run, and debug output if it doesnt do anything? 16:57:24 we pushed this up recently: https://review.openstack.org/#/q/Id962efe16c425424715409f071c4a304f8416001 16:58:28 "I found i don't have /openstack/venvs/nova-17.0.8 directory" <- the whole thing is missing though? 16:58:35 so all that stuff is done after the venv is downloaed and extracted 16:58:40 Well May 31st, I would think he had that patch? 16:58:49 which means that somehow this person got to that task without the venv being there 16:59:00 Ok let me ask him for his steps and maybe he missed a step? 16:59:35 This is going to be the last one we get to today 16:59:51 so basically there should be a failed task somewhere around the venv deployment, or the task was skipped for whatever reason - either way, the reporter need to provide more info 17:00:54 ok asked for steps and for them to look for failed tasks. And we're out of time 17:00:59 #endmeeting