16:03:18 <noonedeadpunk> #startmeeting openstack_ansible_meeting
16:03:19 <openstack> Meeting started Tue Oct 22 16:03:18 2019 UTC and is due to finish in 60 minutes.  The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:03:20 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:03:22 <openstack> The meeting name has been set to 'openstack_ansible_meeting'
16:03:28 <noonedeadpunk> #topic office hours
16:03:37 <mnaser> sorry, my food delivery guy manages to get lost and proceeds to have me turn-by-turn direction to an office building
16:03:37 <guilhermesp> o/
16:04:58 <noonedeadpunk> so, looks like we have pdf docs for all our roles done:)
16:05:18 <mnaser> ++
16:05:18 <guilhermesp> you mentioned zun earlier, done too?
16:05:19 <noonedeadpunk> and we're really close to branching. Better late than never
16:05:50 <noonedeadpunk> nah...
16:05:53 <mnaser> whats our blockers?
16:06:01 <noonedeadpunk> s/all/almost all/
16:06:28 <noonedeadpunk> so currently cirros mirror hangs which makes tempest fail
16:06:53 <noonedeadpunk> so except this infra thing I guess we're doing really good
16:07:47 <openstackgerrit> Merged openstack/openstack-ansible-os_tempest master: Don't download image when it's already there  https://review.opendev.org/689843
16:07:56 <noonedeadpunk> oh
16:07:58 <mnaser> noonedeadpunk: and we just fixed that
16:08:00 <mnaser> lol
16:08:06 <guilhermesp> hahahahaha
16:09:06 <openstackgerrit> Dmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Create Train  https://review.opendev.org/686573
16:09:07 <d34dh0r53> lol
16:09:22 <noonedeadpunk> ^ updated tempest sha in freeze
16:09:44 <mnaser> also i think one of the things we have lost when we refactored db_setup is the ability to use deploy host to create/manage db
16:10:06 <mnaser> esp if the database server is different from one service to another..
16:11:16 <noonedeadpunk> hm, why so?
16:11:31 <mnaser> we always delegate inside the galera container
16:11:33 <noonedeadpunk> we use *_db_setup_host and can set it per service
16:11:47 <mnaser> right but it assumes that .my.cnf is filled out locally on the host
16:12:06 <noonedeadpunk> yep, that's true
16:12:15 <mnaser> so say *_db_setup_host is "localhost" (i.e. deploy host) but the credentials are different (i.e. magnum server is potato, nova is banana)
16:13:41 <noonedeadpunk> not sure that it's db_setup problem... It looks like we just don't have right tempalte https://opendev.org/openstack/openstack-ansible-galera_client/src/branch/master/templates/client.my.cnf.j2
16:14:07 <mnaser> noonedeadpunk: but we template only when we run galera_client, so unless we start running galera_client everytime we run a role..
16:14:19 <mnaser> otherwise my.cnf wont be "updated" with the right values
16:14:38 <noonedeadpunk> yeah, I see what you mean
16:15:05 <mnaser> im wondering if we should just forget the idea of using .my.cnf and go back to providing login info natively into ansible tasks
16:15:26 <noonedeadpunk> But iirc we can set several sections inside my.cnf?
16:16:15 <mnaser> i dunno if mysql_database and friends know how to handle that
16:17:05 <noonedeadpunk> at least we can provide different my.cnf :)
16:17:45 <jrosser> mnaser: all of this was complication when trying to get bind-to-mgmt to work
16:17:45 <noonedeadpunk> not sure if it worth it though
16:17:54 <jrosser> anything we can do to simplifly here is ++
16:18:40 <mnaser> so it seems like there are a few different things that would actually help us improve this
16:19:09 <jrosser> is this sort of what you're looking for https://review.opendev.org/#/c/671454/1/sync/tasks/db_setup.yml
16:19:34 <mnaser> jrosser: alongside login_user and login_password too but yeah
16:22:54 <guilhermesp> indeed, that was the issue we faced when trying to bind-mgmt galera role... did we just abandoned that patch jrosser ?
16:23:00 <guilhermesp> don't remmeber the actual status
16:23:17 <jrosser> well, i hadnt realised that what is was doing there was sort of parallel function to the my.cnf
16:23:25 <mnaser> yeah it was being used :<
16:23:42 <jrosser> hence why i say we should simplify
16:23:51 <mnaser> ok good, so we will do that
16:25:58 <jrosser> unrelatedly i had a stab at a centos8 aio
16:26:07 <jrosser> which is, ummm, interesting
16:26:49 <jrosser> i have a patch here https://review.opendev.org/#/c/689629/
16:27:31 <mnaser> we're kinda gonna have to wait for rdo before we can move on
16:28:06 <jrosser> yeah, it's bringing up some other things though, like no lsyncd
16:28:33 <jrosser> i had no luck recreating the AIO network setup with networkmanager either
16:30:42 <mnaser> `/usr/bin/python: No such file or directory`
16:31:02 <jrosser> oh yes, i have some local hacking to openstack_hosts for that
16:31:49 <mnaser> we will probably start focusing a lot more centos 8 efforts
16:32:01 <mnaser> apparently the amd rome epyc performance is *huge* in 7 vs 8
16:32:12 <mnaser> but need to wait for centos 8 packages :(
16:32:21 <jrosser> http://paste.openstack.org/show/785480/
16:32:35 <mnaser> oh they published it??
16:32:42 <jrosser> oh no
16:32:52 <jrosser> i just prepared the code for where it might end up
16:32:55 <mnaser> ah yeah :(
16:33:20 <jrosser> with that it then blows up on repo server, theres no lsyncd in epel for 8
16:33:56 <jrosser> and then i figured i wasnt going to get much further
16:34:50 <jrosser> anyway theres image-id 1c1fb0a3-1202-4564-81a3-3b547646c06b at ca-ymq-1 if anyone wants to play
16:35:28 <mnaser> we should probably get public images uploaded onto our clouds of centos 8..
16:35:43 <noonedeadpunk> ++
16:35:43 <jrosser> i don't think there is an official cloud image yet
16:36:07 <noonedeadpunk> jrosser: I saw docker image at least
16:36:23 <mnaser> jrosser: oh really
16:36:30 <mnaser> howd you end up with one?
16:36:36 <noonedeadpunk> https://github.com/CentOS/sig-cloud-instance-images/tree/CentOS-8-x86_64/docker
16:36:46 <jrosser> paying attention in #infra :)
16:36:49 <mnaser> oh you're right, centos 8 images arent up at https://cloud.centos.org
16:38:03 <jrosser> mnaser: http://paste.openstack.org/show/783701/
16:38:46 <mnaser> jrosser: i also assume you could probably use dib at this point too
16:39:17 <jrosser> i would expect so yes as the centos-8 nodes are usable now
16:39:39 <jrosser> i also got buster pretty much finished
16:39:54 <jrosser> but that fails tempest in an obscure way that i don't understand
16:40:16 <jrosser> thats here https://review.opendev.org/#/c/681777/
16:40:17 <mnaser> is a tempest fail ever not obsecure :)
16:41:27 <jrosser> oh and i also dug into why it fails the lxc jobs there
16:41:55 <mnaser> jrosser: btw do you have a centos 8 vm handy right now?
16:42:11 <jrosser> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=925899
16:42:11 <openstack> Debian bug 925899 in lxc "lxc: Unprivileged containers fail to start after recent updates" [Important,Fixed]
16:42:13 <jrosser> mnaser: yes
16:42:20 <mnaser> jrosser: dnf info qemu-kvm
16:42:24 <mnaser> im curious what version is there
16:43:20 <jrosser> mnaser: http://paste.openstack.org/show/785481/
16:43:47 <mnaser> well
16:43:52 <mnaser> that's not much newer than centos 7.7 o_O
16:44:07 <jrosser> don't we pick up another repo for that usually
16:44:31 <mnaser> jrosser: so the reason we pick up that extra repo is kinda the same as ubuntu's cloud archive
16:44:39 <noonedeadpunk> wow, it's even more modern than in bionic from default repos :p
16:44:59 <mnaser> initial lts release has $version, but at some point that is so long (cause long release cycle), so new repo starts shipping that newer version
16:45:09 <mnaser> so qemu-kvm in centos 7 is actually 2.12.0
16:45:26 <mnaser> err 1.5.3**
16:45:33 <mnaser> qemu-kvm-ev (which is kinda like the "bleeding edge") is 2.12.0 (which we get from the virt repo)
16:45:57 <mnaser> i kinda wish they had shipped something newer there :\
16:47:46 <openstackgerrit> Jonathan Rosser proposed openstack/openstack-ansible master: Move debian jobs to buster  https://review.opendev.org/681777
16:47:50 <openstackgerrit> Jonathan Rosser proposed openstack/openstack-ansible master: Move debian jobs to buster  https://review.opendev.org/681777
16:48:33 <noonedeadpunk> yeah, 20.04 is going to have 3rd version...
16:48:53 <mnaser> qemu 4 is out :P
16:48:58 <mnaser> heck 4.1.0  even
16:49:17 <mnaser> 2.12.0 was released 2018-04-24 which isnt bad
16:49:23 <noonedeadpunk> actually 19.10 is shipped with 4.0
16:49:27 <noonedeadpunk> https://packages.ubuntu.com/ru/eoan/amd64/qemu-kvm/download
16:49:33 <mnaser> actually it seems 3 was the release that came right after 2.12 so its not hugely old
16:49:41 <mnaser> then 3.1 and then 4
16:51:06 <noonedeadpunk> 4.2 is even out... https://packages.ubuntu.com/ru/eoan/amd64/qemu-kvm/download
16:51:38 <noonedeadpunk> oh https://wiki.qemu.org/ChangeLog/4.2
16:51:42 <noonedeadpunk> that what I meant
16:52:11 <mnaser> noonedeadpunk: 4.2 seems wip, slated for release on 2019-12-03 according to https://wiki.qemu.org/Planning/4.2
16:52:30 <noonedeadpunk> yeah... my bad
16:52:40 <jrosser> we still have things failing on python install https://d98ff9a93a37640186b9-bbe0623e042078b1f8054ea3d769696e.ssl.cf1.rackcdn.com/690009/3/check/openstack-ansible-deploy-aio_metal-debian-stable/c481c10/job-output.txt
16:53:09 <mnaser> ERROR: Could not find a version that satisfies the requirement tempest===22.1.0
16:53:22 <jrosser> is that a broken mirror - becasue i see that on pypi
16:53:26 <mnaser> yeah it might be?
16:54:01 <noonedeadpunk> wondering if it's the same stuf as is with rpm crypto package
16:54:15 <noonedeadpunk> in terms of mirror location
16:55:35 <jrosser> ah cool this works https://review.opendev.org/#/c/690010/
17:04:59 <mnaser> #endmeeting