15:00:09 <mnasiadka> #startmeeting kolla
15:00:09 <opendevmeet> Meeting started Wed Jan 26 15:00:09 2022 UTC and is due to finish in 60 minutes.  The chair is mnasiadka. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:09 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:09 <opendevmeet> The meeting name has been set to 'kolla'
15:00:17 <mnasiadka> #topic rollcall
15:00:36 <yoctozepto> o/
15:00:56 <mnasiadka> o/
15:04:06 <mgoddard> O\
15:04:19 <mnasiadka> oh, and I wanted to write two person meeting ;-)
15:04:21 <hrw> ]o[
15:04:32 <mnasiadka> four, hooray
15:04:40 <mnasiadka> #topic agenda
15:04:50 <mnasiadka> * Announcements
15:04:50 <mnasiadka> * Review action items from the last meeting
15:04:50 <mnasiadka> * CI status
15:04:50 <mnasiadka> * Release tasks
15:04:50 <mnasiadka> * Current cycle planning
15:04:52 <mnasiadka> * Additional agenda (from whiteboard)
15:04:52 <mnasiadka> * Open discussion
15:05:24 <mnasiadka> #topic Announcements
15:05:34 <mnasiadka> None from me
15:06:16 <mnasiadka> #topic Review action items from the last meeting
15:06:52 <mnasiadka> 1. mnasiadka to triage security bugs and update them with resolution plan (if needed)
15:06:52 <mnasiadka> 2. mnasiadka post a patch for docs - standard topics that should be discussed over PTG and then revisited in mid-cycle
15:06:52 <mnasiadka> 3. kevko to let frickler know whether osism's solution is fine for his use case
15:06:52 <mnasiadka> 4. yoctozepto to review going-podman patches
15:06:52 <mnasiadka> 5. mgoddard to review going-podman patches
15:06:54 <mnasiadka> 6. halomiva/hinermar propose change for podman
15:07:27 <mnasiadka> 1. is in progress - haproxy tls ciphers bug doesn't seem to be a bug, I have a WIP change for using testssl.sh on our ciphers to detect issues earlier
15:07:29 <mgoddard> I reviewed the systemd patch
15:07:37 <mnasiadka> 2. still to be done
15:07:41 <yoctozepto> I did not have time to review
15:08:59 <mnasiadka> Ok then
15:09:13 <mnasiadka> #action mnasiadka to triage security bugs and update them with resolution plan (if needed)
15:09:21 <mnasiadka> #action mnasiadka post a patch for docs - standard topics that should be discussed over PTG and then revisited in mid-cycle
15:09:35 <mnasiadka> #action kevko to let frickler know whether osism's solution is fine for his use case
15:09:51 <mnasiadka> #action yoctozepto to review going-podman patches
15:10:10 <mnasiadka> did halomiva/hinermar propose changes for podman?
15:10:25 <mnasiadka> (I guess not only the systemd ones)
15:11:57 <mnasiadka> #action halomiva/hinermar propose changes for podman
15:12:04 <mnasiadka> They are not here, so let's add it back as well.
15:12:17 <mnasiadka> #topic CI status
15:13:24 * hrw 
15:14:13 <mgoddard> they did push the podman changes, although they need a rebase
15:14:17 <hrw> I hope that current train fixes make it pass.
15:14:33 <hrw> once they merge I do not want to touch train again
15:14:47 <yoctozepto> hrw: nobody does
15:15:01 <mnasiadka> Ok, Train situation is clear more or less
15:15:32 <mnasiadka> What about CentOS Linux 8 retirement in CI - is it done on Kolla/Kolla-Ansible side?
15:15:38 <yoctozepto> we have some cephadm issues on ubuntu
15:15:49 <yoctozepto> mnasiadka: pending at least due to bifrost
15:16:06 <yoctozepto> but I have green light from dtantsur to drop the offending CI config
15:16:09 <mnasiadka> Ok, I also need to take a look in the Kayobe patches.
15:16:50 <mgoddard> mnasiadka: I think the ussuri kayobe patch is stuck
15:17:05 <mgoddard> possibly we need to drop upgrade jobs, or backport cs8 to train
15:17:41 <mnasiadka> mgoddard: I noticed it's failing on post upgrade vm tests, but cirros seems to get up - so I don't really know what is the issue
15:18:23 <mnasiadka> But yes, dropping upgrade jobs seems like a good solution (and an easy one).
15:18:51 <mnasiadka> Anyway, let's keep an eye on the CI related work that needs to be done...
15:19:38 <mnasiadka> #topic Release tasks
15:19:57 <mnasiadka> It's R-9 week,
15:20:21 <yoctozepto> so close to R-8
15:20:35 <mnasiadka> R-8: Switch binary images to current release¶
15:21:05 <hrw> easy and worked last time I checked
15:21:21 <mnasiadka> So that's next week - any volunteer?
15:21:24 <hrw> can prepare patch for it
15:21:37 <mnasiadka> thanks
15:21:51 <mnasiadka> #action hrw to prepare patches for R-8 Switch binary images to current release
15:22:08 <mnasiadka> #topic Current cycle planning
15:23:32 <hrw> https://review.opendev.org/c/openstack/kolla/+/826033 - train is +1 on zuul
15:23:49 <mnasiadka> So, we have some features here and there, I see Podman has gained some traction, so I guess we're on the right track.
15:25:22 <mnasiadka> At some point last cycle we promised to look at VMware NSXP support patches - we all know there's no CI for that, but I guess we can't leave those patches hanging there.
15:25:34 <mnasiadka> Add support for VMware NSXP https://review.opendev.org/c/openstack/kolla-ansible/+/807404
15:25:34 <mnasiadka> Add support for VMware First Class Disk (FCD) https://review.opendev.org/c/openstack/kolla-ansible/+/808760
15:25:42 <mnasiadka> Is there any volunteer to look into those?
15:26:21 <opendevreview> Marcin Juszkiewicz proposed openstack/kolla master: Switch to use Yoga binary packages  https://review.opendev.org/c/openstack/kolla/+/826488
15:27:22 <mgoddard> I could take a look
15:27:26 <mgoddard> looks like you already did mnasiadka
15:27:39 <mnasiadka> Yes, I already did some time ago, I'll sign us both into that.
15:28:17 <mnasiadka> Ansible core 2.12 - I think I saw a change around bumping it up
15:28:38 <mnasiadka> Yes, merged already, crossing it out.
15:28:52 <mgoddard> do we have one for kayobe?
15:29:01 <yoctozepto> mnasiadka: yeah, done
15:29:06 <yoctozepto> works perfectly
15:29:08 <yoctozepto> I love such changes
15:29:23 <hrw> +1 for FCD
15:29:33 <mgoddard> didn't 2.12 come with the requirement of python 3.8+?
15:29:40 <yoctozepto> hrw: FCD?
15:29:47 <yoctozepto> ah
15:29:49 <yoctozepto> vmware
15:30:13 <mnasiadka> mgoddard: they claimed in the docs - that it will require it, but it seems to work...
15:30:32 <mgoddard> we do test max & min versions in CI
15:30:37 <yoctozepto> we test max only on ubuntu
15:30:38 <mgoddard> I think centos gets min
15:30:39 <yoctozepto> where it is 3.8
15:30:44 <yoctozepto> yeah
15:30:58 <mgoddard> ok
15:31:36 <mgoddard> probably we could do it using version-specific deps in requirements.txt for kayobe
15:31:50 <mnasiadka> Ok, so Kayobe part is to be done
15:32:44 <mnasiadka> Added to the Kayobe list
15:33:27 <mnasiadka> What about Let's Encrypt? It would be nice to get that merged, but it seems we would need to discuss that again - I personally don't like the supervisor approach (and we probably don't need it anymore - since we bumped haproxy to 2.2)
15:33:55 <mgoddard> is that part still present in the current patch?
15:34:05 <mnasiadka> It's in the Kolla patch to add supervisor packages
15:34:07 <mgoddard> I thought we dropped/split out  the contentious haproxy bits
15:34:51 <mnasiadka> It would be nice to discuss if headphoneJames would have time to put cycles in it
15:35:15 <mnasiadka> or is he more interested in Keystone System Scope
15:35:26 <mnasiadka> I'll try to reach out to him outside of this meeting.
15:35:37 <mgoddard> I think he's put some time into it recently
15:36:24 <mgoddard> secure RBAC work is in fairly good shape for yoga I think
15:36:30 <mnasiadka> That's nice.
15:37:21 <mnasiadka> I haven't seen any progress in the AWS OpenSearch topic - I'll bug parallax later about that.
15:37:50 <mnasiadka> Ok, that's probably enough on the planning side.
15:38:34 <mnasiadka> No additional topics on the whiteboard (for the meeting today)
15:38:59 <mnasiadka> #topic Open discussion
15:39:18 <mnasiadka> Any topic anyone? ;-)
15:39:29 <mgoddard> Libvirt on the host
15:39:58 <mgoddard> I've been working on some kolla-ansible & kayobe patches to run libvirt on the host, rather than in a container
15:40:18 <mgoddard> we see this as a step towards decoupling the host & container OS
15:40:33 <mgoddard> https://review.opendev.org/c/openstack/kayobe/+/825359
15:40:44 <mgoddard> https://review.opendev.org/c/openstack/kolla-ansible/+/825357
15:40:51 <yoctozepto> yet we rely on host's libvirt then
15:40:55 <yoctozepto> why is it better?
15:41:37 <mgoddard> the client-server relationship is loose
15:41:44 <mgoddard> libvirt-kernel less so
15:42:22 <mgoddard> we have seen some issues even between CentOS minor version mismatches
15:42:32 <yoctozepto> interesting
15:42:40 <yoctozepto> I thought it was kernel-independent
15:42:48 <yoctozepto> as libvirt only manages the qemu processes
15:42:49 <mnasiadka> I'm not personally convinced it's a better way than libvirt in container, maybe an alternative for Kayobe users
15:42:58 <yoctozepto> perhaps it's actually the issue with qemu
15:43:13 <yoctozepto> i.e., the thing we should be using from the host is qemu
15:43:14 <mnasiadka> Around kernel mismatches - yes, we've had one or two occurences, but maybe because we haven't been pinning images to CentOS minor release
15:43:16 <mgoddard> most likely
15:43:48 <mgoddard> mnasiadka: but doesn't that suggest that a good match is required?
15:43:49 <yoctozepto> I guess I don't mind it this way, newer libvirt+qemu rarely offer enough if the kernel is old
15:44:19 <yoctozepto> and people preferring less stress will run homogenous hosts
15:44:31 <yoctozepto> so no need to homogenise at the container level
15:45:00 <mgoddard> if we were to consider mixing container & OS distros, it seems more sensible to keep libvirt on the host, right?
15:45:41 <mgoddard> another change that is coming down the line is modular libvirt daemons
15:45:52 <mgoddard> https://libvirt.org/daemons.html
15:46:24 <mgoddard> at some point libvirt will make this mandatory
15:46:28 <mnasiadka> Well, I would be happy to see at least proper logging in the containerised libvirt (currently we don't support virtlogd)
15:46:42 <mgoddard> so you'd need a few virt*d containers
15:47:32 <yoctozepto> yeah, I agree going host is smarter
15:48:07 <yoctozepto> more so that we are likely the only project containerising libvirt
15:48:14 <yoctozepto> khekhe
15:48:26 <yoctozepto> so all dog food there is our dog food
15:48:40 <mgoddard> my current approach makes the libvirt container optional in kolla-ansible
15:48:43 <yoctozepto> and it's usually nice to share this kind of bowl with others
15:49:05 <mgoddard> switching to host paths (/var/lib/libvirt etc) when disabled
15:49:15 <mgoddard> kayobe handles config & deploy of libvirt
15:49:38 <mgoddard> but it uses the stackhpc.libvirt-host ansible role, which could easily be picked up by kolla-ansible if it so desired
15:49:59 <yoctozepto> makes sense
15:50:12 <yoctozepto> I don't mind
15:50:16 <mgoddard> just thought I'd share what will be coming down the pipe
15:50:19 <yoctozepto> but let's make this optional anyway
15:50:48 <mgoddard> +1
15:52:20 <mgoddard> </libvirt on host>
15:55:06 <DK4> it is currently possible to place the haproxy on separate nodes or do i need to turn off haproxy completely and do the lbs on my own?
15:56:00 <mnasiadka> ok, let's finish the meeting
15:56:09 <mnasiadka> thanks guys for attending
15:56:25 <yoctozepto> thanks mnasiadka for chairing
15:56:30 <mnasiadka> #endmeeting