15:00:09 #startmeeting kolla 15:00:09 Meeting started Wed Jan 26 15:00:09 2022 UTC and is due to finish in 60 minutes. The chair is mnasiadka. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:09 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:09 The meeting name has been set to 'kolla' 15:00:17 #topic rollcall 15:00:36 o/ 15:00:56 o/ 15:04:06 O\ 15:04:19 oh, and I wanted to write two person meeting ;-) 15:04:21 ]o[ 15:04:32 four, hooray 15:04:40 #topic agenda 15:04:50 * Announcements 15:04:50 * Review action items from the last meeting 15:04:50 * CI status 15:04:50 * Release tasks 15:04:50 * Current cycle planning 15:04:52 * Additional agenda (from whiteboard) 15:04:52 * Open discussion 15:05:24 #topic Announcements 15:05:34 None from me 15:06:16 #topic Review action items from the last meeting 15:06:52 1. mnasiadka to triage security bugs and update them with resolution plan (if needed) 15:06:52 2. mnasiadka post a patch for docs - standard topics that should be discussed over PTG and then revisited in mid-cycle 15:06:52 3. kevko to let frickler know whether osism's solution is fine for his use case 15:06:52 4. yoctozepto to review going-podman patches 15:06:52 5. mgoddard to review going-podman patches 15:06:54 6. halomiva/hinermar propose change for podman 15:07:27 1. is in progress - haproxy tls ciphers bug doesn't seem to be a bug, I have a WIP change for using testssl.sh on our ciphers to detect issues earlier 15:07:29 I reviewed the systemd patch 15:07:37 2. still to be done 15:07:41 I did not have time to review 15:08:59 Ok then 15:09:13 #action mnasiadka to triage security bugs and update them with resolution plan (if needed) 15:09:21 #action mnasiadka post a patch for docs - standard topics that should be discussed over PTG and then revisited in mid-cycle 15:09:35 #action kevko to let frickler know whether osism's solution is fine for his use case 15:09:51 #action yoctozepto to review going-podman patches 15:10:10 did halomiva/hinermar propose changes for podman? 15:10:25 (I guess not only the systemd ones) 15:11:57 #action halomiva/hinermar propose changes for podman 15:12:04 They are not here, so let's add it back as well. 15:12:17 #topic CI status 15:13:24 * hrw 15:14:13 they did push the podman changes, although they need a rebase 15:14:17 I hope that current train fixes make it pass. 15:14:33 once they merge I do not want to touch train again 15:14:47 hrw: nobody does 15:15:01 Ok, Train situation is clear more or less 15:15:32 What about CentOS Linux 8 retirement in CI - is it done on Kolla/Kolla-Ansible side? 15:15:38 we have some cephadm issues on ubuntu 15:15:49 mnasiadka: pending at least due to bifrost 15:16:06 but I have green light from dtantsur to drop the offending CI config 15:16:09 Ok, I also need to take a look in the Kayobe patches. 15:16:50 mnasiadka: I think the ussuri kayobe patch is stuck 15:17:05 possibly we need to drop upgrade jobs, or backport cs8 to train 15:17:41 mgoddard: I noticed it's failing on post upgrade vm tests, but cirros seems to get up - so I don't really know what is the issue 15:18:23 But yes, dropping upgrade jobs seems like a good solution (and an easy one). 15:18:51 Anyway, let's keep an eye on the CI related work that needs to be done... 15:19:38 #topic Release tasks 15:19:57 It's R-9 week, 15:20:21 so close to R-8 15:20:35 R-8: Switch binary images to current releaseĀ¶ 15:21:05 easy and worked last time I checked 15:21:21 So that's next week - any volunteer? 15:21:24 can prepare patch for it 15:21:37 thanks 15:21:51 #action hrw to prepare patches for R-8 Switch binary images to current release 15:22:08 #topic Current cycle planning 15:23:32 https://review.opendev.org/c/openstack/kolla/+/826033 - train is +1 on zuul 15:23:49 So, we have some features here and there, I see Podman has gained some traction, so I guess we're on the right track. 15:25:22 At some point last cycle we promised to look at VMware NSXP support patches - we all know there's no CI for that, but I guess we can't leave those patches hanging there. 15:25:34 Add support for VMware NSXP https://review.opendev.org/c/openstack/kolla-ansible/+/807404 15:25:34 Add support for VMware First Class Disk (FCD) https://review.opendev.org/c/openstack/kolla-ansible/+/808760 15:25:42 Is there any volunteer to look into those? 15:26:21 Marcin Juszkiewicz proposed openstack/kolla master: Switch to use Yoga binary packages https://review.opendev.org/c/openstack/kolla/+/826488 15:27:22 I could take a look 15:27:26 looks like you already did mnasiadka 15:27:39 Yes, I already did some time ago, I'll sign us both into that. 15:28:17 Ansible core 2.12 - I think I saw a change around bumping it up 15:28:38 Yes, merged already, crossing it out. 15:28:52 do we have one for kayobe? 15:29:01 mnasiadka: yeah, done 15:29:06 works perfectly 15:29:08 I love such changes 15:29:23 +1 for FCD 15:29:33 didn't 2.12 come with the requirement of python 3.8+? 15:29:40 hrw: FCD? 15:29:47 ah 15:29:49 vmware 15:30:13 mgoddard: they claimed in the docs - that it will require it, but it seems to work... 15:30:32 we do test max & min versions in CI 15:30:37 we test max only on ubuntu 15:30:38 I think centos gets min 15:30:39 where it is 3.8 15:30:44 yeah 15:30:58 ok 15:31:36 probably we could do it using version-specific deps in requirements.txt for kayobe 15:31:50 Ok, so Kayobe part is to be done 15:32:44 Added to the Kayobe list 15:33:27 What about Let's Encrypt? It would be nice to get that merged, but it seems we would need to discuss that again - I personally don't like the supervisor approach (and we probably don't need it anymore - since we bumped haproxy to 2.2) 15:33:55 is that part still present in the current patch? 15:34:05 It's in the Kolla patch to add supervisor packages 15:34:07 I thought we dropped/split out the contentious haproxy bits 15:34:51 It would be nice to discuss if headphoneJames would have time to put cycles in it 15:35:15 or is he more interested in Keystone System Scope 15:35:26 I'll try to reach out to him outside of this meeting. 15:35:37 I think he's put some time into it recently 15:36:24 secure RBAC work is in fairly good shape for yoga I think 15:36:30 That's nice. 15:37:21 I haven't seen any progress in the AWS OpenSearch topic - I'll bug parallax later about that. 15:37:50 Ok, that's probably enough on the planning side. 15:38:34 No additional topics on the whiteboard (for the meeting today) 15:38:59 #topic Open discussion 15:39:18 Any topic anyone? ;-) 15:39:29 Libvirt on the host 15:39:58 I've been working on some kolla-ansible & kayobe patches to run libvirt on the host, rather than in a container 15:40:18 we see this as a step towards decoupling the host & container OS 15:40:33 https://review.opendev.org/c/openstack/kayobe/+/825359 15:40:44 https://review.opendev.org/c/openstack/kolla-ansible/+/825357 15:40:51 yet we rely on host's libvirt then 15:40:55 why is it better? 15:41:37 the client-server relationship is loose 15:41:44 libvirt-kernel less so 15:42:22 we have seen some issues even between CentOS minor version mismatches 15:42:32 interesting 15:42:40 I thought it was kernel-independent 15:42:48 as libvirt only manages the qemu processes 15:42:49 I'm not personally convinced it's a better way than libvirt in container, maybe an alternative for Kayobe users 15:42:58 perhaps it's actually the issue with qemu 15:43:13 i.e., the thing we should be using from the host is qemu 15:43:14 Around kernel mismatches - yes, we've had one or two occurences, but maybe because we haven't been pinning images to CentOS minor release 15:43:16 most likely 15:43:48 mnasiadka: but doesn't that suggest that a good match is required? 15:43:49 I guess I don't mind it this way, newer libvirt+qemu rarely offer enough if the kernel is old 15:44:19 and people preferring less stress will run homogenous hosts 15:44:31 so no need to homogenise at the container level 15:45:00 if we were to consider mixing container & OS distros, it seems more sensible to keep libvirt on the host, right? 15:45:41 another change that is coming down the line is modular libvirt daemons 15:45:52 https://libvirt.org/daemons.html 15:46:24 at some point libvirt will make this mandatory 15:46:28 Well, I would be happy to see at least proper logging in the containerised libvirt (currently we don't support virtlogd) 15:46:42 so you'd need a few virt*d containers 15:47:32 yeah, I agree going host is smarter 15:48:07 more so that we are likely the only project containerising libvirt 15:48:14 khekhe 15:48:26 so all dog food there is our dog food 15:48:40 my current approach makes the libvirt container optional in kolla-ansible 15:48:43 and it's usually nice to share this kind of bowl with others 15:49:05 switching to host paths (/var/lib/libvirt etc) when disabled 15:49:15 kayobe handles config & deploy of libvirt 15:49:38 but it uses the stackhpc.libvirt-host ansible role, which could easily be picked up by kolla-ansible if it so desired 15:49:59 makes sense 15:50:12 I don't mind 15:50:16 just thought I'd share what will be coming down the pipe 15:50:19 but let's make this optional anyway 15:50:48 +1 15:52:20 15:55:06 it is currently possible to place the haproxy on separate nodes or do i need to turn off haproxy completely and do the lbs on my own? 15:56:00 ok, let's finish the meeting 15:56:09 thanks guys for attending 15:56:25 thanks mnasiadka for chairing 15:56:30 #endmeeting