15:01:47 <mgoddard> #startmeeting kolla 15:01:47 <opendevmeet> Meeting started Wed Dec 15 15:01:47 2021 UTC and is due to finish in 60 minutes. The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:47 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:01:47 <opendevmeet> The meeting name has been set to 'kolla' 15:02:16 <mgoddard> #topic rollcall 15:02:27 <parallax`> o/ 15:03:06 <hrw> [°][o][°] 15:03:31 <frickler> \o 15:04:33 <bbezak> \o 15:05:30 <mgoddard> \o 15:05:39 <mgoddard> # topic agenda 15:05:42 <mgoddard> #topic agenda 15:06:00 <mgoddard> * Roll-call 15:06:03 <mgoddard> * Agenda 15:06:05 <mgoddard> * Announcements 15:06:07 <mgoddard> * Review action items from the last meeting 15:06:09 <mgoddard> * CI status 15:06:11 <mgoddard> * Release tasks 15:06:13 <mgoddard> * Current cycle planning 15:06:21 <mgoddard> * (mnasiadka) - log4j vulnerability and Kolla - elasticsearch/logstash? 15:06:26 <mgoddard> * Open discussion 15:06:32 <mgoddard> #topic announcements 15:06:41 <mgoddard> mnasiadka asked me to chair today as he is unavailable 15:08:07 <mgoddard> anyone else? 15:08:49 <mgoddard> #topic Review action items from the last meeting 15:09:07 <mgoddard> mnasiadka to triage security bugs and update them with resolution plan (if needed) 15:09:10 <mgoddard> mnasiadka post a patch for docs - standard topics that should be discussed over PTG and then revisited in mid-cycle 15:09:12 <mgoddard> anybody not forget to go through backports for stable branches (L248 on Whiteboard) and do stable releases afterwards. 15:09:14 <mgoddard> parallax look into Grafana Kolla build failures on Ussuri/CentOS 15:09:16 <mgoddard> yoctozepto to send mail to openstack ML about dropping vmtp 15:09:18 <mgoddard> yoctozepto to remove CentOS 8 based CI jobs and manage communication (ML and renos) 15:09:19 <mgoddard> this list seems to just be growing :) 15:09:26 <mgoddard> anyone complete any? 15:10:42 <mgoddard> #action mnasiadka to triage security bugs and update them with resolution plan (if needed) 15:10:50 <mgoddard> #action mnasiadka post a patch for docs - standard topics that should be discussed over PTG and then revisited in mid-cycle 15:11:03 <mgoddard> #action anybody not forget to go through backports for stable branches (L248 on Whiteboard) and do stable releases afterwards 15:11:06 <hrw> vmtp mail happened. no response 15:11:21 <mgoddard> #action parallax look into Grafana Kolla build failures on Ussuri/CentOS 15:11:28 <mgoddard> #action yoctozepto to remove CentOS 8 based CI jobs and manage communication (ML and renos) 15:11:31 <mgoddard> ok 15:11:38 <mgoddard> #topic CI status 15:12:03 <mgoddard> whiteboard mostly green 15:12:33 <hrw> kolla/ussuri is broken due to monasca-grafana iirc 15:12:54 <hrw> there was some patch to fix it, failed on something else, not checked why 15:13:25 <priteau> It was merged? https://review.opendev.org/c/openstack/kolla/+/821533 15:13:58 <hrw> cool ;) 15:14:09 <mgoddard> #topic Release tasks 15:14:38 <mgoddard> we are in R-15 15:15:08 <mgoddard> Nothing to do for now 15:15:15 <mgoddard> #topic Current cycle planning 15:15:26 <mgoddard> I think we're a bit low on numbers for planning today 15:15:38 <mgoddard> #topic (mnasiadka) - log4j vulnerability and Kolla - elasticsearch/logstash? 15:16:14 <mgoddard> priteau: I think you were involved in the internal discussion on this earlier 15:16:18 <priteau> Yes, I can share 15:16:21 <mgoddard> thanks 15:16:28 <priteau> I only looked at victoria so far 15:16:53 <priteau> The latest images on quay.io include the new release from elastic 15:17:05 <priteau> elasticsearch-oss 6.8.21 15:17:49 <priteau> Also, even an old-ish image (built last summer) was not vulnerable because it was 6.8.9+ 15:18:18 <priteau> And JDK11 15:18:36 <priteau> I am checking wallaby now, which is Elasticsearch 7 15:18:48 <priteau> I've not looked at Logstash 15:19:59 <priteau> wallaby: elasticsearch-oss-7.10.2-1.x86_64 15:20:15 <priteau> Note I am only looking at centos-binary 15:20:56 <priteau> So Wallaby (and above I suppose), theoretically not vulnerable because 7.8+, thanks to Java Security Manager 15:21:17 <priteau> However we don't get the proper fix from Elastic, which would be 7.16.1 15:21:52 <priteau> Unless this was also backported to 7.10 15:22:24 <priteau> Nope, looks like 7.10.2 is an old release 15:23:06 <priteau> That's all I have to share so far 15:23:46 <mgoddard> sounds like we are fairly safe for elastic in the maintained releases 15:24:07 <mgoddard> we could apply the java option mitigation 15:24:54 <priteau> That would make it safer indeed, especially for wallaby/xena 15:25:11 <priteau> Since they don't get the new package which applies the java option 15:26:22 <mgoddard> anyone want to pick it up? 15:27:03 <parallax_> me 15:27:46 <mgoddard> #action parallax_ apply jvm option mitigation for log4shell in elasticsearch 15:27:47 <mgoddard> thanks 15:27:52 <parallax_> np 15:27:55 <mgoddard> #topic Open discussion 15:28:02 <mgoddard> Does anyone have anything this week? 15:28:43 <bbezak> There is a bug in k-a ovn implementation of system-id registration in ovs - which suppose to be in UUID format - impacting mostly neutron-ovn-metadata-agent - https://bugs.launchpad.net/kolla-ansible/+bug/1952550 15:28:48 <bbezak> related fix - https://review.opendev.org/c/openstack/kolla-ansible/+/818700 15:29:12 <bbezak> Changing back to UUID in running environment is tricky - I'm working on it currently 15:31:10 <bbezak> changing to UUID works fine with that change above - however one needs to do manually cleanup of old chassis and chassis_private in ovn_sb_db, and then restart metadata agent, and cleanup entries in neutron via openstack network agent list 15:31:34 <bbezak> which is something I'm not sure if we can automate in k-a easily 15:32:07 <mgoddard> what happens if you don't do it? 15:32:57 <bbezak> ovn-metadata agent won't work 15:33:26 <bbezak> connectivity works fine as far as I can see 15:34:02 <bbezak> similar to this - https://bugzilla.redhat.com/show_bug.cgi?id=1948472#c12 15:34:38 <mgoddard> ok, seems like the discussion is ongoing in the bug 15:34:42 <mgoddard> and review 15:34:49 <bbezak> yeap 15:34:59 <bbezak> just to let you know 15:36:14 <bbezak> this also has bigger impact when used with ovs 2.16 - which is doing automatic leadership transfer when doing snapshots of ovsdb - then ovn-metadata-agent cannot reconnect when non UUID system-id is used 15:40:17 <mgoddard> sounds nasty 15:40:21 <mgoddard> any other topics? 15:43:02 <mgoddard> thanks all 15:43:04 <mgoddard> #endmeeting