15:01:47 <mgoddard> #startmeeting kolla
15:01:47 <opendevmeet> Meeting started Wed Dec 15 15:01:47 2021 UTC and is due to finish in 60 minutes.  The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:47 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:47 <opendevmeet> The meeting name has been set to 'kolla'
15:02:16 <mgoddard> #topic rollcall
15:02:27 <parallax`> o/
15:03:06 <hrw> [°][o][°]
15:03:31 <frickler> \o
15:04:33 <bbezak> \o
15:05:30 <mgoddard> \o
15:05:39 <mgoddard> # topic agenda
15:05:42 <mgoddard> #topic agenda
15:06:00 <mgoddard> * Roll-call
15:06:03 <mgoddard> * Agenda
15:06:05 <mgoddard> * Announcements
15:06:07 <mgoddard> * Review action items from the last meeting
15:06:09 <mgoddard> * CI status
15:06:11 <mgoddard> * Release tasks
15:06:13 <mgoddard> * Current cycle planning
15:06:21 <mgoddard> * (mnasiadka) - log4j vulnerability and Kolla - elasticsearch/logstash?
15:06:26 <mgoddard> * Open discussion
15:06:32 <mgoddard> #topic announcements
15:06:41 <mgoddard> mnasiadka asked me to chair today as he is unavailable
15:08:07 <mgoddard> anyone else?
15:08:49 <mgoddard> #topic Review action items from the last meeting
15:09:07 <mgoddard> mnasiadka to triage security bugs and update them with resolution plan (if needed)
15:09:10 <mgoddard> mnasiadka post a patch for docs - standard topics that should be discussed over PTG and then revisited in mid-cycle
15:09:12 <mgoddard> anybody not forget to go through backports for stable branches (L248 on Whiteboard) and do stable releases afterwards.
15:09:14 <mgoddard> parallax look into Grafana Kolla build failures on Ussuri/CentOS
15:09:16 <mgoddard> yoctozepto to send mail to openstack ML about dropping vmtp
15:09:18 <mgoddard> yoctozepto to remove CentOS 8 based CI jobs and manage communication (ML and renos)
15:09:19 <mgoddard> this list seems to just be growing :)
15:09:26 <mgoddard> anyone complete any?
15:10:42 <mgoddard> #action mnasiadka to triage security bugs and update them with resolution plan (if needed)
15:10:50 <mgoddard> #action mnasiadka post a patch for docs - standard topics that should be discussed over PTG and then revisited in mid-cycle
15:11:03 <mgoddard> #action anybody not forget to go through backports for stable branches (L248 on Whiteboard) and do stable releases afterwards
15:11:06 <hrw> vmtp mail happened. no response
15:11:21 <mgoddard> #action parallax look into Grafana Kolla build failures on Ussuri/CentOS
15:11:28 <mgoddard> #action yoctozepto to remove CentOS 8 based CI jobs and manage communication (ML and renos)
15:11:31 <mgoddard> ok
15:11:38 <mgoddard> #topic CI status
15:12:03 <mgoddard> whiteboard mostly green
15:12:33 <hrw> kolla/ussuri is broken due to monasca-grafana iirc
15:12:54 <hrw> there was some patch to fix it, failed on something else, not checked why
15:13:25 <priteau> It was merged? https://review.opendev.org/c/openstack/kolla/+/821533
15:13:58 <hrw> cool ;)
15:14:09 <mgoddard> #topic Release tasks
15:14:38 <mgoddard> we are in R-15
15:15:08 <mgoddard> Nothing to do for now
15:15:15 <mgoddard> #topic  Current cycle planning
15:15:26 <mgoddard> I think we're a bit low on numbers for planning today
15:15:38 <mgoddard> #topic (mnasiadka) - log4j vulnerability and Kolla - elasticsearch/logstash?
15:16:14 <mgoddard> priteau: I think you were involved in the internal discussion on this earlier
15:16:18 <priteau> Yes, I can share
15:16:21 <mgoddard> thanks
15:16:28 <priteau> I only looked at victoria so far
15:16:53 <priteau> The latest images on quay.io include the new release from elastic
15:17:05 <priteau> elasticsearch-oss 6.8.21
15:17:49 <priteau> Also, even an old-ish image (built last summer) was not vulnerable because it was 6.8.9+
15:18:18 <priteau> And JDK11
15:18:36 <priteau> I am checking wallaby now, which is Elasticsearch 7
15:18:48 <priteau> I've not looked at Logstash
15:19:59 <priteau> wallaby: elasticsearch-oss-7.10.2-1.x86_64
15:20:15 <priteau> Note I am only looking at centos-binary
15:20:56 <priteau> So Wallaby (and above I suppose), theoretically not vulnerable because 7.8+, thanks to Java Security Manager
15:21:17 <priteau> However we don't get the proper fix from Elastic, which would be 7.16.1
15:21:52 <priteau> Unless this was also backported to 7.10
15:22:24 <priteau> Nope, looks like 7.10.2 is an old release
15:23:06 <priteau> That's all I have to share so far
15:23:46 <mgoddard> sounds like we are fairly safe for elastic in the maintained releases
15:24:07 <mgoddard> we could apply the java option mitigation
15:24:54 <priteau> That would make it safer indeed, especially for wallaby/xena
15:25:11 <priteau> Since they don't get the new package which applies the java option
15:26:22 <mgoddard> anyone want to pick it up?
15:27:03 <parallax_> me
15:27:46 <mgoddard> #action parallax_ apply jvm option mitigation for log4shell in elasticsearch
15:27:47 <mgoddard> thanks
15:27:52 <parallax_> np
15:27:55 <mgoddard> #topic Open discussion
15:28:02 <mgoddard> Does anyone have anything this week?
15:28:43 <bbezak> There is a bug in k-a ovn implementation of system-id registration in ovs - which suppose to be in UUID format - impacting mostly neutron-ovn-metadata-agent - https://bugs.launchpad.net/kolla-ansible/+bug/1952550
15:28:48 <bbezak> related fix - https://review.opendev.org/c/openstack/kolla-ansible/+/818700
15:29:12 <bbezak> Changing back to UUID in running environment is tricky - I'm working on it currently
15:31:10 <bbezak> changing to UUID works fine with that change above - however one needs to do manually cleanup of old chassis and chassis_private in ovn_sb_db, and then restart metadata agent, and cleanup entries in neutron via openstack network agent list
15:31:34 <bbezak> which is something I'm not sure if we can automate in k-a easily
15:32:07 <mgoddard> what happens if you don't do it?
15:32:57 <bbezak> ovn-metadata agent won't work
15:33:26 <bbezak> connectivity works fine as far as I can see
15:34:02 <bbezak> similar to this - https://bugzilla.redhat.com/show_bug.cgi?id=1948472#c12
15:34:38 <mgoddard> ok, seems like the discussion is ongoing in the bug
15:34:42 <mgoddard> and review
15:34:49 <bbezak> yeap
15:34:59 <bbezak> just to let you know
15:36:14 <bbezak> this also has bigger impact when used with ovs 2.16 - which is doing automatic leadership transfer when doing snapshots of ovsdb - then ovn-metadata-agent cannot reconnect when non UUID system-id is used
15:40:17 <mgoddard> sounds nasty
15:40:21 <mgoddard> any other topics?
15:43:02 <mgoddard> thanks all
15:43:04 <mgoddard> #endmeeting