15:00:15 <mgoddard> #startmeeting kolla
15:00:15 <opendevmeet> Meeting started Wed Jun 16 15:00:15 2021 UTC and is due to finish in 60 minutes.  The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:15 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:15 <opendevmeet> The meeting name has been set to 'kolla'
15:00:30 <mgoddard> #topic rollcall
15:01:03 <mgoddard> 8< 8< 8<
15:03:22 <parallax> o/
15:03:29 <priteau> \o
15:03:39 <yoctozepto> \o/
15:03:43 <hrw> ]o[
15:04:23 <mgoddard> #topic agenda
15:04:28 <yoctozepto> (mgoddard: yup, but it could be threaded)
15:04:33 <mgoddard> * Roll-call
15:04:35 <mgoddard> * Agenda
15:04:37 <mgoddard> * Announcements
15:04:39 <mgoddard> * Review action items from the last meeting
15:04:41 <mgoddard> * CI status
15:04:43 <mgoddard> * Wallaby release planning
15:04:45 <mgoddard> ** libvirt/OVMF bug
15:04:47 <mgoddard> * Xena cycle planning
15:04:49 <mgoddard> ** master branch life cycle https://review.opendev.org/c/openstack/kolla/+/795320
15:04:51 <mgoddard> ** Testing log coverage (and more?) https://review.opendev.org/c/openstack/kolla-ansible/+/795512
15:04:53 <mgoddard> * Open discussion
15:05:39 <mgoddard> #topic announcements
15:05:48 <mgoddard> None from me
15:06:26 <yoctozepto> none
15:06:26 <mgoddard> #topic
15:06:29 <hrw> nope
15:06:29 <mgoddard> #topic  Review action items from the last meeting
15:06:35 <mgoddard> mgoddard email openstack-discuss about quay.io credentials
15:06:37 <mgoddard> mnasiadka perform R-17 release tasks
15:06:45 <mgoddard> mgoddard: nope
15:06:49 <mgoddard> mnasiadka: started
15:07:07 <mgoddard> I'm dropping that one. It's on a personal TODO list, just low priority
15:07:12 <yoctozepto> ok
15:07:17 <mgoddard> #topic CI status
15:07:46 <yoctozepto> centos8 broke masakari-monitors in train
15:07:50 <yoctozepto> otherwise pretty green
15:07:52 <mgoddard> kolla: [train] libvirt-python dep broke masakari-monitors build on c8
15:07:58 <yoctozepto> with the random failure on neutron upgrades
15:08:26 <yoctozepto> re masakari-monitors and c8
15:08:29 <yoctozepto> we have a bug report
15:08:36 <yoctozepto> that it fails up to victoria
15:08:41 <yoctozepto> fails to build*
15:08:48 <yoctozepto> but CI confirms only for train
15:08:59 <yoctozepto> I did not have time to dive in
15:09:00 <yoctozepto> but priteau did
15:09:14 <priteau> I said I would try a build in victoria but lost access to my env so haven't completed it
15:09:22 <yoctozepto> https://review.opendev.org/c/openstack/kolla/+/796387
15:09:27 <yoctozepto> no problem
15:09:34 <yoctozepto> I wanted to mention the workaround ^^
15:10:26 <mgoddard> doesn't seem to work
15:11:07 <yoctozepto> ?
15:11:08 <priteau> Are you saying this because of Zuul -1? The centos jobs passed
15:11:09 <yoctozepto> it built
15:11:14 <mgoddard> not debian
15:11:33 <yoctozepto> debian's patch is broken on centos8 not passing
15:11:43 <yoctozepto> https://review.opendev.org/c/openstack/kolla/+/795874
15:11:50 <yoctozepto> and so is final bump
15:11:55 <yoctozepto> and so is our em release
15:12:21 <mgoddard> so should we combine those patches and merge
15:12:23 <mgoddard> ?
15:12:56 <yoctozepto> debian is non-voting
15:13:08 <mgoddard> the workaround doesn't seem any worse than the existing code
15:13:09 <yoctozepto> I guess we could go with the workaround
15:13:17 <yoctozepto> and bury train
15:13:25 <yoctozepto> then worry about ussuri+
15:13:54 <mgoddard> we might get pressure to tag our EM soon
15:13:59 <yoctozepto> indeed
15:14:01 <mgoddard> don't want to get caught with our pants down
15:14:03 <yoctozepto> I'm rechecking priteau's workaround
15:14:16 <mgoddard> ok
15:14:21 <yoctozepto> priteau to lift w-1
15:14:30 <yoctozepto> and /me rebasing other patches
15:14:33 <mgoddard> k
15:14:58 <priteau> I'll recheck too because of the POST_FAILURE
15:15:03 <opendevreview> Radosław Piliszek proposed openstack/kolla stable/train: Enable Debian updates repo  https://review.opendev.org/c/openstack/kolla/+/795874
15:15:05 <priteau> You already did
15:15:12 <opendevreview> Radosław Piliszek proposed openstack/kolla stable/train: Bump final OpenStack versions for Train  https://review.opendev.org/c/openstack/kolla/+/795258
15:15:17 <yoctozepto> fingers crossed
15:15:22 <yoctozepto> priteau: yeah
15:15:28 <yoctozepto> priteau: just need w0 from you
15:15:31 <priteau> done
15:15:36 <mgoddard> ok
15:15:37 <yoctozepto> thx
15:15:51 <yoctozepto> mgoddard: cr+2w+1?
15:15:54 <mgoddard> the train is ready to go
15:15:56 <yoctozepto> all the patches
15:16:08 <mgoddard> I'll wait for zuul
15:16:21 <mgoddard> #topic Wallaby release planning / libvirt/OVMF bug
15:16:37 <yoctozepto> mgoddard ok
15:16:40 <mgoddard> How are we looking with this one?
15:16:44 <yoctozepto> priteau also knows
15:17:29 <priteau> So, we appear to have a fix because c8 AV imported libvirt 7.0, although I would think it should *not* work
15:17:45 <priteau> But anyway, c8s is due to fix their AV repo package today
15:17:53 <priteau> Which will give us libvirt 7.4
15:18:12 <priteau> So I am refreshing http://mirror.centos.org/centos/8-stream/extras/x86_64/os/Packages/ multiple times, but still no updated package
15:18:53 <priteau> hughesjr said by wednesday cob us time
15:18:56 <mgoddard> So we continue to wait
15:19:08 <priteau> unfortunately yes
15:19:14 <mgoddard> hopefully tomorrow
15:19:44 <mgoddard> #topic Xena cycle planning / master branch life cycle
15:19:44 <priteau> If someone wants to proactively test libvirt 7.4, we could just update the repo file after the package has been installed
15:19:57 <mgoddard> #link https://review.opendev.org/c/openstack/kolla/+/795320
15:20:15 <mgoddard> Not much to say on this one
15:20:21 <mgoddard> Except please review
15:20:52 <mgoddard> #topic Testing log coverage (and more?)
15:20:58 <mgoddard> #link https://review.opendev.org/c/openstack/kolla-ansible/+/795512
15:21:55 <mgoddard> This is a patch that I've been iterating on recently, that could improve our log coverage
15:22:24 <mgoddard> We currently have no real test coverage of our logging
15:22:28 <yoctozepto> yeah, we need something in that direction
15:22:29 <mgoddard> This should help
15:23:24 <mgoddard> Basic idea is, we have an output plugin in fluentd image that writes a tab separated values file containing various labels for each log message
15:23:57 <mgoddard> Then in kolla-ansible we have a new test that ingests that TSV file and verifies that everything is as expected
15:24:35 <mgoddard> The test is implemented using py.test / testinfra
15:24:41 <mgoddard> #link https://review.opendev.org/c/openstack/kolla-ansible/+/795512/13/tests/test-fluentd.py
15:25:03 <yoctozepto> this obviously tests only whether we log at least one message
15:25:06 <mgoddard> I think we could really tighten up our testing by introducing this
15:25:18 <mgoddard> yoctozepto: correct, currently
15:25:22 <yoctozepto> yeah
15:25:36 <yoctozepto> I think someone worked on improving the fluentd integration
15:25:38 <yoctozepto> re parsing
15:25:52 <yoctozepto> so that we don't lose messages or get weird entries
15:25:59 <mgoddard> right
15:26:24 <mgoddard> we could do more in the output plugin to validate each fluentd event, currently it is quite minimal
15:26:37 <mgoddard> e.g. does it have at least Hostname, programname, log_level set
15:27:12 <mgoddard> I wanted to get the moving parts in place then it can be iterated on
15:27:37 <yoctozepto> great, I am all in
15:27:40 <yoctozepto> will review for sure
15:27:42 <mgoddard> excellent
15:28:19 <mgoddard> I think the existing tests/check-* scripts could be added to this py.test
15:28:32 <mgoddard> then we get results in one place
15:28:46 <mgoddard> and zuul and expose them as artifacts
15:29:19 <mgoddard> we're using this approach in kayobe already: https://opendev.org/openstack/kayobe/src/branch/master/playbooks/kayobe-overcloud-host-configure-base/tests/test_overcloud_host_configure.py
15:29:50 <yoctozepto> ++
15:31:00 <mgoddard> Any other comments/suggestions?
15:32:09 <parallax> Looking good :)
15:33:15 <mgoddard> cool
15:33:17 <mgoddard> #topic Open discussion
15:33:28 <mgoddard> Does anyone have anything else to discuss today?
15:34:55 <headphoneJames> I'm reviving https://review.opendev.org/c/openstack/kolla-ansible/+/692179
15:34:55 <hrw> nope
15:35:38 <mgoddard> that's good headphoneJames
15:35:52 <mgoddard> are you still planning to email openstack-discuss?
15:36:28 <headphoneJames> Yes. Would like to get this one at least working first
15:36:59 <mgoddard> ok
15:39:51 <opendevreview> Mark Goddard proposed openstack/kolla master: docs: Update release management for Xena process  https://review.opendev.org/c/openstack/kolla/+/795320
15:41:33 <mgoddard> Let's wrap up there then
15:41:36 <mgoddard> Thanks al
15:41:38 <opendevreview> Mark Goddard proposed openstack/kolla master: docs: Update release management for Xena process  https://review.opendev.org/c/openstack/kolla/+/795320
15:41:40 <mgoddard> *all
15:41:43 <mgoddard> #endmeeting