15:00:44 <mgoddard> #startmeeting kolla
15:00:45 <openstack> Meeting started Wed Oct  7 15:00:44 2020 UTC and is due to finish in 60 minutes.  The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:46 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:49 <openstack> The meeting name has been set to 'kolla'
15:00:54 <mgoddard> #topic rollcall
15:00:57 <mgoddard> \o
15:01:44 <priteau> o!
15:01:47 <headphoneJames> o/
15:03:43 <mgoddard> #topic agenda
15:03:51 <mgoddard> * Roll-call
15:03:53 <mgoddard> * Announcements
15:03:55 <mgoddard> ** Kolla now in feature freeze
15:03:57 <mgoddard> ** Submit Virtual PTG topic proposals: https://etherpad.opendev.org/p/kolla-wallaby-ptg
15:03:59 <mgoddard> * Review action items from the last meeting
15:04:01 <mgoddard> * CI status
15:04:03 <mgoddard> * Victoria release planning
15:04:05 <mgoddard> * Wallaby PTG planning
15:04:07 <mgoddard> #topic announcements
15:04:09 <mgoddard> #info Kolla now in feature freeze
15:04:44 <yoctozepto> o/
15:04:46 <mgoddard> Please do not merge feature patches until the freeze has been lifted, or we have granted a Feature Freeze Exception (FFE)
15:05:04 <mgoddard> #info Submit Virtual PTG topic proposals
15:05:10 <mgoddard> #link https://etherpad.opendev.org/p/kolla-wallaby-ptg
15:05:19 <mgoddard> Any other announcements?
15:05:44 <yoctozepto> TC elections
15:05:51 <yoctozepto> (shameless plug)
15:06:03 * yoctozepto <- vote for this folk
15:06:12 <yoctozepto> :-)
15:06:19 <mgoddard> +1
15:06:35 <mgoddard> #topic Review action items from the last meeting
15:06:44 <mgoddard> mgoddard to email openstack-discuss about debian CI issues
15:06:46 <mgoddard> mgoddard to switch images to victoria stable branches
15:06:48 <mgoddard> yoctozepto to help headphoneJames with neutron backend TLS
15:07:02 * yoctozepto kinda did
15:07:08 <mgoddard> I did not need to email about debian CI, because it was fixed. AARCH64 still failing
15:07:17 <yoctozepto> hrw knows
15:07:37 <mnasiadka> o/
15:07:38 <mnasiadka> late again
15:07:49 <mgoddard> I proposed https://review.opendev.org/755339 for victoria branch switch. Failed until recently because requirements hadn't branched
15:07:49 <patchbot> patch 755339 - kolla - Switch to Victoria stable branches - 4 patch sets
15:07:57 <mgoddard> this one will hopefully pas
15:08:06 <mgoddard> yoctozepto did his
15:08:12 <mgoddard> #topic CI status
15:09:05 <hrw> aarch64 issue was reported to Debian. if nothing change then I will build fixed package and put into Linaro repo
15:10:15 <mgoddard> thanks hrw
15:10:25 <mgoddard> I tried pushing on the bifrost stein fix
15:10:51 <mgoddard> will see where it goes
15:12:09 <mgoddard> I think otherwise we are GREEN
15:12:13 <mgoddard> anyone disagree?
15:12:36 <priteau> Kayobe CI is green but in practice there are some issues
15:12:47 <priteau> on master and ussuri, caused by changes in bifrost
15:12:50 <priteau> working on it
15:13:30 <mgoddard> thanks priteau
15:14:11 <mgoddard> #topic Victoria release planning
15:14:23 <mgoddard> Feature freeze is here
15:14:40 <mgoddard> Whiteboard says: Kolla RC1: Oct 19 - Oct 23
15:14:55 <mgoddard> so we stay frozen until at least then
15:15:23 <yoctozepto> are we merging neutron tls and octavia stacks?
15:15:24 <mgoddard> giving us 2 weeks to stabilise, and focus on testing and bug fixes
15:15:30 <yoctozepto> unclear on what ffe is now
15:15:36 <mgoddard> well
15:15:42 <mgoddard> that is the next question
15:15:46 <mgoddard> are there any FFEs?
15:16:03 <mgoddard> 1. octavia
15:16:05 <headphoneJames> I would like to try and get neutron tls in :)
15:16:08 <mgoddard> if someone else will reciew
15:16:12 <mgoddard> *review
15:16:32 <yoctozepto> mgoddard: did you get it to pass?
15:16:34 <yoctozepto> (not checked)
15:16:45 <mgoddard> no
15:16:55 <mgoddard> chicken and egg problem
15:17:13 <mnasiadka> can we mark what needs to be reviewed ASAP with priority +1/+2?
15:17:15 <mgoddard> br-ex does not exist at the time of fact gathering
15:17:32 <mgoddard> mnasiadka: we can, and should
15:17:37 <mgoddard> we just need to agree on it
15:17:50 <yoctozepto> and mark other features with RP-1
15:18:12 <mnasiadka> I spent three days this week on dancing with mod_auth_mellon and keystone federation, but I should have more time from tomorrow to look at review queue
15:18:13 <mgoddard> it would be nice to see https://review.opendev.org/#/c/754285/ passing before merging the rest of octavia
15:18:14 <patchbot> patch 754285 - kolla-ansible - CI: octavia: create and test a load balancer - 20 patch sets
15:18:40 <yoctozepto> mnasiadka: mellon dance you say
15:18:49 <mnasiadka> yoctozepto: don't even ask
15:19:04 * yoctozepto not asking
15:19:13 * yoctozepto still curious
15:20:21 <mnasiadka> yoctozepto: had to federate with multiple sources using mod_auth_mellon, started drafting a change adding that feature to k-a, so we'll probably be able to merge it in W :)
15:20:54 <mgoddard> hopefully it is compatible with https://review.opendev.org/695432
15:20:54 <patchbot> patch 695432 - kolla-ansible - Add support to OpenID Connect Authentication flow - 37 patch sets
15:21:19 <mgoddard> or at least not conflicting
15:22:32 <mgoddard> #action mgoddard get octavia CI job passing in order to merge patches
15:22:58 <mgoddard> tempted to insert the tenant networking patch in the chain to verify the certs patch works
15:23:10 <mgoddard> since we know it can pass with that
15:23:12 <mgoddard> anyway
15:23:15 <mgoddard> other FFEs
15:23:15 <yoctozepto> I'm not opposing
15:23:22 <mgoddard> neutron backend TLS
15:23:37 <mgoddard> happy to review this, if someone else will also
15:23:44 <yoctozepto> I will
15:23:46 <mgoddard> ok
15:23:55 <mnasiadka> mgoddard: yeah, that one looks good, needs some updates, and I have no clue why shibboleth usage is in there, since it's not in the image
15:23:56 <mgoddard> any other FFEs?
15:24:27 <yoctozepto> lemme see
15:25:20 <mgoddard> Not a feature, but this one cleans up some ironic config: https://review.opendev.org/#/c/754997/
15:25:21 <patchbot> patch 754997 - kolla-ansible - Update ironic iPXE configuration - 2 patch sets
15:25:30 <yoctozepto> I have 3 waiting for mnasiadka:
15:25:30 <yoctozepto> https://review.opendev.org/752907
15:25:31 <patchbot> patch 752907 - kolla-ansible - Add support for changing sysctl.conf path - 1 patch set
15:25:42 <yoctozepto> https://review.opendev.org/752917
15:25:42 <patchbot> patch 752917 - kolla-ansible - Allow to skip and unset sysctl vars - 1 patch set
15:25:48 <yoctozepto> https://review.opendev.org/752917
15:25:48 <patchbot> patch 752917 - kolla-ansible - Allow to skip and unset sysctl vars - 1 patch set
15:25:56 <yoctozepto> oopsie
15:25:56 <yoctozepto> https://review.opendev.org/753380
15:25:57 <patchbot> patch 753380 - kolla-ansible - Add support for with_frontend and with_backend - 2 patch sets
15:26:31 <yoctozepto> actually 4
15:26:31 <yoctozepto> https://review.opendev.org/723342
15:26:32 <patchbot> patch 723342 - kolla-ansible - Coordinate haproxy and keepalived restarts - 12 patch sets
15:26:45 <yoctozepto> and one release critical also for mnasiadka: https://review.opendev.org/755328
15:26:45 <patchbot> patch 755328 - kolla-ansible - Control Neutron migrations - 4 patch sets
15:26:54 <yoctozepto> (or any other hero core)
15:27:33 <yoctozepto> mgoddard: +2 on ironic
15:27:38 <mgoddard> thx
15:29:26 <mgoddard> added some RP+1s
15:30:37 <mnasiadka> yoctozepto: done
15:31:21 <yoctozepto> mnasiadka: thanks, missed: https://review.opendev.org/752917
15:31:21 <patchbot> patch 752917 - kolla-ansible - Allow to skip and unset sysctl vars - 1 patch set
15:32:15 <mnasiadka> yoctozepto: done :)
15:32:21 <yoctozepto> mnasiadka: thx
15:32:23 <mgoddard> Not seeing any FFE candidates for kayobe
15:32:24 <yoctozepto> mnasiadka: any from you?
15:32:57 <mnasiadka> yoctozepto: not today, but I'll look into my backlog and come back tomorrow :D
15:33:05 <yoctozepto> mnasiadka: ok
15:34:08 <mgoddard> I guess we should look at bugs too
15:34:43 <mgoddard> https://bugs.launchpad.net/kolla
15:34:52 <mgoddard> are there any that we need in the release?
15:36:31 <yoctozepto> hmm, good question Mark
15:37:12 <yoctozepto> the 2nd one on that page is easy but burdensome
15:37:37 <mnasiadka> drop root on wsgi?
15:37:44 <yoctozepto> nope
15:38:05 <mnasiadka> so which is the 2nd one? :)
15:38:15 <yoctozepto> https://bugs.launchpad.net/kolla/+bug/1874298
15:38:17 <openstack> yoctozepto: Error: malone bug 1874298 not found
15:38:27 <yoctozepto> it's a sec bug
15:38:32 <mnasiadka> it's not found
15:38:36 <yoctozepto> but minor, we can probably open it up
15:38:42 <yoctozepto> mgoddard: wdyt?
15:39:07 <yoctozepto> or I can grant mnasiadka access to sec bugs
15:39:16 <yoctozepto> please ensure you are logged in
15:39:21 <yoctozepto> and let me check how that was done
15:39:24 <mnasiadka> ah, great, you need special permissions to see a sec bug?
15:39:25 <mnasiadka> lol
15:40:06 <yoctozepto> given now
15:40:07 <mgoddard> I suppose we could give mnasiadka access :)
15:40:11 <yoctozepto> refresh the listing
15:40:15 <yoctozepto> mgoddard: I did
15:40:43 <mnasiadka> yeah
15:40:47 <yoctozepto> if you think it worthy of pushing, I can
15:40:50 <mnasiadka> yoctozepto: well, you raised it, you fix it :)
15:41:05 <yoctozepto> sure, the question is whether we want that for this release
15:41:20 <yoctozepto> or any better candidates I can spend my time on for now
15:42:03 <openstackgerrit> Pierre Riteau proposed openstack/kayobe master: Configure bifrost to use firewalld trusted zone  https://review.opendev.org/756541
15:42:10 <mgoddard> it could be a good one to fix
15:43:52 <mgoddard> what about kolla-ansible?
15:44:00 <mgoddard> https://bugs.launchpad.net/kolla-ansible
15:44:09 <mgoddard> (or https://bugs.launchpad.net/kolla-ansible/+bugs?orderby=-id&start=0)
15:45:05 <mgoddard> more fernet issues: https://bugs.launchpad.net/kolla-ansible/+bug/1898765
15:45:07 <openstack> Launchpad bug 1898765 in kolla-ansible "keystone-fernet cron job not triggering, results in keys not rotating" [Undecided,New]
15:46:18 <mnasiadka> mgoddard: yeah, I added more logging in one deployment and testing
15:46:46 <yoctozepto> fernet sad
15:48:34 <mnasiadka> cron broken :)
15:48:53 <yoctozepto> cron never breaks! :D
15:48:58 <mgoddard> any others?
15:50:47 <yoctozepto> headphoneJames: how is neutron tls going with the new approach?
15:50:47 <mgoddard> https://bugs.launchpad.net/kolla-ansible/+bug/1894056
15:50:48 <openstack> Launchpad bug 1894056 in kolla-ansible "DB migration broken" [Critical,Triaged]
15:51:55 <yoctozepto> mgoddard: invalid, undecided
15:51:57 <yoctozepto> just switched
15:52:06 <yoctozepto> neutron problem, not affecting us atm
15:53:07 <mgoddard> Could we fix https://bugs.launchpad.net/kolla-ansible/+bug/1891469 by using throttle=1?
15:53:08 <openstack> Launchpad bug 1891469 in kolla-ansible "Neutron: Stopping neutron L3 agent containers breaks data plane" [High,Triaged]
15:53:32 <mgoddard> now that we have ansible 2.9 (throttle is a good reason to use 2.9)
15:54:17 <mgoddard> I guess it's unlikely to completely fix it, but rolling restarts could reduce downtime
15:55:48 <yoctozepto> mgoddard: it won't help much
15:55:59 <yoctozepto> the torn down containers are the reason
15:57:46 <mgoddard> ok
15:58:00 <mgoddard> is https://bugs.launchpad.net/kolla-ansible/+bug/1891145 a dup of https://bugs.launchpad.net/kolla-ansible/+bug/1891704 ?
15:58:03 <openstack> Launchpad bug 1891145 in kolla-ansible victoria "post-deploy different behaviour with new ansible versions" [Critical,Triaged] - Assigned to Radosław Piliszek (yoctozepto)
15:58:04 <openstack> Launchpad bug 1891704 in kolla-ansible ussuri "admin-openrc.sh is world readable" [High,Fix committed] - Assigned to Radosław Piliszek (yoctozepto)
15:59:39 <yoctozepto> mgoddard: kinda; actually the other way around but fine
15:59:43 <yoctozepto> :D
15:59:52 <yoctozepto> better just tick it as dup
16:00:39 <mgoddard> done
16:01:11 <mgoddard> and we're done
16:01:16 <mgoddard> #topic Open discussion
16:01:20 <mgoddard> One last thing
16:01:26 <mgoddard> yoctozepto: this ansible meeting
16:01:34 <mgoddard> shall we send a representative?
16:01:52 <mgoddard> I didn't want to cancel the klub at late notice
16:02:25 <mgoddard> (representative being yoctozepto :D)
16:02:45 <mgoddard> or me if yoctozepto handles klub
16:05:29 <mgoddard> well let's end it there
16:05:33 <mgoddard> Thanks
16:05:35 <mgoddard> #endmeeting