15:02:04 <mgoddard> #startmeeting kolla
15:02:14 <openstack> Meeting started Wed Sep 30 15:02:04 2020 UTC and is due to finish in 60 minutes.  The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:02:15 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:02:18 <openstack> The meeting name has been set to 'kolla'
15:02:19 <openstack> mgoddard: Error: A meeting name is required, e.g., '#startmeeting Marketing Committee'
15:02:24 <mgoddard> #topic rollcall
15:02:28 <mgoddard> \o
15:02:32 <priteau> o/
15:02:37 <jovial[m]> 1o
15:02:38 <osmanlicilegi> o/
15:02:47 <hrw> ]o[
15:03:25 <mgoddard> ioj
15:03:29 <headphoneJames> o/
15:03:42 <mgoddard> #topic announcements
15:04:05 <mgoddard> #info Kolla feature freeze fast approaching: Sep 28 - Oct 02
15:04:15 <mgoddard> end of this week
15:04:18 <mgoddard> the freeze will begin
15:04:28 <mgoddard> #info Submit Virtual PTG topic proposals
15:04:38 <mgoddard> #link https://etherpad.opendev.org/p/kolla-wallaby-ptg
15:04:53 <mgoddard> #info mgoddard remains PTL for Wallaby
15:05:20 <mgoddard> guess you're stuck with me for another cycle :)
15:05:22 <headphoneJames> hooray!
15:05:26 <mgoddard> Any others?
15:06:13 <mgoddard> #info TC campaining begins this week
15:06:28 <mgoddard> watch out for yoctozepto
15:06:35 <yoctozepto> o/
15:06:48 <yoctozepto> vote yoctozepto
15:06:58 <mgoddard> #topic Review action items from the last meeting
15:08:01 <mgoddard> priteau to bump minimum Ansible to 2.9
15:08:03 <mgoddard> priteau to investigate kayobe kayobe-tox-molecule failures
15:08:05 <mgoddard> mnasiadka to investigate ubuntu binary CI failures
15:08:11 <mgoddard> priteau: check, check
15:08:16 <mgoddard> mnasiadka: hmm
15:08:24 <yoctozepto> he fixed
15:08:27 <mnasiadka> isn't it green now?
15:08:44 <mgoddard> ah yes, the heat deps thing
15:08:46 <yoctozepto> only debian is red -> hrw
15:08:48 <mgoddard> ok
15:08:54 <hrw> qemu?
15:09:00 <mgoddard> good work all
15:09:01 <yoctozepto> last time I checked!
15:09:11 <yoctozepto> best team around :-)
15:09:13 <mgoddard> #topic CI status
15:10:27 <mgoddard> starting with kolla
15:10:43 <mgoddard> bifrost stein & rocky unchanged AFAIK
15:10:52 <mgoddard> master neutron migrations
15:11:13 <mnasiadka> did they finally fix it, or they prefer to ship a broken release? :)
15:11:28 <mgoddard> pass
15:12:22 <yoctozepto> they reverted
15:12:29 <yoctozepto> but I need to fix k-a still
15:12:32 <yoctozepto> on it at the very moment
15:12:48 <mgoddard> ok
15:13:18 <mgoddard> kolla ansible
15:13:53 <mgoddard> the main issue right now is the debian jobs
15:13:57 <mgoddard> has anyone looked at them?
15:14:06 <mgoddard> are the debian maintainers aware?
15:15:01 <hrw> not at all
15:15:13 <hrw> was in completely different projects last two weeks
15:15:31 <hrw> started debian/source/x86-64 build to check status
15:16:13 <mgoddard> ok, maybe we should raise it on the ML?
15:16:17 <mgoddard> anyone want to do that?
15:16:48 <hrw> are those kolla or k-a bugs?
15:17:01 <yoctozepto> debian
15:17:03 <yoctozepto> :-)
15:17:06 <hrw> ha ha ha
15:17:10 <hrw> ha
15:17:13 <hrw> ha ha
15:17:20 <yoctozepto> I mean, it's hard to pinpoint, choose either
15:17:23 <yoctozepto> k-a breaks
15:17:25 <yoctozepto> k does not
15:17:29 <yoctozepto> but images are unusable
15:17:46 <hrw> ok, that's some info
15:17:58 <yoctozepto> see :-)
15:20:01 <mgoddard> I will email the list
15:20:18 <mgoddard> #action mgoddard to email openstack-discuss about debian CI issues
15:20:28 <mgoddard> kayobe
15:20:59 <mgoddard> we are currently blocked by an ironic issue which is migrating nodes to use the direct driver during upgrade
15:21:05 <mgoddard> *deploy driver
15:21:45 <mgoddard> fix approved, hopefully it lands in time for tomorrows images
15:21:47 <mgoddard> https://review.opendev.org/755295
15:21:47 <patchbot> patch 755295 - ironic - Don't migrate away from iscsi if it is the default - 1 patch set
15:21:58 <mgoddard> although kayobe uses binary, so depends on RDO
15:22:18 <mgoddard> maybe we should use a workaround
15:22:48 <yoctozepto> sorry for wrong time but should not we move images to victoria right away (to avoid wallaby in)?
15:22:54 <yoctozepto> now that they have been branched
15:23:53 <mgoddard> yes, we could do that
15:24:31 <yoctozepto> any volunteers?
15:24:44 <yoctozepto> (to swtich kolla sources)
15:25:38 <mgoddard> #action mgoddard to switch images to victoria stable branches
15:25:42 <mgoddard> easy one
15:25:55 <mgoddard> #topic Victoria release planning
15:27:17 <mgoddard> we enter the last days of feature merging
15:27:22 <mgoddard> which will we choose?
15:27:56 <mnasiadka> healthchecks are halfway through :)
15:27:56 <headphoneJames> I could use help with backend TLS for neutron
15:28:32 <yoctozepto> hacluster and others of mine are ready to merge, they don't impact existing features
15:28:44 <yoctozepto> headphoneJames: ++
15:28:56 <hrw> https://review.opendev.org/#/c/706982 will complete infra part of V
15:28:56 <patchbot> patch 706982 - kolla-ansible - Introduce 'kolla_infra_install_type' variable - 14 patch sets
15:29:35 <mgoddard> I had some concerns about backwards compatibility with that one
15:30:10 <hrw> so leave it for W and stop worry then?
15:30:15 <mgoddard> +1
15:30:36 <mgoddard> it will make more sense to users in W
15:31:22 <mgoddard> yoctozepto: can you list your patches in order of preference?
15:32:15 <yoctozepto> mgoddard: surrrreeee
15:32:36 <hrw> postponed all infra to W
15:33:21 <yoctozepto> mgoddard: this stack -> https://review.opendev.org/754568
15:33:22 <patchbot> patch 754568 - kolla-ansible - Add support for ACME http-01 challenge - 1 patch set
15:33:32 <yoctozepto> very simple, huge gains
15:34:42 <yoctozepto> https://review.opendev.org/723342
15:34:42 <patchbot> patch 723342 - kolla-ansible - Coordinate haproxy and keepalived restarts - 11 patch sets
15:34:52 <yoctozepto> because I did what you wanted and it's a matter of merging
15:35:01 <mgoddard> ok
15:35:04 <yoctozepto> https://review.opendev.org/747592
15:35:05 <patchbot> patch 747592 - kolla-ansible - Make keep-alive timeout configurable - 1 patch set
15:35:13 <yoctozepto> because it has CR+1 from mnasiadka
15:35:50 <yoctozepto> then https://review.opendev.org/752917
15:35:50 <patchbot> patch 752917 - kolla-ansible - Allow to skip and unset sysctl vars - 1 patch set
15:35:57 <yoctozepto> because it's another very simple stack
15:36:11 <mgoddard> ok only 2 days yoctozepto
15:36:32 <yoctozepto> this one for mnasiadka https://review.opendev.org/749908
15:36:33 <patchbot> patch 749908 - kolla-ansible - Make no_proxy handling more robust - 3 patch sets
15:36:39 <yoctozepto> mgoddard: ok
15:36:50 <mgoddard> headphoneJames: are you still seeing the memcache issue?
15:36:51 <yoctozepto> mgoddard: we can postpone masakari to make you happier :-)
15:37:16 <headphoneJames> mgoddard: yes
15:37:24 <mgoddard> any idea what's going wrong?
15:37:36 <headphoneJames> not at all
15:37:40 <yoctozepto> :-(
15:38:07 <headphoneJames> very odd, trying to get it working with uwsgi to see if similar failure
15:39:40 <mgoddard> have you checked if it happens on binary images too?
15:39:49 <mgoddard> could be some dependency issue
15:40:18 <yoctozepto> what's the exact issue
15:40:20 <yoctozepto> please link me
15:40:22 <headphoneJames> the docker needs minor changes to support httpd, so haven't tried binary
15:40:34 <headphoneJames> https://review.opendev.org/#/c/749616/
15:40:34 <patchbot> patch 749616 - kolla-ansible - Add support for encrypting Neutron Server API - 12 patch sets
15:41:22 <openstackgerrit> Radosław Piliszek proposed openstack/kolla-ansible master: Control Neutron migrations  https://review.opendev.org/755328
15:41:39 <yoctozepto> AttributeError: 'MemcacheClientPool' object has no attribute 'getters'
15:41:59 <yoctozepto> is not it because we switched that pool for neutron?
15:42:23 <yoctozepto> it might have worked only with eventlet ;/
15:42:48 <yoctozepto> that's the issue that was tried to be mentioned I guess
15:42:55 <yoctozepto> it was pretty unclear
15:43:37 <priteau> Is this related to using the advanced memcache pool?
15:44:06 <yoctozepto> https://review.opendev.org/746966
15:44:07 <patchbot> patch 746966 - kolla-ansible - Add workaround for keystonemiddleware/neutron memc... (MERGED) - 4 patch sets
15:44:09 <yoctozepto> priteau
15:44:12 <yoctozepto> oh, you here already
15:44:19 <headphoneJames> sort of. If I disable advanced memcached pool, I get a different error
15:44:29 <yoctozepto> headphoneJames: gimme the other error
15:44:30 <yoctozepto> :D
15:45:00 <yoctozepto> duh, time really flies
15:47:07 <mgoddard> we're not going to solve this now
15:47:17 <mgoddard> does someone want to help headphoneJames offline?
15:47:52 <headphoneJames> 2020-09-24 23:57:33.637 948 ERROR oslo_middleware.catch_errors AttributeError: module 'select' has no attribute 'poll'
15:48:18 <headphoneJames> help would be greatly appreciated!
15:48:56 <yoctozepto> headphoneJames: I feel it's important to get in so let me help
15:49:09 <mgoddard> #action yoctozepto to help headphoneJames with neutron backend TLS
15:49:14 <yoctozepto> headphoneJames: I want you to summarise what you already know in that review
15:49:23 <headphoneJames> will do
15:49:34 <mgoddard> other patches we want to land?
15:49:48 <yoctozepto> mgoddard: how's octavia/designate?
15:49:48 <mgoddard> I have given the main octavia patch a +2
15:49:54 <yoctozepto> (sorry for being unhelpful)
15:50:06 <mgoddard> https://review.opendev.org/740180
15:50:06 <patchbot> patch 740180 - kolla-ansible - Implement automatic deploy of octavia - 45 patch sets
15:50:13 <yoctozepto> (he be reading my mind again)
15:50:42 <yoctozepto> it has no docs
15:50:52 <mgoddard> docs are in a separate patch
15:51:01 <openstackgerrit> Radosław Piliszek proposed openstack/kolla-ansible master: [DNM] Add NFV upgrade job  https://review.opendev.org/755331
15:51:13 <mgoddard> https://review.opendev.org/#/q/topic:bp/implement-automatic-deploy-of-octavia+(status:open+OR+status:merged)
15:51:13 <yoctozepto> duh, which?
15:51:16 <yoctozepto> ok
15:51:56 <yoctozepto> do they actually consider all patches to be merged?
15:52:05 <mgoddard> yes
15:52:06 <yoctozepto> hard life
15:52:18 <mgoddard> but majority of the code is in the first one
15:52:30 <yoctozepto> yeah, looking by the lines modified count ;-)
15:52:52 <yoctozepto> choices, choices...
15:53:27 <mgoddard> designate is simpler, but a bit less mature
15:53:35 <mgoddard> I've given it a try
15:53:54 <mgoddard> there's a pretty nasty designate bug that prevents deploys once you have registered a zone
15:54:10 <yoctozepto> argh, hurts
15:54:23 <mgoddard> I should probably focus on octavia rather than splitting effort
15:55:31 <mgoddard> any others?
15:55:49 <mgoddard> I guess we have a few minutes for the last topic
15:55:56 <mgoddard> #topic Wallaby PTG planning
15:56:11 <mgoddard> #link https://etherpad.opendev.org/p/kolla-wallaby-ptg
15:56:25 <mgoddard> Does anyone have any thoughts on how to run it this time
15:56:27 <yoctozepto> I added one topic inspired by LinPeiWen
15:56:38 <yoctozepto> what do you mean
15:57:00 <mgoddard> well, we can do what we have done in the past
15:57:05 <mgoddard> or we could try something new
15:57:08 <hrw> zoom/gmeet all day with breaks?
15:57:34 <mgoddard> zoom or meetpad
15:57:45 <mgoddard> times are on the etherpad
15:57:50 <hrw> no meetpad
15:57:54 <mgoddard> 2x 4h sessions, plus 2h for kayobe
15:57:58 <yoctozepto> meetpad = no hrw
15:58:01 <yoctozepto> but I like hrw
15:58:05 <yoctozepto> so zoom
15:58:15 <mgoddard> ok
15:58:38 <yoctozepto> current formula worked quite well
15:59:02 <yoctozepto> I don't see how it can be improved without a major overhaul of rescheduling (e.g. to have longer breaks or whatever)
15:59:08 <yoctozepto> my stance: let's keep it
15:59:28 <mgoddard> ok
16:00:00 <mgoddard> well do all try to have a think about what are the things that we need to discuss as a group
16:00:16 <yoctozepto> ++
16:00:16 <mgoddard> what needs to be done in the next few cycles?
16:00:47 <yoctozepto> I feel we need some considerable refactoring
16:00:56 <mgoddard> blue sky ideas or iteration welcome
16:00:56 <yoctozepto> it's happening with perf patches too
16:01:16 <yoctozepto> otherwise in my usage we are pretty reliable now
16:01:23 <yoctozepto> and cover most of the common usecases
16:01:28 <mgoddard> and on that note
16:01:31 <mgoddard> let's end
16:01:33 <mgoddard> thanks all
16:01:36 <mgoddard> #endmeeting