15:00:42 <mgoddard> #startmeeting kolla
15:00:45 <openstack> Meeting started Wed Jul 22 15:00:42 2020 UTC and is due to finish in 60 minutes.  The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:46 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:49 <openstack> The meeting name has been set to 'kolla'
15:00:51 <mgoddard> #topic rollcall
15:00:54 <mgoddard> \o
15:01:12 <priteau> o/
15:01:20 <osmanlicilegi> o/
15:01:47 <headphoneJames> o/
15:02:44 <yoctozepto> o/
15:03:46 <bbezak> o/
15:03:47 <mgoddard> #topic announcements
15:03:53 <mgoddard> #undo
15:03:54 <openstack> Removing item from minutes: #topic announcements
15:03:56 <mgoddard> #topic agenda
15:04:04 <mgoddard> * Roll-call
15:04:07 <mgoddard> * Announcements
15:04:09 <mgoddard> * Review action items from last meeting
15:04:11 <mgoddard> * CI status
15:04:13 <mgoddard> * Victoria release planning (kayobe)
15:04:15 <mgoddard> * Victoria release planning (kolla & kolla ansible)
15:04:17 <mgoddard> * Kolla klub and kall
15:04:19 <mgoddard> #topic announcements
15:04:25 <mgoddard> Anyone have any?
15:05:08 <mgoddard> #info 3 months until OpenStack Victoria release
15:05:10 <mgoddard> #link https://releases.openstack.org/victoria/schedule.html
15:05:31 <mgoddard> Always good to get an idea of where we are
15:05:35 <mgoddard> brb, doorbell
15:06:33 <mgoddard> back
15:06:44 <mgoddard> #topic Review action items from last meeting
15:06:48 <mgoddard> No action items
15:06:55 <mgoddard> #topic CI status
15:07:51 <mgoddard> How are we looking?
15:07:59 <mgoddard> #link https://etherpad.opendev.org/p/KollaWhiteBoard
15:08:05 <mgoddard> L24
15:08:20 <mgoddard> Claims to be green
15:08:24 <osmanlicilegi> pretty green with some yellow flowers
15:08:52 <mgoddard> #topic Victoria release planning (kayobe)
15:09:06 <mgoddard> ok kayobians, how are we doing?
15:09:32 <mgoddard> priteau, dougsz
15:09:39 <priteau> o/
15:09:50 <dougsz> |o
15:10:15 <mgoddard> kayobe has been pretty quiet recently
15:10:26 <mgoddard> everyone tired after ussuri?
15:10:34 <jovial[m]> holiday season ;-)
15:10:57 <priteau> Customer deployment season ;-)
15:11:12 <dougsz> It just works (Tm)
15:11:21 <priteau> We have a few patches that could be merged soon, they already have a +2
15:11:23 <mgoddard> all good excuses
15:11:49 <priteau> https://review.opendev.org/#/q/status:open+project:openstack/kayobe+branch:master+label:Code-Review%252B2
15:12:44 <mgoddard> +1, let's at least get those in
15:13:16 <mgoddard> a few stable backports open too: https://review.opendev.org/#/q/status:open+project:openstack/kayobe+NOT+branch:master
15:13:42 <mgoddard> I remembered that we do have some CI issues with Kayobe
15:13:57 <mgoddard> overcloud upgrade jobs fail from time to time, out of disk
15:14:15 <mgoddard> is there anyone who would (ahem) like to investigate this?
15:14:34 <priteau> No time for CI work at the moment :(
15:14:58 <mgoddard> yeah, I've been busy too
15:15:10 <jovial[m]> Are we getting nodes with smaller disks occasionally?
15:15:16 <mgoddard> perhaps
15:15:44 <jovial[m]> I could put it on my backlog
15:16:01 <mgoddard> is there a quick fix we could apply/
15:16:12 <mgoddard> prune old images after the upgrade?
15:16:16 <jovial[m]> don't suppose you have an example failure?
15:16:42 <jovial[m]> Interesting idea
15:16:47 <jovial[m]> any idea where it fails?
15:17:51 <mgoddard> looks like it is a bit more reliable recently: https://zuul.openstack.org/builds?job_name=kayobe-overcloud-upgrade-centos8
15:17:58 <wuchunyang> if we prune old images, How to ensure fast rollback ?
15:18:20 <jovial[m]> I think this is only for CI
15:18:56 <mgoddard> +1
15:20:20 <mgoddard> there is also this one:
15:20:23 <mgoddard> Failed to prepare to deploy: IPMI call failed: chassis bootdev pxe
15:20:58 <jovial[m]> that is just simulating a flaky BMC ;-)
15:21:39 <mgoddard> so every BMC?
15:22:04 <mgoddard> well let's keep an eye on kayobe CI failures, and keep the whiteboard updated
15:22:13 <jovial[m]> sounds good :)
15:22:21 <mgoddard> #topic Victoria release planning (kolla & kolla ansible)
15:22:37 <mgoddard> How are we doing with victoria??
15:22:54 <mgoddard> Does anyone have a feature they'd like to discuss?
15:23:17 <wuchunyang> i am now doing automatic octavia
15:24:14 <headphoneJames> What is automatic octavia?
15:24:44 <wuchunyang> deploy octavia  automatically
15:25:23 <yoctozepto> mgoddard: we are kinda doing more complete masakari
15:25:25 <wuchunyang> just enable octavia , and upload iamge, then octavia will work.
15:25:35 <yoctozepto> anyone with any masakari experience?
15:26:14 <mgoddard> wuchunyang: yes, good to see progress on that. Are there any questions you have about it?
15:26:40 <wuchunyang> yes , i am now just test on my aio ubuntu and 3 nodes centos8
15:27:08 <wuchunyang> i have no debian system
15:27:35 <mgoddard> would you be able to add a CI test for octavia?
15:28:00 <wuchunyang> maybe i can try to add it
15:29:13 <mgoddard> you could start with a test that just deploys octavia containers, then improve the test to use your feature when it merges
15:29:33 <mgoddard> we can help with CI tests if you like
15:30:39 <wuchunyang> is there a example to add this ?
15:32:05 <mgoddard> https://opendev.org/openstack/kolla-ansible/commit/f44876c406fcb7edf28e1d2f9ee894a220c362bc
15:32:32 <mgoddard> that one adds a prometheus/EFK test
15:33:09 <wuchunyang> good ps, i will try to add too
15:33:35 <headphoneJames> I am still struggling with TLS in nova-api
15:33:41 <wuchunyang> thanks mgoddard
15:34:21 <mgoddard> headphoneJames: still hitting the timeout issue?
15:34:32 <mgoddard> headphoneJames: have you tried looking at httpd timeout configs?
15:35:29 <headphoneJames> It's the random remote disconnect errors
15:35:56 <mgoddard> yeah, I thought it seemed to be cause by a timeout?
15:36:14 <mgoddard> have you been able to reproduce locally?
15:36:22 <headphoneJames> Not yet
15:36:44 <headphoneJames> Am going to spend more time today trying to do that
15:37:51 <mgoddard> you could try lowering timeouts in htpd
15:37:55 <mgoddard> httpd
15:38:30 <mgoddard> since CI can be slow, it hits issues you might not hit locally
15:39:43 <mgoddard> Anyone else have things to discuss re victoria features?
15:40:44 <mgoddard> #topic Kolla klub and kall
15:40:57 <mgoddard> We have a kolla kall tomorrow
15:41:14 <mgoddard> I think the plan is to continue looking at documnetation
15:41:24 <mgoddard> How did the klub go last week?
15:41:34 <mnasiadka> I made it before the end of the meeting! ;-)
15:43:10 <mgoddard> Is there anything we should discuss next week?
15:43:25 <mgoddard> https://docs.google.com/document/d/1EwQs2GXF-EvJZamEx9vQAOSDB5tCjsDCJyHQN5_4_Sw/edit#
15:44:41 <mgoddard> #topic Open discussion
15:46:53 <mgoddard> Seems people are quiet today
15:46:56 <mgoddard> Let's end the meeting
15:47:03 <mgoddard> Thanks all
15:47:06 <mgoddard> #endmeeting