15:02:46 <mgoddard> #startmeeting kolla
15:02:47 <openstack> Meeting started Wed Aug 21 15:02:46 2019 UTC and is due to finish in 60 minutes.  The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:02:48 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:02:50 <openstack> The meeting name has been set to 'kolla'
15:02:52 <mgoddard> #topic rollcall
15:03:05 <mgoddard> Raise those hands now
15:03:10 <yoctozepto> o/
15:03:12 <Wasaac> o/
15:03:18 <dougsz> Lo
15:03:27 <scottsol> o/
15:03:37 <generalfuzz> o/ (for about 30 mins)
15:04:00 <mnasiadka> o/
15:04:05 <mgoddard> #topic agenda
15:04:07 <mgoddard> * Roll-call
15:04:10 <mgoddard> * Announcements
15:04:12 <mgoddard> ** Kayobe RC1 and stable/stein branches created
15:04:14 <mgoddard> * Review action items from last meeting
15:04:16 <mgoddard> * Kolla whiteboard https://etherpad.openstack.org/p/KollaWhiteBoard
15:04:18 <mgoddard> * Kayobe Stein release status
15:04:20 <mgoddard> * Train release planning
15:04:22 <mgoddard> * Ceph ansible migration
15:04:24 <mgoddard> * Kolla Ansible TLS Internal API
15:04:26 <mgoddard> #topic announcements
15:04:28 <mgoddard> #info Kayobe RC1 and stable/stein branches created
15:04:32 <mgoddard> #info Kayobe RC1 and stable/stein branches created
15:04:53 <yoctozepto> finally :-)
15:05:00 <yoctozepto> congrats kayobees
15:05:05 <mgoddard> I think there are a couple more bug fixes we might want to fix and/or backport
15:05:14 <mgoddard> then we should be good to release
15:05:24 <mgoddard> Any others?
15:05:57 <mgoddard> #topic review action items from last meeting
15:06:00 <mgoddard> mgoddard to ask infra about restarting gerrit
15:06:03 <mgoddard> mgoddard or someone else to check stable backports
15:06:29 <mgoddard> 1 - just did it, clarkb said they will schedule one soon
15:06:40 <mgoddard> 2 - I backported a bunch of stuff last week, mostly merged now
15:06:55 <mgoddard> I guess we should do stable branch releases
15:06:58 * yoctozepto the merge ensurer
15:07:10 <mgoddard> #action mgoddard to release stable branches
15:07:19 <yoctozepto> sure, why not, just merge me those latest Zun fixes
15:07:20 <mgoddard> #topic Kolla whiteboard https://etherpad.openstack.org/p/KollaWhiteBoard
15:07:50 <mgoddard> had some CI hiccups last week, thanks for fixing
15:08:00 <mgoddard> I think we're green again now?
15:08:06 <yoctozepto> green again
15:08:17 <yoctozepto> I think TripleO is back in green as well
15:08:31 <yoctozepto> nope
15:08:38 <yoctozepto> was for a moment though for sure
15:08:43 <yoctozepto> could be something random
15:08:52 <yoctozepto> cloudnull will let us know when they fix themselves
15:09:25 <mgoddard> k
15:09:38 <mgoddard> please everyone keep priority feature status up to date
15:10:29 <mgoddard> moved ubuntu py3 to done
15:11:31 <mgoddard> ok, move on
15:11:34 <mgoddard> #topic Kayobe Stein release status
15:11:40 <mgoddard> Covered it earlier
15:12:10 <mgoddard> Any patches in particular we need?
15:12:21 <mgoddard> priteau, dougsz, jovial?
15:13:15 <priteau> Would be nice to get the venvs update merged for the release, still need to review it
15:13:17 <dougsz> https://review.opendev.org/#/c/666635/ could be nice
15:13:26 <mgoddard> priteau: +1
15:13:37 <dougsz> (that's the one)
15:13:46 <mgoddard> added RP+1
15:13:58 <mgoddard> I think that's the main one
15:14:09 <dougsz> What about https://review.opendev.org/#/c/670502/?
15:14:19 <dougsz> ( iPXE boot with UEFI compute nodes)
15:14:20 <priteau> I was about to suggest it
15:14:28 <priteau> It's quite useful to have it
15:14:30 <mgoddard> oh yeah
15:14:35 <mgoddard> that would also be good
15:14:53 <priteau> What about the Arista code from Stig?
15:15:48 <mgoddard> too late I think
15:15:51 <mgoddard> he's out this week
15:16:19 <priteau> If the code is ready, we could merge and add the release note ourselves?
15:16:36 <mgoddard> that's true
15:16:59 <mgoddard> is that something you have time for priteau ?
15:17:21 <priteau> Writing the release note? yes, shouldn't take long
15:17:25 <mgoddard> thanks
15:17:47 <priteau> We're fairly confident that the Arista code works well to be shipped?
15:18:02 <mgoddard> Well I assume Stig tested it...
15:18:11 <mgoddard> it looked ok to me
15:18:15 <priteau> I guess we can do a 6.0.1 if needed
15:18:19 <mgoddard> yeah
15:18:41 <mgoddard> ok, I think we're good. Let's aim to have a release before next meeting
15:18:50 <mgoddard> hi jovial
15:19:00 <mgoddard> just finishing up kayobe stein discussion
15:19:02 <jovial[m]> hey
15:19:27 <mgoddard> if there are any patches you want in the release, ping me
15:19:39 <mgoddard> #topic Train release planning
15:20:26 <mgoddard> #link https://releases.openstack.org/train/schedule.html
15:21:00 <mgoddard> It's probably a good time to define our own delayed release schedule
15:21:21 <mgoddard> Main feature freeze is Sep 09 - Sep 13
15:21:59 <mgoddard> I think for Stein we lagged by 3 weeks, so Sep 30 - Oct 04 ?
15:22:30 <mnasiadka> there is no new Ceph, so sounds reasonable :D
15:23:43 <mgoddard> let's be optimistic and aim for 3 week lag on release :)
15:23:44 <yoctozepto> sounds reasonable
15:23:49 <mgoddard> main release is Oct 14 - Oct 18
15:24:07 <mgoddard> so we should aim for Nov 04 - Nov 08
15:24:34 <mgoddard> of course we're limited by RDO and others here, so let's not kick ourselves if we miss :)
15:24:47 <mgoddard> but hopefully we don't have last minute mariadb issues
15:25:14 <yoctozepto> mgoddard: only binaries, sources should be safer
15:25:24 <yoctozepto> let's aim for that on our sources
15:26:20 <mgoddard> there is also CentOS 8 to trip us up this time
15:27:00 <mgoddard> I've added those dates to the release status section on the whiteboard
15:27:09 <mgoddard> and added some risks to release
15:27:22 <mgoddard> any other risks, apart from RDO and centos 8?
15:27:36 <mnasiadka> well, for binaries we can test packages from trunk.rdoproject.org for CentOS at least
15:27:38 <yoctozepto> centos 8 is not ready yet
15:27:56 <mnasiadka> yoctozepto: well, it's semi-ready, so in two weeks we might be in a different position
15:28:01 <yoctozepto> nor 7.7 for that matter, there is some serious lag
15:28:17 <mgoddard> #link https://wiki.centos.org/About/Building_8
15:28:45 <yoctozepto> do we have to promise releasing for centos 8? centos-7-based images should be fine for train
15:28:46 <mgoddard> only one step left incomplete
15:29:02 <mgoddard> it means we don't easily get py3 on centos
15:29:19 <mnasiadka> yoctozepto: for now rdo builds on top of centos7, so we are safe
15:29:22 <mgoddard> not sure if RDO train will support centos7
15:29:39 <mnasiadka> but I remember them claiming they will support centos 8 only
15:29:44 <mgoddard> +1
15:30:08 <yoctozepto> oh noez
15:30:14 <yoctozepto> -100
15:30:44 <mgoddard> anything else to discuss for train?
15:30:51 <stackedsax> oh, hi, sorry I'm late
15:30:59 <mgoddard> any feature progress need to be discussed?
15:31:02 <mgoddard> hi stackedsax, np
15:31:33 <mgoddard> ok, let's get to the main course
15:31:46 <mgoddard> #topic Ceph ansible migration
15:31:55 <mnasiadka> oh boy
15:31:58 <mgoddard> mnasiadka has been doing some good investigation here
15:32:21 <mnasiadka> yes, hacking ceph-ansible stable-4.0 so it works
15:32:33 <mnasiadka> but investigation is in the spec more or less
15:32:43 <mnasiadka> https://review.opendev.org/#/c/544980/
15:33:16 <mnasiadka> bottom line is - there are no ceph container images for Ubuntu
15:33:26 <yoctozepto> mnasiadka to provide them
15:33:28 <mnasiadka> there are for Debian, OpenSuse and CentOS
15:33:46 <mnasiadka> yoctozepto: if I'll be bored - then maybe :)
15:34:06 <yoctozepto> did you collect intel as to why they are gone
15:34:27 <mnasiadka> for now I propose to work on CentOS, so we have some experience regarding deployment, migration, etc
15:34:41 <yoctozepto> yes, definitely
15:34:44 <mnasiadka> yoctozepto: simply there was no maintainer, funny thing is ceph-ansible started with Ubuntu as the only platform supported
15:34:55 <yoctozepto> yeah, I heard so
15:34:56 <mnasiadka> yoctozepto: but you know, Red Hat and so on :)
15:35:45 <mnasiadka> Ubuntu shouldn't be hard to add back based on the PR to remove it - the problematic thing is nfs-ganesha 2.8 vs 2.7 in CentOS, and only built for xenial - but probably we could surive
15:35:47 <mnasiadka> survive
15:36:06 <yoctozepto> rebuild ganesha?
15:36:09 <yoctozepto> should not be hard
15:36:11 <mnasiadka> I don't think there is a lot of users of ceph-nfs
15:36:20 <yoctozepto> me neither
15:36:29 <mnasiadka> I think CephFS and Manila has bigger user base in OpenStack world
15:37:25 <mnasiadka> So, bottom line is - please review the spec, I'll share my deployment experiences in the form of updating the external Ceph docs
15:37:34 <mgoddard> I think we need to know what they would expect in terms of maintenance for ubuntu, and whether anyone is prepared to take it on
15:37:52 <mnasiadka> mgoddard: judging by the fact Debian has not been updated in 8 months - I don't think that's a lot :)
15:38:19 <mgoddard> perhaps not, but maybe they would want someone from their core team to be interested
15:38:45 <mgoddard> it's the key thing that could block the whole proposal, right
15:38:48 <mnasiadka> mgoddard: and still the ceph-container builder pushes only centos builds to docker hub - so we would need to add container image building on our side I think
15:39:14 <mnasiadka> mgoddard: I'll try to find out what means maintaining, if that's not a lot of work, I think I can do this
15:39:22 <mgoddard> is that process automated?
15:40:46 <mnasiadka> mgoddard: you mean the builds? there is a travis job that builds the centos ones, it's as simple as make FLAVORS="what do you want to build" build, and then push it somewhere
15:41:12 <mgoddard> would be better if they just published ubuntu
15:41:30 <mnasiadka> they don't publish debian, so I guess that's the same department
15:41:33 <mnasiadka> I'll ask about it
15:41:57 <yoctozepto> and what if we have to do baremetal ceph
15:42:07 <yoctozepto> how well is it supported there
15:42:13 <mgoddard> I suppose we'd need to document building their images, but wouldn't really want to start pushing them to our dockerhub or anything
15:42:26 <mgoddard> yoctozepto: better than containers I think :)
15:44:00 <mgoddard> any more to say on ceph?
15:44:12 <yoctozepto> mnasiadka: ^ baremetal
15:44:58 <mnasiadka> baremetal means not containerized, just packages installed in the OS?
15:45:07 <mgoddard> yes
15:46:41 <mnasiadka> well, that's two settings in group_vars less than containerized?
15:47:00 <mnasiadka> Do we want to have that tested in the CI as well? I don't think it goes well with Kolla philosophy :)
15:47:11 <yoctozepto> ceph is not openstack
15:47:16 <yoctozepto> if it feels better outside
15:47:24 <yoctozepto> then be it
15:48:04 <mgoddard> do we have any info on how many people use ceph containers vs baremetal?
15:48:05 <yoctozepto> though would prefer containers
15:49:15 <mnasiadka> mgoddard: nope, ceph user survey did not include that question in 2018
15:49:46 <mgoddard> ok
15:49:57 <mnasiadka> The free-of-charge Linux distributions – Ubuntu (65.9%), Debian (8.6%), CentOS (28%), openSUSE (0.34%) – combined make up the largest share of deployments. Red Hat Enterprise Linux is used by 8.9%, and SUSE Linux Enterprise by 2.3%.
15:50:05 <mnasiadka> but that shows Ubuntu is on the top :D
15:50:46 <mnasiadka> https://ceph.com/ceph-blog/ceph-user-survey-2018-results/
15:51:01 <mgoddard> that suggests either people are using old releases of ceph-containers, or not using ceph-containers
15:51:14 <mgoddard> i.e. baremetal
15:51:56 <mgoddard> unless we have any more, let's move onto the last topic
15:52:05 <mnasiadka> Most users are using ceph-deploy instead of ceph-ansible
15:52:12 <mnasiadka> so that answers your question
15:52:19 <mgoddard> ok
15:52:34 <mgoddard> #topic Kolla Ansible TLS Internal API
15:52:54 <mgoddard> stackedsax, scottsol, generalfuzz, kklimonda: how's it going?
15:53:16 <stackedsax> generalfuzz has been working away at this
15:53:24 <stackedsax> essentially, we'll be pursuing the pan that seemed to be the consensus from our discussion last week: enabling TLS termination with each and every service
15:53:53 <generalfuzz> I am working on bringing the already in progress transactions to completion
15:54:03 <stackedsax> (generalfuzz had to step away, so I'm speaking for him at the moment)
15:54:06 <stackedsax> oh, or that
15:54:11 <stackedsax> hi generalfuzz
15:54:19 <mgoddard> great, frontend patch is looking close
15:54:51 <generalfuzz> I replied to your comments about documentation, we can talk about that after the meeting
15:54:59 <mgoddard> sure
15:55:08 <mgoddard> so what's the next steps from here?
15:55:33 <stackedsax> after that, we wanted to help get some of scottsol's reviews worked through, and then start the long road of going through each service and getting them going with TLS
15:55:50 <generalfuzz> Next I am tackling https://review.opendev.org/548407 which introduces a new test that needs fixing
15:56:43 <generalfuzz> With the goal of completing all transactions in https://review.opendev.org/#/q/topic:bp/add-ssl-internal-network+(status:open+OR+status:merged
15:57:18 <mgoddard> generalfuzz: I think there's a bit of overlap between that one and https://review.opendev.org/664517
15:57:50 <mnasiadka> are all PRs linked to bp/add-ssl-internal-network? it would ease the pain of reviewing :)
15:58:28 <generalfuzz> Ok, do you have a recommendation on which one I should focus on?
15:59:54 <mgoddard> https://review.opendev.org/548407 uses a slightly different approach to the currently proposed design
16:00:28 <stackedsax> we really weren't keen on 548407's approach
16:00:57 <mgoddard> the changes to the tests and certificate generation are probably worth extracting, but I think much of the rest is covered in https://review.opendev.org/#/c/664517/2
16:01:04 <generalfuzz> I'm on board to be not keen
16:01:10 <stackedsax> or, scottsol and co. would prefer not to go that route for our own infrastructure
16:01:55 <mgoddard> I quite like that they are separate patches, so maybe we need a new patch (or two) with the internal certificate generation and test updates?
16:02:18 <mgoddard> keeping the self-signed certs patch as it is
16:02:21 <mgoddard> make sense?
16:02:45 <mgoddard> then we can abandon Clint's patch and stop confusing people :)
16:02:53 <generalfuzz> Yes. One patch for internal certificate generation and testing
16:03:25 <mgoddard> sure
16:04:03 <mgoddard> ok, we're out of time
16:04:16 <mgoddard> thanks for joining, everyone
16:04:23 <generalfuzz> Just FYI, I'm on vacation next week, but will be back at it on September 3rd
16:04:26 <yoctozepto> \o
16:04:29 <mgoddard> #endmeeting