15:02:17 <mgoddard> #startmeeting kolla
15:02:17 <openstack> Meeting started Wed Sep 11 15:02:17 2019 UTC and is due to finish in 60 minutes.  The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:02:18 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:02:20 <openstack> The meeting name has been set to 'kolla'
15:02:23 <openstackgerrit> Merged x/kayobe master: Fix typo in docs  https://review.opendev.org/681505
15:02:25 <mgoddard> #topic rollcall
15:02:28 <mgoddard> \o
15:02:36 <generalfuzz> \o
15:02:46 <yoctozepto> o/
15:02:51 <hrw> \o\ /o/ \o/ /o/ /o\
15:03:04 <yoctozepto> hrw: dancing?
15:03:13 <hrw> hands waving
15:03:31 <yoctozepto> will do
15:04:03 <hrw> mixor: could you rebase your patch on top of previous versions?
15:04:22 <mgoddard> #topic agenda
15:04:34 <mgoddard> * Roll-call
15:04:37 <mgoddard> * Announcements
15:04:39 <mgoddard> ** Kayobe stein 6.0.0 release now available
15:04:41 <mgoddard> ** mgoddard continues as PTL for another cycle
15:04:43 <mgoddard> ** OpenStack Train feature freeze this week: https://releases.openstack.org/train/schedule.html
15:04:45 <mgoddard> * Review action items from last meeting
15:04:47 <mgoddard> * Kolla whiteboard https://etherpad.openstack.org/p/KollaWhiteBoard
15:04:49 <mgoddard> * Train release planning
15:04:51 <mgoddard> * Review priorities
15:04:53 <mgoddard> #topic announcements
15:04:55 <mgoddard> #info Kayobe stein 6.0.0 release now available
15:05:09 <mgoddard> #link https://docs.openstack.org/releasenotes/kayobe/stein.html
15:05:17 <mgoddard> #info mgoddard continues as PTL for another cycle
15:05:26 <mgoddard> #info OpenStack Train feature freeze this week
15:05:26 <hrw> mgoddard: thanks!
15:05:33 <mgoddard> #link https://releases.openstack.org/train/schedule.html
15:06:04 <mgoddard> #topic Review action items
15:06:05 <hrw> cool. less reasons for failures
15:06:29 <mgoddard> I didn't ask if anyone has any other announcements
15:06:33 <mgoddard> no one ever does :)
15:06:42 <hrw> CentOS 8 will be delayed
15:06:58 <hrw> whole devteam time went for CentOS 7.7 work
15:06:59 <mgoddard> hrw: until when?
15:07:04 <mgoddard> :(
15:07:09 <hrw> mgoddard: Ucycle I would say
15:07:16 <hrw> no eta given etc
15:07:26 <mgoddard> hmm
15:07:47 <mgoddard> let's discuss it later
15:07:53 <mgoddard> #info CentOS 8 will be delayed
15:07:55 <hrw> mgoddard: with small CentOS devteam it was better to spend time on getting 7.7 update to users than wait with it because of 8.0 work
15:08:13 <mgoddard> Actions from last meeting:
15:08:18 <mgoddard> mgoddard to release stable branches
15:08:21 <mgoddard> mgoddard to mark Train priority patches ready for review with RP+1
15:08:26 <mgoddard> Done and done
15:08:38 <mgoddard> #topic Kolla whiteboard
15:08:43 <mgoddard> #link https://etherpad.openstack.org/p/KollaWhiteBoard
15:08:59 <mgoddard> CI update
15:09:09 <mgoddard> For k-a we have a note saying 'Ubuntu jobs seem to be failing in neutron deployment'
15:09:14 <mgoddard> is this current?
15:09:45 <yoctozepto> no, fixed
15:09:48 <yoctozepto> with mariadb
15:09:50 <yoctozepto> upstream
15:09:58 <mgoddard> oh it's that one
15:10:01 <mgoddard> I'll remove
15:10:05 <yoctozepto> ok
15:10:44 <mgoddard> any other CI issues?
15:10:55 <yoctozepto> none I know of
15:11:01 <yoctozepto> but i'm not so current
15:11:10 <hrw> arm64 CI has to wait til Nov ;(
15:11:18 <mgoddard> :(
15:11:30 <hrw> we have to upgrade our ceph cluster with ssds
15:11:42 <yoctozepto> :-(
15:11:58 <hrw> and have to solve ipv6 issues to get other cloud (ssd only) running with kolla
15:12:10 <yoctozepto> I have Ipv6 patch ready for the base openstack
15:12:21 <hrw> yoctozepto: give it! gerrit it!
15:12:27 <yoctozepto> will be submitting today yeah
15:12:28 <mnasiadka> merge it! ;)
15:12:51 <mgoddard> ooh nice
15:13:07 <hrw> to get arm64 ci we have two options
15:13:10 <mgoddard> do we need an IPv6 CI job?
15:13:17 <hrw> mgoddard: would be lovely
15:13:35 <mgoddard> ok, let's move on
15:13:39 <hrw> 1. add ssd to ceph cluster in linaro-london. November earliest
15:13:44 <yoctozepto> we need but that's not done yet
15:13:56 <hrw> 2. run ssd-only cluster in other location. but ipv6 only
15:14:16 <hrw> so for now we go with 2nd
15:14:33 <hrw> once it gets on, we add nodes to infra, disable linaro-london and check how it goes
15:14:40 <mixor> hi, is there way to delete last uploaded change in review
15:14:42 <yoctozepto> happy to help
15:14:45 <mixor> ?
15:14:52 <hrw> mixor: upload next one
15:14:52 <yoctozepto> mixor: no
15:14:59 <yoctozepto> ^ exactly
15:15:22 <mgoddard> #topic Train release planning
15:15:23 <hrw> mixor: when you get to 50th revision start worrying
15:15:36 <mgoddard> ok, let's start with centos 8
15:15:50 <hrw> centos:8 -> Ucycle imho
15:15:57 <mgoddard> I'll update the whiteboard
15:16:10 <mnasiadka> currently looks like CentOS QA team is more into releasing 7.7
15:16:25 <mgoddard> cloudnull might have opinions about pushing centos 8 to U cycle
15:16:29 <hrw> even f28 patch which someone added to get 'rhel8 like' environment goes with py2 packages
15:16:32 <mixor> yoctozepto, hrw: thanks
15:16:40 <mnasiadka> So we might get away and they don't release 8 before T :-)
15:17:25 <hrw> mgoddard: I would treat centos:8 as 'U-cycle unless arrive around Train release and will be working in one-two weeks of time'
15:17:51 <yoctozepto> +1
15:18:04 <hrw> mgoddard: we can always backport if RDO release Train packages for centos:8
15:18:34 <mgoddard> it is a good point - we are somewhat at the mercy of RDO
15:18:47 <mgoddard> if they wait for CentOS 8 then so must we
15:19:02 <mnasiadka> yup
15:19:37 <mgoddard> I remember someone said RH had backported a lot of py3 packages to RHEL7.7. Will the same be true of CentOS 7.7?
15:19:48 <mgoddard> i.e. could we support py3 on CentOS 7.7?
15:20:01 <hrw> centos 7.7 gets python 3.6 in base
15:20:07 <hrw> from what I heard
15:20:17 <mgoddard> ok
15:20:19 <yoctozepto> gr8
15:20:31 <hrw> RDO builds packages for Fedora 28 with Py3
15:21:36 <hrw> so once centos8 gets released then switch to it and do mass rebuild
15:22:05 <mgoddard> 7.7 claims to be 1908, so assume it will be released soon?
15:22:21 <yoctozepto> it's in cr already
15:22:42 <hrw> https://trunk.rdoproject.org/fedora/current/
15:23:05 <hrw> mgoddard: rhel 7.7 was released in 2019.08 so centos 7.7.1908 ;D
15:23:14 <mgoddard> https://wiki.centos.org/Manuals/ReleaseNotes/CentOS7.1908
15:23:26 <hrw> like with centos 8.0.1905
15:23:27 <mgoddard> ah
15:25:30 <mgoddard> ok, let's move on
15:25:54 <mgoddard> support matrix is making good progress
15:25:57 <hrw> http://mirror.centos.org/centos-7/7/cr/x86_64/Packages/python3-3.6.8-10.el7.x86_64.rpm D
15:26:13 <mgoddard> nice
15:26:37 <mgoddard> nova cells work is ongoing
15:26:47 <mgoddard> Support newer ansible versions
15:26:54 <mgoddard> we have kolla-toolbox
15:27:02 <hrw> mnasiadka: for support matrix we need to mark what we build in CI as 'T', right?
15:27:19 <mnasiadka> hrw: yes, that's on my list
15:27:31 <hrw> I would say that we support ansible 2.7+ now. deployment on debian work
15:27:32 <mgoddard> well, do you mean build or test?
15:27:43 <mgoddard> build != test
15:27:48 <mnasiadka> I think T == we test in k-a CI
15:27:51 <mgoddard> +1
15:27:59 <yoctozepto> +1
15:28:04 <hrw> +1
15:28:08 <mgoddard> 3
15:28:40 <mgoddard> re ansible versions
15:28:58 <mgoddard> I proposed https://review.opendev.org/675863
15:29:05 <mgoddard> bump minimum to 2.6
15:29:34 <mgoddard> because I needed a feature from 2.6
15:30:00 <mgoddard> any other reason seems overly restrictive and arbitrary to me
15:30:15 <hrw> why not 2.7? present in most distros nowadays
15:30:19 <mgoddard> but I think I'm in the minority here
15:30:24 <openstackgerrit> Viktor Michalek proposed openstack/kolla master: Fix images based on Debian Buster Stein packages  https://review.opendev.org/680447
15:30:40 <mgoddard> why would we be more restrictive than we need to be?
15:31:11 <yoctozepto> futureproofing
15:31:48 <mgoddard> ansible can have bugs, and sometimes being able to pick between releases can be helpful - often newer releases are less stable for a time
15:31:54 <hrw> 2.6 is supported (security fixes, no critical fixes)
15:31:56 <mgoddard> how is it futureproofing?
15:32:18 <hrw> mgoddard: ok, lets go with 2.6 for Train and we bump to 2.7/2.8 in Ucycle
15:32:46 <openstackgerrit> caoyuan proposed openstack/kolla-cli master: Add upgrade unit test  https://review.opendev.org/681456
15:33:04 <mgoddard> ok
15:33:09 <hrw> Ansible supports last 3 releases. 2.9 will be next release
15:33:32 <mnasiadka> last minute we should not bump a lot, but looking at for example OSA - they use 2.8.3 in the CI - so I guess we are the only ones that care about older releases of Ansible :)
15:34:02 <hrw> so train gets released with still-supported-required, Ucycle with get 'current-1', Vcycle similar bump?
15:34:34 <mnasiadka> I'm still interested why Ubuntu gets a newer Ansible version in the CI, and CentOS gets minimum supported :)
15:35:01 <mgoddard> why not? it allows us to test both
15:35:59 <mgoddard> IMO if we want to be more strict about the version, we need to put Ansible in requirements.txt (and add it to upper-constraints)
15:36:04 <hrw> where from we grab ansible for centos? centos 7 has 2.4.2, epel has 2.8.4
15:36:13 <mgoddard> until then, we're decoupled from it
15:36:27 <mgoddard> hrw: pypi
15:36:31 <mnasiadka> Well, we could make a universal approach and put it in requirements.txt - and use pypi
15:36:41 <mnasiadka> but not in T, we can try that in U and brainstorm on the PTG
15:36:58 <mgoddard> it's too late to add a package to U-C for train, but we could propose in U
15:37:24 <yoctozepto> mgoddard: what about osa
15:37:26 <hrw> mgoddard: right
15:37:30 <yoctozepto> how they handle it
15:37:31 <yoctozepto> hmm
15:37:47 <mgoddard> not sure about OSA
15:37:52 <mgoddard> kayobe includes it in requirements.txt
15:38:06 <hrw> mgoddard: with versions?
15:38:09 <mgoddard> yes
15:38:11 <mgoddard> min and max
15:38:29 <mgoddard> normally two major versions
15:38:38 <mgoddard> bump each release
15:38:43 <mnasiadka> OSA has a bootstrap-ansible.sh script, and they pin it to detailed version, e.g. 2.8.3
15:38:50 <mgoddard> ew
15:38:52 <hrw> ok, can we continue with next step?
15:39:01 <mgoddard> sorry, no bug fixes for you :p
15:39:17 <hrw> we can discuss ansible for any random amount of time ;d
15:39:40 <yoctozepto> rand(0,1)
15:39:58 <mgoddard> favourite pastime
15:40:17 <mgoddard> ok, let's get that patch in and we can cross off another priority feature
15:40:39 <mgoddard> I probably need to rebase it, otherwise it depends on https://review.opendev.org/#/c/665922
15:40:59 <mgoddard> internal TLS
15:41:20 <mgoddard> reviews appreciated on this: https://review.opendev.org/#/q/topic:bp/add-ssl-internal-network+(status:open+OR+status:merged)
15:41:40 <generalfuzz> that be my focus
15:41:47 <mgoddard> right
15:42:00 <mgoddard> are you looking at the insecure flag?
15:42:06 <mgoddard> https://review.opendev.org/#/c/664517
15:42:11 <generalfuzz> yes
15:42:42 <generalfuzz> it will require a kolla fix, which I hope to have finished today
15:42:50 <mgoddard> great
15:43:47 <mgoddard> #topic Review priorities
15:43:56 <openstackgerrit> caoyuan proposed openstack/kolla-cli master: Add stop unit test with services and hosts  https://review.opendev.org/681520
15:44:02 <mgoddard> Are there any patches that we need to focus on?
15:44:13 <mgoddard> https://tiny.cc/kolla-dashboard
15:44:18 <mgoddard> https://tiny.cc/kolla-ansible-dashboard
15:44:22 <mgoddard> https://tiny.cc/kayobe-review-dash
15:44:32 <mnasiadka> do we want to backport the Fluentd stuff?
15:44:35 <hrw> I want https://review.opendev.org/#/c/680399/ - just mnasiadka or yoctozepto need to +w+2 ;d
15:44:55 <openstackgerrit> Merged openstack/kolla-cli master: Add defualt property list  https://review.opendev.org/672488
15:45:11 <hrw> mnasiadka: I see that you volounter for it?
15:46:01 <mnasiadka> I don't know, the zun job is failing :)
15:46:07 <mnasiadka> (kidding, give me a minute)
15:46:47 <mnasiadka> done
15:47:15 <hrw> btw - asked on #rdo about py3 based packages for centos:7.7
15:48:44 <mgoddard> I added RP+1 to https://review.opendev.org/#/c/676412
15:49:47 <hrw> no py3 packages from RDO for centos 7.7 - they wait for centos 8 still
15:50:37 <mgoddard> hmm
15:51:01 <mgoddard> so will train packages still support centos 8 only?
15:52:05 <hrw> py2 for centos7
15:52:09 <mgoddard> ok
15:52:11 <hrw> py3 for centos8
15:52:50 <hrw> in worst case we get rid of py2 in Ucycle.
15:52:58 <hrw> can live with it
15:53:12 <mgoddard> yeah
15:53:21 <hrw> some of deb/ubu images will have py2 anyway
15:53:39 <mgoddard> potentially we could backport support for centos8, but keep 7 as the default
15:54:02 <mgoddard> let's see how it works out
15:54:22 <hrw> 17:53 < ykarel|away> hrw, if we have centos8 sooner we will have cloud/openstack for centos8 and python3
15:54:25 <hrw> 17:54 < ykarel|away> hrw, you can refer https://review.rdoproject.org/etherpad/p/moving-rdo-to-centos8 for plan
15:54:28 <hrw> 17:54 < ykarel|away> but it all depends on centos8 availability and it's timing
15:54:32 <hrw> simple
15:54:54 <mgoddard> thanks
15:55:02 <mgoddard> #topic Open discussion
15:55:12 <mgoddard> Anyone have anything else today?
15:55:33 <yoctozepto> already said about ipv6 patch coming today
15:55:42 <yoctozepto> so no
15:55:45 <hrw> no
15:55:55 <yoctozepto> mnasiadka
15:55:59 <hrw> gave info about arm64 ci already
15:56:31 <mnasiadka> I don't have anything :)
15:56:49 <yoctozepto> mnasiadka: you have us :-)
15:57:19 <mnasiadka> to add! ;)
15:57:49 <mgoddard> ok, let's wrap up 3 minutes early
15:57:53 <mgoddard> Thanks alll
15:57:53 <mnasiadka> mgoddard: do you plan to iterate on the support matrix ,,automation'', or should I just try to finish the current form and we can try to think about it in future?
15:58:03 <mnasiadka> (think about automating in future)
15:58:37 <mgoddard> mnasiadka: let's go with the manual approach, I was just playing and did the "fun" 10% effort 90% working
15:58:48 <mgoddard> Still needs the remaining 90% effort for the last 10% :)
15:58:58 <mgoddard> and probably not worth it at this stage
15:59:03 <yoctozepto> +1
15:59:14 <mnasiadka> Agreed and understood
15:59:21 <mgoddard> I think it's a nice idea though, that we define our support matrix in our code
15:59:26 <openstackgerrit> caoyuan proposed openstack/kolla-cli master: Add upgrade unit test  https://review.opendev.org/681456
15:59:44 <mgoddard> we have most of it already, just need the mapping of C, M and T
15:59:50 <yoctozepto> +1
15:59:55 <mgoddard> could probably use a similar regex method
16:00:13 <mgoddard> for now, s/C/T/ will do
16:00:44 <hrw> 18:00 < jpena> hrw: to add some data, the python3 stack in centos 7.7 is quite minimal, so we'd be lacking a lot of the python libs that are required by RDO
16:01:10 <mgoddard> #endmeeting