15:00:15 <yoctozepto> #startmeeting kolla
15:00:16 <openstack> Meeting started Wed Jan  8 15:00:15 2020 UTC and is due to finish in 60 minutes.  The chair is yoctozepto. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:17 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:19 <openstack> The meeting name has been set to 'kolla'
15:00:29 <yoctozepto> #topic rollcall
15:00:33 <hrw> patch ^^ has to wait for https://review.opendev.org/701552 one
15:00:33 <patchbot> patch 701552 - opendev/system-config - UCA: mirror Ussuri packages - 1 patch set
15:00:34 <hrw> o/
15:00:38 <yoctozepto> \o
15:00:41 <chason> o/
15:01:02 <cosmicsound> o/
15:02:01 <yoctozepto> mgoddard and mnasiadka have limited availability for today
15:02:05 <yoctozepto> so let's continue
15:02:21 <mnasiadka> o/
15:02:31 <yoctozepto> #topic agenda
15:02:39 <yoctozepto> * Roll-call
15:02:39 <yoctozepto> * Announcements
15:02:40 <yoctozepto> ** Welcome Dincer Celik (osmanlicilegi) to kolla core team
15:02:40 <yoctozepto> ** Ceph deployment support will be removed in Ussuri
15:02:40 <yoctozepto> * Review action items from last meeting
15:02:40 <yoctozepto> * CI status
15:02:40 <yoctozepto> * Train release planning
15:02:41 <yoctozepto> * Ussuri release planning
15:02:41 <yoctozepto> * So long, Ceph deployment
15:02:42 <yoctozepto> * CentOS 8 migration
15:02:42 <yoctozepto> * CI scenarios
15:02:43 <yoctozepto> * unpublishing images, e.g. https://hub.docker.com/r/kolla/ubuntu-source-qdrouterd/tags
15:02:51 <yoctozepto> #topic announcements
15:03:01 <yoctozepto> Welcome Dincer Celik (osmanlicilegi) to kolla core team!
15:03:17 <yoctozepto> our newest addition to the best team around 8-)
15:03:20 <hrw> +2
15:03:34 <yoctozepto> osmanlicilegi is probably swimming now
15:03:49 <yoctozepto> so might join us later
15:04:02 <yoctozepto> still, sincere congratulations!
15:04:04 <openstackgerrit> Merged openstack/kolla-ansible stable/stein: Fix nova and cinder handlers with no ceph  https://review.opendev.org/698680
15:04:05 <chason> osmanlicilegi, congras
15:04:15 <cosmicsound> congrats osmanlicilegi
15:04:31 <yoctozepto> Ceph deployment support will be removed in Ussuri
15:04:44 <yoctozepto> we kind-of knew that already
15:05:02 <yoctozepto> but let's keep that in mind we are forced to remove it atm due to c8
15:05:09 <yoctozepto> c8=centos8
15:05:28 <hrw> and we wanted to remove it for quite a while
15:05:30 <yoctozepto> mnasiadka working on better/tighter ceph-ansible
15:05:32 <cosmicsound> rhel8 in same category?
15:05:33 <yoctozepto> integration
15:05:34 <hrw> now have more excuses
15:05:44 <yoctozepto> hrw: exactly
15:05:57 <yoctozepto> cosmicsound: yes, unless explicitly noted otherwise rhel8=centos8 for kolla purposes
15:06:11 <cosmicsound> thanks for clarification
15:06:12 <yoctozepto> this applies to kolla-ansible
15:06:21 <yoctozepto> we will no longer *deploy* ceph cluster
15:06:35 <yoctozepto> but integration with an existing cluster is supported
15:06:35 <hrw> cosmicsound: the only user of rhel is tripleo - they only build templates
15:06:46 <cosmicsound> good to know hrw
15:06:47 <hrw> cosmicsound: at least that's how I prefer to think
15:06:51 <mnasiadka> Did we deprecate Ceph images in Kolla as well? Or only K-A part?
15:06:51 <yoctozepto> hrw, cosmicsound: ^ that we know of
15:07:16 <yoctozepto> mnasiadka: not sure on that, c8 forces removal of centos parts
15:07:32 <yoctozepto> we could keep ubuntu/debian lying around for another release if need be
15:07:40 <yoctozepto> but there are not users outside k-a there
15:07:43 <yoctozepto> no known*
15:07:55 <kplant> does this mean ceph-ansible will not be a supported role in kolla-ansible?
15:08:20 <kplant> or only referring to legacy ceph deployment via k-a
15:08:35 <yoctozepto> kplant: legacy
15:08:48 <mnasiadka> We are removing legacy ceph in k-a
15:08:49 <yoctozepto> k-a will stop deploying ceph cluster in ussuri release
15:08:53 <yoctozepto> it has been deprecated
15:08:57 <yoctozepto> and is being removed
15:09:02 <mnasiadka> Only external ceph will be supported
15:09:05 <yoctozepto> ceph external worked and will work
15:09:10 <yoctozepto> and we support that
15:09:16 <yoctozepto> and we <3 ceph
15:09:19 <mnasiadka> No ceph-ansible or other deployer integration
15:09:20 <yoctozepto> etc.
15:09:25 <kplant> right, what i mean specifically is that there was an effort to integrate ceph-ansible
15:09:28 <kplant> that's been abandoned?
15:09:32 <yoctozepto> just stop deploying/upgrading cluster part
15:09:44 <yoctozepto> kplant: mnasiadka working on that exactly
15:09:47 <yoctozepto> I mentioned that above
15:09:49 <cosmicsound> seen also discussion about replacing k-a recipe
15:09:52 <mnasiadka> kplant: there was, but ceph-ansible doesn’t have a future
15:10:07 <cosmicsound> makes sense mnasiadka then
15:10:10 <kplant> ouch
15:10:11 <yoctozepto> yes, the integration will not be tight
15:10:20 <yoctozepto> just enough to follow instructions
15:10:56 <yoctozepto> let's move on
15:11:06 <yoctozepto> #topic Review action items from last meeting
15:11:17 <yoctozepto> mnasiadka to remove priority from inactive blueprints
15:11:24 <yoctozepto> that's the only one
15:11:29 <yoctozepto> mnasiadka: done?
15:11:42 <mnasiadka> Partly, will finish this week
15:11:58 <yoctozepto> #action mnasiadka to continue removing priority from inactive blueprints
15:12:22 <yoctozepto> #topic CI status
15:12:38 <yoctozepto> we had this horizon failure on all the branches
15:12:46 <hrw> done
15:12:48 <yoctozepto> but it has been fixed
15:13:02 <yoctozepto> thanks hrw, mnasiadka (and myself) for handling
15:13:04 <hrw> aarch64 publishing waits in queue
15:13:23 <hrw> this time done properly
15:13:43 <hrw> can not be tested on gerrit so we need to merge it, observe and revert if broken
15:14:21 <yoctozepto> k green, k-a green, k-cli no recent jobs
15:14:35 <hrw> kayboe?
15:14:43 <yoctozepto> hrw: yes, I saw that, flaky but no better way for now
15:14:49 <yoctozepto> kayobe relatively green
15:15:25 <hrw> https://review.opendev.org/#/c/701011/ waits for fresh votes
15:15:25 <patchbot> patch 701011 - kolla - Add AArch64 publisher for Debian/source - 12 patch sets
15:15:41 <yoctozepto> hrw: after meeting :-)
15:15:54 <yoctozepto> #topic Train release planning
15:16:07 <yoctozepto> so we still have a few pending issue to fix on our recent release
15:16:18 <yoctozepto> most notably the check for ansible being installed properly along k-a
15:16:38 <yoctozepto> also, recently people again use too new k-a to deploy older k
15:16:51 <yoctozepto> (due to pip install kolla-ansible giving them train now)
15:17:01 <yoctozepto> so it might be worth looking into
15:17:36 <yoctozepto> I would include the mariadb deployment fix pack, and possibly these two, and then release
15:17:49 <yoctozepto> recently we got ipv6 fixed for centos7 courtesy of 99cloud
15:17:56 <yoctozepto> (incl. Jeffrey4l)
15:18:11 <yoctozepto> again, great job, folks
15:18:29 <yoctozepto> anyone anything regarding train?
15:18:32 <yoctozepto> hrw, mnasiadka
15:19:02 <hrw> not that I know
15:19:13 <mnasiadka> Still working on the fernet fix, but it doesn’t show up often
15:19:31 <yoctozepto> mnasiadka: and yeah, fernet if we get that reviewed, but we probably will
15:19:55 <yoctozepto> #topic Ussuri release planning
15:20:03 <yoctozepto> how's ussuri?
15:20:27 <hrw> c8 is blocked but there are patches for reviews
15:20:50 <yoctozepto> ah yes, many pending for me
15:20:54 <hrw> centos team says that SIGs may get buildroots after 8.1 release finally done
15:21:10 <yoctozepto> nice, hope it happens soon enough
15:21:17 <hrw> ubuntu got patch to move to UCA/ussuri
15:21:25 <yoctozepto> I see mnasiadka having progress on ovn
15:21:27 <hrw> infra got patch to start mirroring UCA/ussuri as well
15:21:47 <yoctozepto> there is also some moves in the TLS zone, in need of reviewing
15:22:00 <yoctozepto> hrw: uca +1
15:22:08 <mnasiadka> yoctozepto: yeah, need to polish testing (core-openstack)
15:22:35 <yoctozepto> mnasiadka: does it break the normal procedures?
15:23:04 <hrw> Polish testing... start to be afraid
15:23:37 <yoctozepto> hrw: shibboleth?
15:24:11 <mnasiadka> yoctozepto: no, have to adapt init-runonce to use geneve instead vzlan only on ovn scenario
15:24:18 <yoctozepto> chrząszcz brzmi w trzcinie w Szczebrzeszynie, i Szczebrzeszyn z tego słynie :-)
15:24:34 <mnasiadka> yoctozepto: something like that :)
15:24:36 <yoctozepto> mnasiadka: ah, it's geneve finally somewhere
15:24:49 <hrw> mnasiadka: init-runonce... it waits for years to be playbooked...
15:25:00 <yoctozepto> hrw: it's optional
15:26:16 <mgoddard> o/
15:26:28 <yoctozepto> we welcome our PTL, mr mgoddard
15:26:29 <mgoddard> sorry for lateness, busy with a client
15:26:49 <mgoddard> (still busy :))
15:27:10 <yoctozepto> well, let's move on
15:27:12 <yoctozepto> #topic So long, Ceph deployment
15:27:20 <yoctozepto> it has been discussed already above
15:27:50 <yoctozepto> mgoddard: did you remember deprecating something cephish on kolla side?
15:27:58 <yoctozepto> do* you
15:28:16 <yoctozepto> I seem unable to find traces of it
15:28:26 <mgoddard> yoctozepto: I thought I had
15:29:13 <yoctozepto> mgoddard: no signs of it
15:29:16 <mgoddard> perhaps it was just k-a
15:29:27 <yoctozepto> yeah, that was probably what it was
15:29:40 <cosmicsound> So ceph deployment still works on all previous releases beside ussuri?
15:29:52 <cosmicsound> Or will be completely deprecated from k-a
15:30:02 <yoctozepto> cosmicsound: yeah, you can essentially deploy still with Train
15:30:10 <yoctozepto> and then care for such deployment yourself
15:30:20 <cosmicsound> Was just getting ready to do so
15:30:21 <yoctozepto> older branches are still supported
15:30:25 <openstackgerrit> Viktor Michalek proposed openstack/kolla-ansible master: Support for custom API paste files  https://review.opendev.org/695054
15:30:50 <yoctozepto> if someone is to pick it up, we could externalize the code for it
15:30:59 <yoctozepto> but that's not the topic for today I guess
15:31:09 <yoctozepto> all righty
15:31:12 <yoctozepto> #topic CentOS 8 migration
15:31:17 <yoctozepto> the huge on
15:31:32 <yoctozepto> I hope you guys did homework and read mgoddard's notes - hrw, mnasiadka
15:31:41 <yoctozepto> or rather a "spec"
15:31:46 <hrw> iirc +2 it
15:31:55 <yoctozepto> yeah, I think you did
15:32:56 <yoctozepto> due to your time constraints, mgoddard and mnasiadka, we might have to postpone this and remaining topics for the next meeting?
15:32:58 <mgoddard> any issues with the proposed approach?
15:33:08 <yoctozepto> ok, you here
15:33:29 <yoctozepto> I think it's quite well-covering now
15:33:52 <yoctozepto> I guess this is as much as one can get without actually doing it
15:34:21 <mgoddard> anyone know the latest on CentOS 8 + ceph?
15:34:31 <mnasiadka> I think it’s good, do want to merge it and add stuff later, or leave it dangling?
15:34:33 <mgoddard> I think that's the main gap in the support we have
15:34:50 <mnasiadka> mgoddard: in progress
15:34:58 <mnasiadka> Slowly...
15:35:05 <hrw> mgoddard: still same: no packages
15:35:09 <mgoddard> if everyone's happy then let's merge
15:35:28 <yoctozepto> yup, let's merge
15:35:29 <hrw> https://koji.mbox.centos.org/koji/packageinfo?packageID=220 still lists only old build of ceph 12
15:36:41 <openstackgerrit> Merged openstack/kolla-ansible master: Change Manila cephfs driver  https://review.opendev.org/701515
15:36:44 <yoctozepto> #topic CI scenarios
15:36:51 <yoctozepto> my one finally
15:37:11 <yoctozepto> because someone has been prioritizing theirs :-)
15:37:18 <yoctozepto> so, folks
15:37:26 <yoctozepto> I made the extra scenarios
15:37:35 <yoctozepto> we agreed to and merged the swift one
15:37:37 <hrw> yoctozepto: who cares about CI?
15:37:41 <yoctozepto> plus the great refactoring
15:37:53 <yoctozepto> hrw: I don't know
15:38:08 <yoctozepto> senlin and qinling are dangling atm
15:38:35 <yoctozepto> it surely looks bad to indefinitely grow the list of scenarios for almost each single service
15:38:58 <yoctozepto> especially since they use a lot of the same infra services
15:39:39 <yoctozepto> e.g. one could save cycles on all preparation, bootstrapping, common, rabbitmq, mariadb, memcached etc.
15:40:00 <hrw> yoctozepto: group them, check do they fit in 3h time?
15:40:23 <yoctozepto> might be as easy as that, surely would work senlin
15:40:27 <yoctozepto> and qinling
15:40:49 <mgoddard> I think we just need to come up with some sensible groupings
15:40:58 <yoctozepto> we could get all, let's say, "less impacting" services into one extra job
15:41:06 <hrw> just not stretch time to much so changes will be checked in sane time
15:41:11 <yoctozepto> mgoddard: any proposals of such?
15:41:32 <mgoddard> no
15:41:46 <mgoddard> impact is one helpful dimension though
15:42:02 <yoctozepto> then we might be best throwing in those extra little devils in that one job
15:42:08 <yoctozepto> keeping current scenarios for now
15:42:08 <mgoddard> as is dependencies - group things that depend on nova etc
15:42:40 <yoctozepto> ah, yeah, this is a good hint
15:42:51 <mgoddard> and similaritity
15:42:59 <mgoddard> monitoring, storage, etc.
15:43:16 <mgoddard> although overlapping roles might cause problems
15:43:53 <mgoddard> magnum & octavia could go together
15:44:10 <mgoddard> maybe qinling too
15:44:22 <mgoddard> containery stuff
15:44:35 <yoctozepto> mhm, that makes sense
15:44:38 <cosmicsound> kuryr/zun
15:44:53 <hrw> cosmicsound: +2
15:45:08 <mgoddard> I guess we need a list of services on an etherpad, and we can propose some groupings
15:45:20 <mgoddard> anyone want to sign up to starting the list?
15:45:20 <yoctozepto> though zun does not do nova
15:46:18 <yoctozepto> mgoddard: you know who the CI guy is, why do you ask?
15:46:19 <cosmicsound> then kuryr added next to magnum
15:47:11 <mgoddard> #action CI guy to start an etherpad with a list of services for CI testing
15:49:02 <yoctozepto> works for me
15:49:11 <mnasiadka> mgoddard: maybe we could create the promised doc?:)
15:49:24 <mgoddard> mnasiadka: that would be nice
15:49:51 <mnasiadka> csv-table and go :)
15:50:00 <yoctozepto> ah, the promised doc
15:50:09 <yoctozepto> mnasiadka is right
15:50:22 <yoctozepto> it was on me and I forgot, sorry
15:51:12 <mnasiadka> yoctozepto: no need to be sorry, I forget what I do on daily basis :)
15:51:43 <yoctozepto> mnasiadka: if you say so
15:51:46 <yoctozepto> let's move on
15:52:04 <yoctozepto> #topic unpublishing images, e.g. https://hub.docker.com/r/kolla/ubuntu-source-qdrouterd/tags
15:52:11 <yoctozepto> so we have some images dangling
15:52:20 <yoctozepto> preteding to be the latest, greatest in the world
15:52:34 <yoctozepto> yet not so much in reality
15:52:49 <yoctozepto> a policy on unpublishing is needed
15:52:53 <cosmicsound> i hit this issue in train
15:53:17 <hrw> we can create unpublish job
15:53:35 <yoctozepto> to clean up master tags at least
15:53:36 <hrw> with some list of images to unpublish maintained in repo
15:55:48 <yoctozepto> could work
15:55:59 <hrw> quite easy stuff
15:56:00 <yoctozepto> better to download manifest and compare against current list
15:56:20 <yoctozepto> anyone volunteering?
15:57:22 <mjturek> hrw sorry for the delay but in response to yesterday's question, I'm fine with it as is!
15:57:38 <mjturek> referring to https://review.opendev.org/#/c/699361/
15:57:38 <patchbot> patch 699361 - kolla - build: remove not buildable templates for --templa... - 2 patch sets
15:57:40 <hrw> mjturek: thanks
15:57:55 <yoctozepto> I guess not, so I will sketch it as blueprint and leave it be for now
15:58:07 <yoctozepto> #action yoctozepto to create bp for unpublishing
15:58:53 <yoctozepto> we don't really have time for free-form discussion in today's session I'm afraid
15:59:03 <hrw> happens
15:59:11 <yoctozepto> thank you, folks, for attending :-)
15:59:20 <yoctozepto> #endmeeting