15:00:13 <yoctozepto> #startmeeting kolla
15:00:13 <openstack> Meeting started Wed Apr 14 15:00:13 2021 UTC and is due to finish in 60 minutes.  The chair is yoctozepto. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:14 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:17 <openstack> The meeting name has been set to 'kolla'
15:00:59 <yoctozepto> #topic Roll-call
15:01:02 <yoctozepto> \o/
15:01:03 <openstackgerrit> Michal Arbet proposed openstack/kolla-ansible master: Add mariadb arbitrator to mariadb role  https://review.opendev.org/c/openstack/kolla-ansible/+/780811
15:01:04 <openstackgerrit> Michal Arbet proposed openstack/kolla-ansible master: [CI] Test Mariadb-Arbitrator with shards in the nova cells scenario  https://review.opendev.org/c/openstack/kolla-ansible/+/780970
15:01:26 <headphoneJames> o/
15:01:30 <mnasiadka> o/
15:03:39 <yoctozepto> it's not quantity but quality that counts they say
15:03:45 <yoctozepto> #topic Agenda
15:03:56 <yoctozepto> * Roll-call
15:03:56 <yoctozepto> * Announcements
15:03:56 <yoctozepto> ** mgoddard on holiday this week
15:03:56 <yoctozepto> ** Xena PTG next week! https://etherpad.opendev.org/p/kolla-xena-ptg
15:03:56 <yoctozepto> ** OpenStack User Survey 2020 is in! https://www.openstack.org/analytics
15:03:57 <yoctozepto> * Review action items from the last meeting
15:03:57 <yoctozepto> * CI status
15:03:58 <yoctozepto> * Review requests
15:03:58 <yoctozepto> * Wallaby release planning
15:03:59 <yoctozepto> ** Debian bullseye progress
15:03:59 <yoctozepto> * Xena PTG planning https://etherpad.opendev.org/p/kolla-xena-ptg
15:04:22 <yoctozepto> #topic Announcements
15:04:31 <yoctozepto> mgoddard on holiday this week
15:04:43 <yoctozepto> all wish him a good rest; he deserves it ;-)
15:04:59 <yoctozepto> Xena PTG next week! https://etherpad.opendev.org/p/kolla-xena-ptg
15:05:13 <yoctozepto> yeas! it's this time of the year again
15:05:40 <yoctozepto> grab your entry cards and join us on the ptg
15:05:58 <yoctozepto> and add discussion topics to the etherpad
15:06:07 <mnasiadka> it's this week where you end up exhausted like after running a marathon? ;)
15:06:21 <yoctozepto> mnasiadka: yeah, I already feel it in my bones, ouch
15:06:37 <yoctozepto> but it's worth the effort! :D
15:06:58 <yoctozepto> #info OpenStack User Survey 2020 is in! https://www.openstack.org/analytics
15:07:16 <yoctozepto> 15% of deployments are powered by kolla
15:07:21 <yoctozepto> kolla stronk
15:07:56 <mnasiadka> 17% of deployments on Stein? :)
15:08:16 <mnasiadka> and Queens
15:08:17 <mnasiadka> yay
15:08:21 <yoctozepto> yeas ;-)
15:08:30 <yoctozepto> many sad numbers there
15:09:05 <mnasiadka> 2% of people using Kolla-Kubernetes
15:09:08 <mnasiadka> good one
15:09:29 <yoctozepto> more masakari than blazar priteau
15:09:37 * yoctozepto feels the pressure
15:09:50 <yoctozepto> yeah, kolla-kubernetes kills the system
15:09:53 <mnasiadka> oh, TripleO only used in 10% deployments and K-A in 15%
15:09:59 <mnasiadka> that's interesting
15:10:33 <yoctozepto> yeah, I wonder what those using RHOSP chose
15:10:43 <yoctozepto> some might be remembering the good ol' days of puppet
15:10:59 <mnasiadka> Which OpenStack Network (Neutron) drivers are you using? - this is my readability winner
15:11:25 <yoctozepto> well, and pure "ansible" is interesting as the deployment choice
15:12:12 <yoctozepto> yeah, re neutron I have no idea who looked at this and thought: yeas, it's perfect to display ;D
15:12:41 <yoctozepto> only table makes sense for that
15:12:46 <yoctozepto> and sorted by pop
15:12:48 <yoctozepto> but meh
15:13:25 <yoctozepto> well, anyhow
15:13:30 <wuchunya_> ubuntu used 40%
15:13:32 <yoctozepto> in storage ceph is the king
15:14:08 <mnasiadka> nothing new
15:14:08 <yoctozepto> wuchunya_: yeah but it's for 2020 and centos+rhel = 47
15:14:13 <yoctozepto> but hard to tell if they overlap or not
15:14:15 <yoctozepto> perhaps they do
15:14:28 <yoctozepto> nowadays ubuntu would probably be getting a boost
15:14:41 <yoctozepto> many simply decided to migrate to ubuntu
15:15:15 <yoctozepto> for lolz: more windows than suse
15:15:39 <yoctozepto> all right, enough dwelling on stats
15:15:48 <yoctozepto> #topic Review action items from the last meeting
15:16:23 <yoctozepto> mgoddard email openstack-discuss about quay.io credentials
15:16:23 <yoctozepto> he did not
15:16:23 <yoctozepto> #action mgoddard email openstack-discuss about quay.io credentials
15:16:26 <yoctozepto> yoctozepto deprecate enable_host_ntp
15:16:31 <yoctozepto> he did not either
15:16:35 <yoctozepto> #action yoctozepto deprecate enable_host_ntp
15:16:57 <yoctozepto> #topic CI status
15:17:13 <yoctozepto> I was not angry at CI so it must be pretty green
15:17:23 <yoctozepto> I like quay so far
15:17:32 <yoctozepto> it only failed once since
15:17:46 <yoctozepto> move to Bullseye needs mariadb work
15:17:54 <yoctozepto> ^ that has happened already
15:17:55 <yoctozepto> removing
15:18:20 <mnasiadka> yep, we're waiting for hrw to test on aarch
15:18:45 <yoctozepto> nobody cares about monasca-grafana I guess?
15:19:19 <mnasiadka> seems so
15:19:35 <mnasiadka> are we deprecating in W or in X?
15:19:41 <yoctozepto> kayobe looks green too
15:19:54 <yoctozepto> mnasiadka: would ask dougsz
15:20:01 <yoctozepto> but I love to deprecate early
15:20:06 <yoctozepto> and drop early :-)
15:20:16 <yoctozepto> it's a monstrosity anyway
15:20:22 <mnasiadka> Will ask when I'll see him online
15:20:25 * yoctozepto feels bad recommending it
15:21:21 <mnasiadka> https://review.opendev.org/c/openstack/kolla/+/784901
15:21:27 <mnasiadka> we have this, seems there's consensus
15:21:47 <yoctozepto> mnasiadka: does your cephadm fix helps with that random issue mentioned on the whiteboard?
15:22:04 <mnasiadka> yoctozepto: well, I haven't seen it in there, but I'll do a couple of rechecks to be sure.
15:22:19 <yoctozepto> mnasiadka: thanks
15:22:25 <yoctozepto> let me check that monasca-grafana
15:23:30 <yoctozepto> +2
15:25:13 <yoctozepto> mnasiadka: you +2 too?
15:25:56 <mnasiadka> sure, I'll try to ask Doug if he needs help in K-A deprecation
15:26:03 <yoctozepto> #topic Review requests
15:26:07 <yoctozepto> mnasiadka: very well, thanks!
15:26:26 <yoctozepto> what do you want reviewed?
15:26:40 <yoctozepto> your favourite plant?
15:26:45 <yoctozepto> your favourite meal?
15:26:50 <headphoneJames> https://review.opendev.org/c/openstack/kolla-ansible/+/741340/
15:26:53 <yoctozepto> I'm here to grant you your wish
15:27:12 <yoctozepto> ah yes
15:27:15 <yoctozepto> +966
15:27:15 <yoctozepto> -18
15:27:25 <headphoneJames> I also enjoy orchids
15:27:29 <yoctozepto> and now I know why it is still not merged :-)
15:27:42 <yoctozepto> orchids can be pretty
15:28:33 <mnasiadka> headphoneJames: well, my main issue is with supervisord (so multiple processes) in haproxy image... can we do anything to not do it?
15:29:40 <headphoneJames> I'm open to other approaches.
15:30:07 <mnasiadka> I'll try to look into, but that might happen in two weeks time...
15:30:08 <headphoneJames> The issue is that we need a process that monitors for new certificates, and causes an haproxy reload
15:30:52 <mnasiadka> or we need a newer haproxy, which is a crazy thing to do - because we would need additional repos...
15:30:58 <headphoneJames> I have a suspicion this is not going in the current  release, so two weeks should be fine
15:31:05 <hrw> morning
15:31:11 <yoctozepto> you might be right, headphoneJames, I am sorry
15:31:19 <yoctozepto> I really wished to get this in this cycle :/
15:31:39 <mnasiadka> we could run a separate container with shared ipc namespace
15:31:42 <headphoneJames> s'all good
15:31:49 <mnasiadka> headphoneJames: thought about that?
15:32:22 <headphoneJames> I certainly have not. I would have to learn with that means ;)
15:32:38 <mnasiadka> headphoneJames: https://docs.docker.com/engine/reference/run/#ipc-settings---ipc - look at shareable
15:32:50 <mnasiadka> probably we could then restart haproxy from another container
15:33:13 <yoctozepto> we could send a signal
15:33:15 <priteau> Is there a less automated version that could be merged in wallaby, with a manual command to update certificates and reload haproxy?
15:33:18 <mnasiadka> but what happens when you restart haproxy and if it joins the same ipc group, that is a mystery :)
15:33:20 <yoctozepto> but that would need share pid namespace, no?
15:33:42 <mnasiadka> yoctozepto: but it doesn't need to be host namespace, it seems in Docker you can create a ,,shareable'' one
15:33:43 <yoctozepto> priteau: yeah, my thoughts exactly
15:33:51 <yoctozepto> I remember we discussed some layering to this
15:34:05 <yoctozepto> mnasiadka: I wonder what the lifetime is ;-)
15:34:10 <yoctozepto> as you need that PID 1
15:34:26 <yoctozepto> it would be fugly to have either break the other
15:34:37 <yoctozepto> perhaps it's better to push newer haproxy for this
15:34:41 <mnasiadka> yoctozepto: that's why I mentioned a mystery, meaning it would need some degree of testing
15:34:47 <yoctozepto> headphoneJames: do you have an answer for priteau?
15:34:50 <mnasiadka> yoctozepto: both centos and ubuntu would need a repo
15:35:05 <yoctozepto> even current ubuntu? doh
15:35:07 <mnasiadka> yoctozepto: which in centos case there's a nfv repo - but for Ubuntu it's just somebody's PPA
15:35:34 <yoctozepto> we need
15:35:35 <yoctozepto> 2.2?
15:36:05 <headphoneJames> We can definitely push a version that doesn't do the automatic restart, and would require customers to restart ha proxy manually
15:36:44 <openstackgerrit> Merged openstack/kolla master: centos: enable repos where needed  https://review.opendev.org/c/openstack/kolla/+/785871
15:36:45 <mnasiadka> yoctozepto: ideally 2.2
15:36:53 <headphoneJames> However, the test case would need to then manually restart haproxy
15:37:01 <yoctozepto> headphoneJames: let's do it in another change then
15:37:05 <yoctozepto> perhaps we can merge that
15:37:09 <yoctozepto> mnasiadka: meh
15:37:11 <mnasiadka> headphoneJames: well, how do we notify them to restart haproxy? :) and how often do they need to do that?
15:37:36 <headphoneJames> They would have to check logs :)
15:37:38 <mnasiadka> yoctozepto: yeah, so centos nfv repo and ubuntu PPA https://launchpad.net/~vbernat/+archive/ubuntu/haproxy-2.2 - no other option with Focal
15:38:11 <headphoneJames> Keep in mind supervisord is only deployed and used with the let's encrypt option enabled
15:38:30 <headphoneJames> And works really well over several months of testing
15:38:54 <mnasiadka> I don't doubt it does work, but we don't run more than one process in any container image :)
15:39:26 <headphoneJames> Anyhow, my expectations are properly set for next release
15:40:21 <yoctozepto> I made a few updates to https://ethercalc.openstack.org/kolla-infra-service-matrix
15:40:28 <yoctozepto> I assumed we move to bullseye
15:41:05 <yoctozepto> headphoneJames: how long will it take you to simplify it as we discussed?
15:42:26 <headphoneJames> I can easily remove the supervisord part.
15:42:50 <headphoneJames> I will look into the shared ipc today
15:43:11 <priteau> mnasiadka: don't look at the bifrost image then ;-)
15:43:29 <yoctozepto> priteau: bifrost is on the death path
15:43:41 <headphoneJames> Having customers manually restart haproxy is probably not the right approach
15:43:44 <yoctozepto> just no executioners around
15:43:59 <yoctozepto> headphoneJames: yes, it's not
15:44:22 <yoctozepto> but I have a feeling we are better off merging it in parts rather than postponing till the end of the world
15:45:29 <headphoneJames> Let me see how far I get today. I'm happy to discuss breaking it into smaller chunks
15:45:52 <yoctozepto> all right, ping us whenever
15:46:11 <yoctozepto> ooh, I have a last minute announcement
15:46:17 <yoctozepto> #topic Last minute announcements
15:46:31 <yoctozepto> #info OpenStack Wallaby released! yahoo!
15:46:44 <hrw> yay!
15:47:04 <mnasiadka> priteau: bifrost is a monster
15:47:05 <hrw> so when we branch?
15:47:12 <yoctozepto> that's the question for ptg
15:47:30 <yoctozepto> #topic Wallaby release planning
15:47:33 <mnasiadka> I think we can branch Kolla once we merge all hrw's patches ;) (UCA/Debian/RDO)
15:47:35 <yoctozepto> Debian bullseye progress
15:47:42 <yoctozepto> I think it's only pending a review
15:47:56 <yoctozepto> https://review.opendev.org/c/openstack/kolla/+/772479
15:48:05 <yoctozepto> I pasted it on the whiteboard already
15:48:08 <yoctozepto> anything to discuss?
15:49:30 <mnasiadka> well, since Debian is now on good path, I guess it's time to look into UCA/RDO changes to get them merged and images built
15:49:36 <hrww> I will read meeting log :(
15:49:50 <mnasiadka> I'll try to look into those
15:50:18 <hrww> UCA is upgrade V->W issue iirc
15:50:43 <mnasiadka> upgrade is always a release change issue :)
15:51:39 <mnasiadka> ah, looking for somebody brave enough to merge this: https://review.opendev.org/c/openstack/kolla-ansible/+/783669
15:53:17 * hrw drops - too many lags and reconnects today to be able to follow discussion
15:53:21 <hrw> upc--
15:53:35 <mnasiadka> yoctozepto: anything else left on agenda? because it seems I'm speaking with myself...
15:53:43 <yoctozepto> check the recorded logs hrw
15:54:02 <yoctozepto> mnasiadka: open discussion
15:54:13 <mnasiadka> ok then
15:54:15 <yoctozepto> #topic Xena PTG planning https://etherpad.opendev.org/p/kolla-xena-ptg
15:54:30 <yoctozepto> it was mentioned already
15:54:37 <yoctozepto> please add yourself and your discussion topics
15:55:46 <yoctozepto> #topic Open discussion
15:56:06 <yoctozepto> mnasiadka wanted to participate in open discussion
15:56:29 <mnasiadka> naah, I speak too much and mainly to myself, it doesn't sound like an open discussion ;)
15:56:38 <mnasiadka> and we will have too much time for discussions next week
15:57:46 <yoctozepto> mnasiadka: well, it's open-ended then!
15:57:53 <yoctozepto> but you are right
15:58:10 <yoctozepto> we are all tired simply by thinking how tired we will be after the ptg
15:58:16 <yoctozepto> that said
15:58:19 <yoctozepto> thank you for attending
15:58:28 <yoctozepto> #endmeeting