Thursday, 2023-06-29

mnasiadkamorning06:21
mnasiadkahrw: https://review.opendev.org/c/openstack/kolla/+/887223 - ubuntu/aarch64 fix06:21
hrw+206:34
fricklermnasiadka: any idea what's up with ovn? https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_9f0/887225/1/check/kolla-ansible-ubuntu-ovn/9f03229/primary/logs/container_logs/neutron_ovn_agent.txt06:46
fricklerguess I can ask neutron ppl if that rings a bell06:46
mnasiadkafrickler: I think there were some changes in neutron-ovn-agent - we should have a look what is required now for using it, it seems base_mac is06:49
mnasiadkabut maybe it's a bug06:52
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: ansible: bump min version to 2.14 and max to 2.15  https://review.opendev.org/c/openstack/kolla-ansible/+/88722706:54
fricklerhttps://review.opendev.org/c/openstack/neutron/+/878565 looks related and matches timing wise, will move to #-neutron06:56
mnasiadkathanks06:59
*** ralonsoh_ is now known as ralonsoh07:11
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Add support for LetsEncrypt-managed certs  https://review.opendev.org/c/openstack/kolla-ansible/+/74134007:45
kevkomnasiadka: why your patch single external frontend has public port listen port and other friends for keystone service little bit different  as others ? 07:46
kevkoit looks like it is somehow mixed because keystone public has a public string in variable ..so maybe you mixed ? 07:47
kevkoconfused maybe ? 07:47
kevkomnasiadka but i like it ..it's quite simple ..and it can be very usable with letsencrypt ...because then you can fetch list of domains and ask for cert renewal 07:48
mnasiadkaI don't remember :)07:49
mnasiadkahttps://review.opendev.org/c/openstack/kolla/+/886964 - another easy backport that passed07:49
SvenKieskethe pypi mirror issues should hopefully be gone now: https://github.com/pypi/infra/pull/14307:54
opendevreviewMerged openstack/kolla master: rabbitmq: Fix repo for ubuntu aarch64  https://review.opendev.org/c/openstack/kolla/+/88722307:58
opendevreviewMichal Nasiadka proposed openstack/kolla stable/2023.1: rabbitmq: Fix repo for ubuntu aarch64  https://review.opendev.org/c/openstack/kolla/+/88717708:04
opendevreviewMichal Nasiadka proposed openstack/kolla stable/zed: rabbitmq: Fix repo for ubuntu aarch64  https://review.opendev.org/c/openstack/kolla/+/88717808:04
mnasiadkathose aarch64 NODE_FAILUREs are not helping...08:37
mnasiadkaSvenKieske: seems it did not help, or ORD RAX pypi cache needs some manual operation like dropping the cache contents08:46
SvenKieskemhm, can you ping Clark on #opendev? he was very helpful yesterday and is familiar with this issue08:48
SvenKieskeor maybe pypi has even more defunct mirrors08:49
SvenKieskesad that we don't have the capacity to just mirror pypi completely08:49
opendevreviewMichal Nasiadka proposed openstack/kolla master: rabbitmq: bump version to 3.12  https://review.opendev.org/c/openstack/kolla/+/88722508:56
mnasiadkaSvenKieske: let's wait a couple more job runs, maybe that cache invalidates itself09:10
mnasiadkaSvenKieske: and Clark is probably still sleeping ;-)09:10
SvenKiesketrue09:21
SvenKieskethinking about it, of course our internal cache will still have the wrong things cached09:22
frickleralso not obvious whether pypi already deployed that change09:28
opendevreviewVerification of a change to openstack/kolla stable/2023.1 failed: Use erlang-25 from copr on aarch64  https://review.opendev.org/c/openstack/kolla/+/88696409:33
opendevreviewRobin Klostermeyer proposed openstack/kolla-ansible master: Add ironic-prometheus-exporter  https://review.opendev.org/c/openstack/kolla-ansible/+/87438009:56
opendevreviewRobin Klostermeyer proposed openstack/kolla master: Add ironic-prometheus-exporter  https://review.opendev.org/c/openstack/kolla/+/87441510:00
opendevreviewRobin Klostermeyer proposed openstack/kolla master: Add ironic-prometheus-exporter  https://review.opendev.org/c/openstack/kolla/+/87441510:12
opendevreviewRobin Klostermeyer proposed openstack/kolla-ansible master: Add ironic-prometheus-exporter  https://review.opendev.org/c/openstack/kolla-ansible/+/87438010:14
opendevreviewMerged openstack/kolla master: mariadb: Bump to current LTS (10.11)  https://review.opendev.org/c/openstack/kolla/+/88292410:37
opendevreviewMerged openstack/kolla stable/yoga: Update support matrix  https://review.opendev.org/c/openstack/kolla/+/87534110:37
opendevreviewVerification of a change to openstack/kolla stable/2023.1 failed: Use erlang-25 from copr on aarch64  https://review.opendev.org/c/openstack/kolla/+/88696411:00
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Add support for LetsEncrypt-managed certs  https://review.opendev.org/c/openstack/kolla-ansible/+/74134011:27
kevkofolks, how did you fix this ModuleNotFoundError: No module named 'tox.config' ? :D 11:51
fricklerENOCONTEXT12:20
SvenKieskekevko: what frickler is trying to convey is that we don't have any clue what you're talking about? :D12:25
kevkotox -e releasenotes 12:26
kevkotox -e py3912:26
kevkoetc ...12:26
kevkotox installed in venv 12:27
kevkopython3 -m venv /tmp/ihatetox ; . /tmp/ihatetox/bin/activate ; pip install tox ; tox -e releasenotes in cwd of some project ...for example kolla-ansible :)12:27
hrwkevko: 'mkvirtualenv kolla;workon kolla'12:31
kevkohrw: hmmmmmmm, i don't know why but i had tox executable in /usr/local/bin :D 12:35
kevkoremoved and installed again from pip12:36
kevkobtw, why we have in kolla-ansible  something like this 12:38
kevkokolla_base_distro in ['ubuntu', 'debian'] else 'vitrage-api.conf'12:38
kevkoi know what it is for ... but now i am wondering that it is ugly :/12:38
kevkoshouldn't we build images which are same ...12:39
kevkoi mean ..for example this one 12:39
kevko{% set wsgi_conf_dir = 'apache2/conf-enabled' if kolla_base_distro in ['ubuntu', 'debian'] else 'httpd/conf.d' %}12:39
kevkoi think better way is to build same image ... same path ...doensn't matter if rocky ubuntu debian ..12:40
hrwkevko: wsgi depends on how apache is handled by distro. RHEL ones do it different than Debian ones12:40
kevkoi know 12:40
hrwthat vitrage-api.conf is probably some kind of leftover of binary images12:40
kevkowhy don't we unite it?12:41
hrwgo for it12:41
kevko:D12:41
hrwI have to find out how to disable GIC ITS in IORT if GIC_ITS SMC call does not exists12:41
kevkopoint is that for example now i am debugging rocky/ubuntu/debian letsencrypt upstream patchset12:41
kevkoso, i am not going to run kolla-ansible everytime 12:42
kevkoi just kill one docker container ...and run another container ..12:42
kevkobut scripts are mounted from kolla-ansible 12:42
kevkoand kolla-ansible is rendering scripts based on kolla_base_distro 12:42
kevkoso ..i need to rewrite scripts :/12:43
kevkoanother thing is that you can have situation when you want to run some service on debian/ubuntu and some on rocky 12:43
kevkoagain ... you need to comment kolla_base_distro in globals and define it in inventory for every host ...12:44
hrwkevko: I do not remember when last time I used kolla-ansible12:44
kevko:D12:44
kevkobut you probably got it 12:44
hrwyep12:44
hrwC sucks. compared to Python12:46
kevkowell, most of the time it's just configuration paths 12:46
kevkosymlink is my friend :)12:47
mnasiadkahrw: rmq 3.12 seems working - https://review.opendev.org/c/openstack/kolla/+/88722512:50
hrwmnasiadka: good12:51
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Add support for LetsEncrypt-managed certs  https://review.opendev.org/c/openstack/kolla-ansible/+/74134012:51
mnasiadkahrw: mariadb also bumped today, so I believe we're good regarding Bookworm12:51
hrwmnasiadka: is there erlang 26 for bookworm?12:52
mnasiadkawell, we always installed Ubuntu packages on Debian, right?12:52
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Add support for LetsEncrypt-managed certs  https://review.opendev.org/c/openstack/kolla-ansible/+/74134012:52
hrwmnasiadka: just asking ;D12:52
mnasiadkabullseye uses focal, so I guess bookworm should be good with jammy?12:52
hrwmnasiadka: distro version is less relevant. openssl matters.12:53
mnasiadkahrw: Don't get me started, I love when RMQ guys create mayhem and then tell that it's OpenStack's fault12:53
hrwjammy was 1.1 still iirc12:53
mnasiadkahrw: well then RMQ team is not going to build it for you12:53
hrw3>12:54
hrwor was it 3<== rather?12:55
mnasiadkaI don't know and I don't want to know - tomorrow I close my laptop and forget about typing on keyboard for three weeks :)12:56
hrwmnasiadka: if things go right then I do similar for 2 weeks12:57
hrwkevko: have you though about splitting that LE patch? 180 revision beats everything I had to deal with13:01
hrw(in revision numbers, not size)13:02
mnasiadka+1 for splitting, question if that's not too much work right now :D13:03
mnasiadkaSvenKieske, frickler: looking at the gate going green I think pypi started to get better13:06
SvenKieske:)13:09
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: CI: drop ceph jobs ulimit config for EL9  https://review.opendev.org/c/openstack/kolla-ansible/+/88448813:30
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: rabbitmq: ensure Ranch ETS tables are populated before draining node  https://review.opendev.org/c/openstack/kolla-ansible/+/88475913:31
opendevreviewMichal Arbet proposed openstack/kolla-ansible master: Add support for LetsEncrypt-managed certs  https://review.opendev.org/c/openstack/kolla-ansible/+/74134013:54
opendevreviewMerged openstack/kolla master: base: install adduser on Debian/Ubuntu  https://review.opendev.org/c/openstack/kolla/+/88194814:11
opendevreviewMerged openstack/kolla master: base: Debian 12 uses new APT sources file  https://review.opendev.org/c/openstack/kolla/+/88195014:11
hrwnice14:11
SvenKieskemhm, does anyone know something about the source of this? https://quay.io/repository/openstack.kolla/ubuntu-source-rabbitmq/ ? this is a 404 for me, but apparently it worked some time ago?14:11
opendevreviewMarcin Juszkiewicz proposed openstack/kolla master: Move to Debian 12 'bookworm'  https://review.opendev.org/c/openstack/kolla/+/88608814:13
SvenKieskee.g. https://quay.io/repository/openstack.kolla/kolla-toolbox still exists.. the webinterface for quay.io is not helpful at all, it's not possible to directly reference an org apparently?14:13
hrwSvenKieske: we dropped "-source" from name14:13
SvenKieskeare these images officially built from us?14:14
hrwiirc, yes14:14
hrwhttps://quay.io/repository/openstack.kolla/ubuntu-source-rabbitmq/ -> https://quay.io/repository/openstack.kolla/ubuntu-rabbitmq/ now14:14
SvenKieskeah right14:14
SvenKieskelol, now quay.io greets me with " Your user session has expired. Please sign in to continue. " -> I have no account; I don't want an account; I was never signed in, so my session can't expire..14:15
hrw;D14:25
SvenKieskeI guess I tried to many permutations of invalid repo names and some IDS flagged me14:34
opendevreviewVerification of a change to openstack/kolla stable/2023.1 failed: Use erlang-25 from copr on aarch64  https://review.opendev.org/c/openstack/kolla/+/88696414:36
fricklerthe repo is https://quay.io/repository/openstack.kolla/rabbitmq , distros are just tags14:38
fricklerwe might want to clean up some old tags like master-ubuntu-focal14:41
mnasiadkaI have some scripts for that, was thinking of moving that to kolla repo and maybe adding support to publish job to maintain those tags15:11
mnasiadkafrickler: summarised CI job failures in a comment here: https://review.opendev.org/c/openstack/kolla-ansible/+/88448815:30
fricklermnasiadka: ach, thx15:36
opendevreviewMichal Nasiadka proposed openstack/kolla-ansible master: zun: Fix capsule functionality and tests  https://review.opendev.org/c/openstack/kolla-ansible/+/88729515:37
mnasiadkathat might fix the zun failures15:37
opendevreviewVerification of a change to openstack/kolla stable/2023.1 failed: Use erlang-25 from copr on aarch64  https://review.opendev.org/c/openstack/kolla/+/88696416:29
opendevreviewMerged openstack/kolla stable/zed: rabbitmq: Fix repo for ubuntu aarch64  https://review.opendev.org/c/openstack/kolla/+/88717816:48
opendevreviewMerged openstack/kolla stable/zed: Use erlang-25 from copr on aarch64  https://review.opendev.org/c/openstack/kolla/+/88696516:48
mnasiadkafrickler: https://review.opendev.org/c/openstack/kolla-ansible/+/887295 - seems it fixes zun :)18:01
opendevreviewVerification of a change to openstack/kolla stable/2023.1 failed: Use erlang-25 from copr on aarch64  https://review.opendev.org/c/openstack/kolla/+/88696418:17
guesswhat[m]kevko: ping, got the oidc integration working with keycloak21:38
guesswhat[m]found a bug there, this one https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/horizon/templates/local_settings.j2#L227 should be probably WEBSSO_KEYSTONE_URL = "{{ keystone_public_url }}/v3"21:39
*** _nick is now known as yankcrime21:45
guesswhat[m]otherwise redirects to apache mod_oidc wont work, see https://github.com/openstack/kolla-ansible/blob/master/ansible/roles/keystone/templates/wsgi-keystone.conf.j2#L9721:50
guesswhat[m]also skyline with oidc is working, see skyline.yaml, https://pastebin.com/raw/2ehpgCf1 , seems that it does not have concept of public and private endpoints, so keycloak_url has to be same as WEBSSO_KEYSTONE_URL for horizon22:02
opendevreviewRobin Klostermeyer proposed openstack/kolla-ansible master: Add ironic-prometheus-exporter  https://review.opendev.org/c/openstack/kolla-ansible/+/87438022:36
opendevreviewMichal Arbet proposed openstack/kolla master: review letsencrypt  https://review.opendev.org/c/openstack/kolla/+/88734723:30

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!