Wednesday, 2023-03-29

opendevreviewNeil Hanlon proposed openstack/openstack-ansible-plugins master: DNM: try forcing systemd-udev to install for CentOS Stream  https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/87880300:51
opendevreviewNeil Hanlon proposed openstack/openstack-ansible master: Add rockylinux-9 distro metal job to check pipeline  https://review.opendev.org/c/openstack/openstack-ansible/+/87811500:54
NeilHanlonoh.. this is a fun one... 🙃https://zuul.opendev.org/t/openstack/build/ed9ac39a6127407a8ecb5e0952ebb3cb00:57
NeilHanlonjrosser, noonedeadpunk: seeing CI failures on rocky and CentOS now due to.. something. there's a difference in the rocky packages from yesterday vs today. in particular, libibverbs, but, there's more than just that. https://paste.opendev.org/show/bUXUY3qlG3QULRixC5kU/01:15
jrosserNeilHanlon: `nothing provides libmlx5.so.1(MLX5_1.24)(64bit)` if i had to completely guess than than version number almost suggests that library originating from the OFED driver?05:31
jrosserwhich actually i think is not the case,  but the naming is pretty unusual05:33
hamidlotfiHi there,07:04
hamidlotfiI wanted to scale up my OSA environment with your guide published at `https://docs.openstack.org/openstack-ansible/latest/admin/scale-environment.html‍‍‍‍‍‍‍‍‍‍`07:04
hamidlotfiI want to add infra04 in my env as a controller07:04
hamidlotfiBut in the `setup-infrastructure.yml` show me this error:07:04
hamidlotfi`hostvars[groups[rabbitmq_host_group][0]]['ansible_facts']['hostname'] }}: 'dict object' has no attribute 'hostname'`07:04
gokhanisiGood morning folks, I need to install OSA on offline environment. I need your advises. for pypi mirror I am thinking to use python-pypi-mirror lib for mirroring and for pypi server using pypiserver. for distrubution packects use apt-mirror. Can you share your advises for offline OSA deployment? 07:21
jrosserhamidlotfi: it is possible that the documentation does not match with changes made in the code07:25
jrosserhamidlotfi: the error is that facts have not been gathered from the first rabbitmq host, and that is likley due to the --limit in step 307:26
jrosserhamidlotfi: is it step 4 that fails for you?07:27
jrossergokhanisi: i have previously used devpi as a caching proxy for pypi, so not completely offline but almost. I think that noonedeadpunk also has some experience with offline installs so might also have a suggestion for the python mirror07:28
hamidlotfi:jrosser I use this command `openstack-ansible playbooks/setup-infrastructure.yml --limit localhost,infra04,infra04-host_containers`07:31
hamidlotfi@jrosser 07:31
hamidlotfi@jrosser 07:32
jrosserhamidlotfi: i'm not sure thats going to work - step 4 says `openstack-ansible playbooks/setup-infrastructure.yml -e galera_force_bootstrap=true`07:35
jrosserand it also gives a specific reason for that, the rabbit and galera clusters need expanding to include the new node, so it's reasonable that data is required about all of the infra nodes during that process07:36
gokhanisithanks jrosser I am also waiting for noonedeadpunk suggestions :) 07:38
gokhanisijrosser, it seems you used like this https://gist.github.com/aschmu/4bdb40f5212eea6bf3ea698eeea2d60c07:41
hamidlotfi@jrosser It's this mean that I run this `openstack-ansible playbooks/setup-infrastructure.yml -e galera_force_bootstrap=true` plus a command like this for rabbit or just run this for both configurations?07:44
jrossergokhanisi: well we specifically didnt use bandersnatch as if you mirror the whole of pypi it is absolutely gigantic07:45
jrosserthats why we made a caching proxy instead, but like i said the environment was not totally isolated and the apt mirror and devpi servers had interfaces on inside and outside networks07:47
jrosserhamidlotfi: that command should deploy everything "infrastructure" onto your infra host, like repo server, haproxy, rabbitmq, galera etc07:48
jrossergokhanisi: if you want something that is absolutely isolated, like air-gap networks then you will need something thats is more like a true mirror07:49
gokhanisijrosser, yes it is absolutely isolated07:50
jrosserok sure07:51
hamidlotfi@jrosser  That command stoped with this error `mysqlcheck: Got error: 1102: Incorrect database name '#mysql50#tmp.J7AnZnVRKB' when selecting the database` 07:59
jrosserhamidlotfi: can you tell me which release of OSA you are using?08:00
hamidlotfiZED08:00
jrosserhmm ok08:00
noonedeadpunkgokhanisi: we're using pulp and aptly just in case08:24
noonedeadpunkno idea what's the best choice to be frank, wasn't doing any comparison, but these 2 worked fine 08:25
noonedeadpunkyeah, having ovs dependency on melanox stuff is just wrong08:45
noonedeadpunkI don't think it's related in any way to Rocky, as ovs is taken from another repo08:45
noonedeadpunkthat is very annoying though08:49
noonedeadpunkI kind of wonder if that might be tighten to rdo release 2023.1 or smth08:50
*** kleini_ is now known as kleini08:59
noonedeadpunkIt seems that's kind of the reason https://review.rdoproject.org/r/c/rdoinfo/+/4801409:02
noonedeadpunkAs I was quite sure that 3.1 was not the case before... And now it's backported down to W...09:04
jrossernoonedeadpunk: well tbh rdo/RH product cycle is totally decoupled from regular openstack releases isnt it?09:37
jrosserjust happens that the branches are the same name09:37
noonedeadpunkrhop is totally decoupled, but RDO is kind trailing thing09:38
noonedeadpunkbut I mean - it's not under governance, so...09:38
jrosserright - but like we see with tripleo they only care about very specific branches? not each release09:38
noonedeadpunkyeah, kind of decoupled09:38
noonedeadpunktripleo switched to independent release cycles since Wallaby09:39
jrosserso where that rdo commit says "OpenStack networking team has requested to backport this up to Wallaby." i have no idea how that is compatible with stable branch policy for wider opensack09:39
noonedeadpunkthe problem with tripleo is that we should not have allowed them to make same branch naming09:39
noonedeadpunkbut rdo does have all releases and they care about them kinda09:40
noonedeadpunkSo if you check this - https://www.rdoproject.org/rdo/release-cadence/ it's kinda aligned with openstack releases despite they're not uner governance09:40
jrosserso question really is if we as OSA can commit to supporting stable branches deployed with rdo packages when they might make huge backports like that09:43
noonedeadpunkI'm not sure we have better place to take ovs from?09:47
noonedeadpunkas this is not distro path I'm talking about09:47
noonedeadpunkhttps://zuul.opendev.org/t/openstack/build/f295470ef7fe4ad3ae25d5c985c3d304/log/job-output.txt#1895109:48
noonedeadpunkas native ovs provided in centos repos used to be too old to meet requirements 09:48
noonedeadpunkmaybe it's not the case anymore - don't know09:49
admin126.0.1  .. trying to add compute note .. get this -> https://paste.openstack.org/raw/bCuztg9kMITqiXNqzhH5/ 10:05
admin1i don't think i have seen this error before . 10:05
noonedeadpunkyeah, we have a bug for that already10:06
noonedeadpunkNot really sure how to workaround this properly. I truly believe that it's result of things being changed in Z10:06
noonedeadpunkin ansible10:06
admin1i ran the nova playbook without limit as well ( so that facts are refreshed ) .. stuck in the same 10:15
admin1i can delelete ansible_facts and try again 10:15
admin1if its some cache thing . 10:15
noonedeadpunkSo, you need to gather facts for both computes and api/scheduler10:18
noonedeadpunkAnd that might be also troublesome if you have some computes down10:18
admin1if i delete ansible_facts, and re-run setup-hosts and setup infra with no limits, might it fix ? 10:25
admin1i do have some hosts that are down and we are waiting for replacement parts 10:25
* jrosser still thinks we should add `-e adding_compute_node=true`10:25
* admin1 wants (start) as well .. 10:26
admin1(star) 10:26
admin1ignore my last 2 lines .. was testing the /me 10:26
noonedeadpunkNeilHanlon: so... with rdo folks upating ovs/ovn, now rocky misses libmlx5.so.1. They claim that package comes from CentOS Base at the moment and it's required due to mlx now is in enabled drivers by default https://git.centos.org/rpms/openvswitch/blob/c9s-sig-nfv-openvswitch-3.1/f/SPECS/openvswitch3.1.spec#_30910:28
noonedeadpunkCentOS is broken differently though then Rocky - it fails on tempest with https://git.centos.org/rpms/openvswitch/blob/c9s-sig-nfv-openvswitch-3.1/f/SPECS/openvswitch3.1.spec#_30910:29
noonedeadpunk*https://git.centos.org/rpms/openvswitch/blob/c9s-sig-nfv-openvswitch-3.1/f/SPECS/openvswitch3.1.spec#_30910:29
noonedeadpunk(if forget about udev issue for a second)10:29
noonedeadpunkWhich goes down to https://zuul.opendev.org/t/openstack/build/64d8812538584e4b9d2b95e4c0907299/log/logs/host/ovn/ovn-northd.log.txt#3210:36
noonedeadpunkwe also have now couple of ssl issues logged here https://zuul.opendev.org/t/openstack/build/64d8812538584e4b9d2b95e4c0907299/log/logs/host/ovn/ovsdb-server-nb.log.txt10:38
noonedeadpunkjamesdenton: we might need some your help here to be frank...10:40
jamesdentonhi noonedeadpunk - i will try and check that out this moprnming10:45
jamesdenton*morning10:45
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-role-systemd_mount master: Ensure systemd-udev is installed for gluster  https://review.opendev.org/c/openstack/ansible-role-systemd_mount/+/87885110:51
noonedeadpunkso they've basically upgraded ovs/ovn from 2.17/22.06 to 3.1/22.1210:52
jamesdentonok10:52
noonedeadpunkssl probably not an issue though. It's that aio is not registered somewhat, despite we obviously running command to set hostname11:03
jamesdentonat one time there was a command to set the hostname within ovs, but i believe at some point that became problematic, not sure if it's still there11:04
jamesdentonI'll see if i can stand up a Rocky AIO and tshoot the OVS issue. The MLX library requirement is likely due to that being the DPDK build of OVS11:05
noonedeadpunkI will check that, but at least we're running this thing https://opendev.org/openstack/openstack-ansible-os_neutron/src/branch/master/tasks/providers/setup_ovs_ovn.yml#L1811:05
jamesdentonahh yes, that's the one.11:06
noonedeadpunkI'm spawning aio for centos metal already11:06
NeilHanlonif we install rdma-core we should be okay, provided the soname is still provided 11:06
noonedeadpunkrdma-core is... a repo or a package?11:07
NeilHanlonrdma-core provides libibverbs, which provides the mlx lib11:07
NeilHanlonpackage 11:07
noonedeadpunkaha, ok11:07
jamesdentonugly ugly. :|11:07
noonedeadpunkyeah, that sucks for sure11:07
noonedeadpunkdamn, I pasted same link 3 times :(11:08
jamesdentonnoonedeadpunk at one point, the hostname was never set and that had to be manually do. I believe with newer OVS it is done automatically, so we *might* be able to do without that command, now. Can try and patch it out and see11:08
jamesdentonneed to get the kids off to school, but will be back in a bit11:09
noonedeadpunkyeah, sure, no huge rush11:09
noonedeadpunkthe error neutron fails to bind the port is `Refusing to bind port c6249ef3-801f-4615-8630-5dcaf2a7c3ee due to no OVN chassis for host: aio1` https://zuul.opendev.org/t/openstack/build/64d8812538584e4b9d2b95e4c0907299/log/logs/host/neutron-server.service.journal-02-32-13.log.txt#654511:10
noonedeadpunkSo it kinda feels that hostname is not set or set wrongly, but I'm pretty sure it should be fine...11:11
noonedeadpunkjamesdenton: you're right - ovs now auto-adds hostname, but it does that incorrectly11:29
noonedeadpunkhttps://zuul.opendev.org/t/openstack/build/64d8812538584e4b9d2b95e4c0907299/log/logs/etc/host/openvswitch/conf.db.txt#2011:31
noonedeadpunk(and L8)11:31
noonedeadpunkOh! And then L6411:32
noonedeadpunkwould be great to have a way to disable that...11:36
jrosseroh! is this my fault? i recently added that aio1.openstack.local fqdn to /etc/hosts in the aio11:52
admin1noonedeadpunk, do you know of any way ( even manual or hacky) that will allow me to add that new compute node ?11:53
noonedeadpunkI think it's ovs 3+ thingy11:53
jrosseradmin1: to work around that compute node add issue we allowed the playbook to run without --limit until it had gathered facts11:54
jrosserthen we CTRL-C11:54
noonedeadpunkadmin1: eventually it fails _after_ compute is deployed11:54
jrosserand run it again with the limit, but in our case we did not have any nodes that were offline11:54
jrosserso i think that this is still going to be troublesome for you11:54
noonedeadpunkWe already have a bug report coming https://bugs.launchpad.net/openstack-ansible/+bug/201318912:01
noonedeadpunkSo... ovs solutions likely are either to set `RECORD_HOSTNAME=no` or `FULL_HOSTNAME=no` and drop our task for setting it12:43
noonedeadpunkand it's `/usr/share/openvswitch/scripts/ovs-ctl` that messes things around12:47
noonedeadpunkhttps://github.com/openvswitch/ovs/blob/master/utilities/ovs-ctl.in#L42-L5212:47
noonedeadpunkdoes anybody has ubuntu aio with ovn to check smth?12:58
damiandabrowskinoonedeadpunk: i can13:11
noonedeadpunkcan you place my key there then ?:)13:11
damiandabrowskisure, one sec13:11
jrosserdoes anyone have a moment to look at https://lists.openstack.org/pipermail/openstack-discuss/2023-March/033008.html13:29
jrosserits kind of an OSA question not designate13:29
noonedeadpunkit slipped me due to the tag...13:30
admin1noonedeadpunk, i have multi deplpoyment 13:35
admin1osa + ovn 13:35
admin11 controller + 2 compute ( also as network nodes ) 13:36
admin1testing k8s in that 13:36
admin1k8s + octavia 13:36
admin1k8s + octavia on top of osa+ovn :D13:37
noonedeadpunkthat's already too hard for me :D13:44
*** spotz__ is now known as spotz_13:51
noonedeadpunkjamesdenton: NeilHanlon so just to update you a bit what we've found with RDO folks regarding OVS and their update - that openvswitch 3.1 has broken down `ovs-vsctl add` command. As it intends to adding things rather then replacing14:00
noonedeadpunkSo it's regression of https://github.com/openvswitch/ovs/commit/2221e8b034298bd77dc9282895c4b32fd11583e8 basically14:01
noonedeadpunkor well. not exactly, but idea is that `add` should not override things, while it obviously does for 3.1. It's not for 3.0.1 which ubuntu is running and worked properly in 2.1714:03
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Workaround ovs bug that resets hostname with add command  https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/87891114:24
noonedeadpunkI feel we're in a horrible dependency loop as we have 3 separate issues in rhel distros that arised for the last couple of days14:26
noonedeadpunknot counting rabbit on stable branches...14:31
* NeilHanlon sighs14:55
* NeilHanlon wonders if it's acceptable to crack a beer at 11am14:55
noonedeadpunkYeah... I'm was thinking the same but wrt more strong drinks...14:56
NeilHanlonfor the rocky problem missing libibverbs (mlx SO) -- something "feels" wrong about that since a CI job from a day ago is pulling in this package, and now all of a sudden.. not 14:58
NeilHanlonI wonder if some previous package from centos sig-nfv  had an explicit `Requires` that is no longer there15:00
noonedeadpunkday ago it was ovs2.17, now it's 3.115:00
noonedeadpunkI have centos spawned and now trying to spawn rocky to check15:00
NeilHanloni'm rebuilding my AIO as well...15:00
noonedeadpunkwell, yeah, libibverbs-44.0-2 is being installed from base15:02
jrosserthis feels like the sort of trouble you get when packages are not build in clean environments15:03
* jrosser thinks back to days with pbuilder for deb packages15:03
NeilHanlonstream is _just_ far enough ahead that we run into these sorts of things :( 15:32
jrosseri guess that any degree of parallelism in the builds of different packages is a complete minefield as well15:34
NeilHanloni'll write up a _something_ and pass it over to Amy to see what we can do overall. i can imagine this won't be the last time something like this happens, so it'd be good to have more/better communication coverage over these impactful sorts of things due to the delta between CentOS Stream and RHEL15:39
noonedeadpunkI'd still propose quick dirty fix to install old ovs until this solved so we could unblock gates15:52
noonedeadpunkAs discussion sounded to me like not this week fix for sure15:52
opendevreviewNeil Hanlon proposed openstack/openstack-ansible-os_neutron master: Pin openvswitch package on RHEL-likes to 2.17  https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/87892415:53
NeilHanlonnoonedeadpunk: ^15:54
noonedeadpunkNeilHanlon: this is not working jsut in case ^15:54
noonedeadpunkI had to add `exclude = *rdo-openvswitch*3.1*,*rdo-ovn*3.1*` to avoid this from being isntalled15:55
NeilHanlonah :( 15:55
* NeilHanlon sighs15:55
noonedeadpunkhttps://paste.openstack.org/show/bGFidAmbzq9Uom2Buze8/15:55
noonedeadpunkThat is annoying, I know15:56
noonedeadpunkI think we're adding repo in openstack_hosts repo15:56
noonedeadpunkI will push a patch now15:57
NeilHanlonwe could maybe pin that rdo-openvswitch pkg instead?15:58
noonedeadpunkUm, might be but I struggled with that a bit16:02
NeilHanlonno prob. exclude is a good/better option16:05
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-openstack_hosts master: Pin openvswitch package on Rocky to 2.17  https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/87892616:06
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-openstack_hosts master: Pin openvswitch package on Rocky to 2.17  https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/87892616:06
noonedeadpunkI;m just not sure if exclude for everything or jsut rocky16:07
noonedeadpunkAs we're having serious dependency loop now16:07
NeilHanlonstream doesn't need it16:07
NeilHanlonafaik16:07
noonedeadpunkAs we need to merge 3 patches in 3 repos now to fix all breakages16:07
noonedeadpunkand backport all that16:08
NeilHanlonright 16:11
NeilHanlonIMO we only need the openvswitch change for rocky tests; then we need the hostname change only for stream, i guess? 16:12
NeilHanlonOr if we do the openvswitch change for both for now, maybe we don't need that hostname patch16:12
noonedeadpunkYeah, I kind of thinking what would be easier16:15
noonedeadpunkBut we also have this things as well https://review.opendev.org/c/openstack/ansible-role-systemd_mount/+/878851 that breaks stream16:16
noonedeadpunkmaybe indeed we can merge using old ovs repo easy enough16:17
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-openstack_hosts master: Pin openvswitch package on RHEL to 2.17  https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/87892616:18
opendevreviewNeil Hanlon proposed openstack/openstack-ansible-plugins master: Ensure systemd-udev is installed for gluster  https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/87880316:22
NeilHanlonremoved the DNM from that one just in case..16:23
noonedeadpunkare you sure it's needed?16:24
noonedeadpunkAs it looks to be in place? https://opendev.org/openstack/openstack-ansible-plugins/src/branch/master/roles/glusterfs/vars/redhat.yml#L2216:24
noonedeadpunkAh...16:25
noonedeadpunkI think we're having ordering issue16:25
noonedeadpunkSo L60 should be kinda moved to L4416:25
damiandabrowskijrosser: can you please explain why did you include haproxy_base_service in horizon_haproxy_services?16:26
damiandabrowskihttps://opendev.org/openstack/openstack-ansible/src/commit/7f358cd906d27582da8affb11de4d7b1c3b7465a/inventory/group_vars/horizon_all.yml#L4916:26
damiandabrowskiit's not working properly with separated haproxy config because horizon_all group does not have an access to haproxy_base_service var.16:26
damiandabrowskiBut horizon works fine on my AIO without that.*16:27
damiandabrowski* after fixing this bug: https://opendev.org/openstack/openstack-ansible/src/commit/4195a2937c0281b49bdad9633fb4b237d58fc77e/inventory/group_vars/haproxy/haproxy.yml#L26216:27
damiandabrowskiit should be '.* horizon-back' instead of '.* horizon'16:27
noonedeadpunkor well... should it...16:27
opendevreviewDmitriy Rabotyagov proposed openstack/ansible-role-systemd_mount master: Ensure systemd-udev is installed for gluster  https://review.opendev.org/c/openstack/ansible-role-systemd_mount/+/87885116:28
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Workaround ovs bug that resets hostname with add command  https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/87891116:30
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Workaround ovs bug that resets hostname with add command  https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/87891116:31
noonedeadpunkdoh16:37
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-plugins master: Ensure systemd-udev is installed for gluster  https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/87880316:41
noonedeadpunkI think I'm a bit lost now in all dependencies we need...16:43
jrosserdamiandabrowski: doesnt that patch come from before the work with haproxy maps was done?16:43
jrosserdamiandabrowski: remember that the step before doing maps was moving to 'default_backend' for horizon on the base service16:44
jrosserso i think that now that the maps work is done, that line is no longer needed16:47
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-openstack_hosts master: Pin openvswitch package on RHEL to 2.17  https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/87892616:47
damiandabrowskithe 'Prepare main repo for separated haproxy config' patch? yes, be AFAIK we agreed to rebase it on top of your haproxy maps changes16:48
noonedeadpunkhuh, whaat, zuul now re-schedules automatically if you update dependant patch?16:49
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Workaround ovs bug that resets hostname with add command  https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/87891116:51
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-openstack_hosts master: Limit usage of old OVS to Rocky only  https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/87892916:51
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-openstack_hosts master: Limit usage of old OVS to Rocky only  https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/87892916:54
noonedeadpunkso we kinda have this topic to track patches https://review.opendev.org/q/topic:osa%252Frdo_ovs_3.116:56
noonedeadpunkSe we will have troubles with plugins repo patching at very least, as we in fact don't install collection version we're currently testing17:48
noonedeadpunkI think this is due to setting git+file here - https://opendev.org/openstack/openstack-ansible/src/branch/master/scripts/get-ansible-collection-requirements.yml#L6317:48
noonedeadpunkI think worth trying to switch type there to `dir`17:48
noonedeadpunkso we'll likely need to disable centos jobs for plugins repo until ovs patches land17:49
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-plugins master: Ensure systemd-udev is installed for gluster  https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/87880317:51
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-plugins master: Restore CentOS jobs  https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/87893317:53
opendevreviewDamian Dąbrowski proposed openstack/openstack-ansible master: Use a map file to select haproxy horizon backend from the base frontend  https://review.opendev.org/c/openstack/openstack-ansible/+/87685119:50
opendevreviewDamian Dąbrowski proposed openstack/openstack-ansible master: Enable TLS frontend for repo_server by default  https://review.opendev.org/c/openstack/openstack-ansible/+/87642619:50
opendevreviewDamian Dąbrowski proposed openstack/openstack-ansible master: Implement separated haproxy service config  https://review.opendev.org/c/openstack/openstack-ansible/+/87118919:50
opendevreviewDamian Dąbrowski proposed openstack/openstack-ansible master: Update letsencrypt docs  https://review.opendev.org/c/openstack/openstack-ansible/+/87893919:50

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!