Friday, 2020-07-03

*** KeithMnemonic has quit IRC00:30
*** cshen has joined #openstack-ansible00:31
*** cshen has quit IRC00:36
*** gary_perkins has quit IRC01:02
*** gary_perkins has joined #openstack-ansible01:04
*** markvoelker has joined #openstack-ansible01:38
*** markvoelker has quit IRC01:43
*** schwicht has quit IRC02:09
*** cshen has joined #openstack-ansible02:31
*** cshen has quit IRC02:35
*** rh-jlabarre has quit IRC03:15
*** markvoelker has joined #openstack-ansible03:39
*** markvoelker has quit IRC03:43
*** viks____ has joined #openstack-ansible04:25
*** evrardjp has quit IRC04:33
*** evrardjp has joined #openstack-ansible04:33
*** udesale has joined #openstack-ansible04:39
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible master: Add Centos-8 support  https://review.opendev.org/68962904:56
noonedeadpunkbtw, I think now we can wait with https://review.opendev.org/#/c/738689/ and add centos 8 with focal04:58
*** akahat|pto is now known as akahat05:10
*** markvoelker has joined #openstack-ansible05:23
*** markvoelker has quit IRC05:27
*** miloa has joined #openstack-ansible06:49
CeeMacmorning06:55
*** luksky has joined #openstack-ansible06:58
*** markvoelker has joined #openstack-ansible07:18
*** markvoelker has quit IRC07:23
*** tosky has joined #openstack-ansible07:26
openstackgerritMerged openstack/openstack-ansible master: Fix mysql binary name  https://review.opendev.org/73541807:26
*** cshen has joined #openstack-ansible07:34
*** ysastri has joined #openstack-ansible07:35
ysastriHello we are planning to upgrade our Openstack-ansible cluster from Rocky to Stein. All our environments are based on ubuntu 16.04. I know for stein ubuntu 18.04 is required. While doing this OS upgrade from 16.04 to 18.04 what all things we need to keep in mind?? We are evaluating pre tasks for Stein major upgrade. Any pointers in this direction07:56
ysastriis much appreciated.07:56
*** miloa has quit IRC08:07
*** tosky has quit IRC08:32
arkanhi guys, I've done a fresh openstack installation, but I'm getting SSLError in octavia container09:01
arkanJul 03 09:01:53 controller1-octavia-server-container-d8bd0abc octavia-housekeeping[12716]: 2020-07-03 09:01:53.380 12716 WARNING octavia.amphorae.drivers.haproxy.rest_api_driver [-] Could not connect to instance. Retrying.: requests.exceptions.SSLError: HTTPSConnectionPool(host='192.168.210.67', port=9443): Max retries exceeded with url: // (Caused by SSLError(SSLError("bad handshake: Error([('rsa routines', 'RSA_padding_check_PKCS1_typ09:02
arkane_1', 'invalid padding'), ('rsa routines', 'rsa_ossl_public_decrypt', 'padding check failed'), ('asn1 encoding routines', 'ASN1_item_verify', 'EVP lib'), ('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))09:02
*** sshnaidm|afk is now known as sshnaidm|off09:15
*** markvoelker has joined #openstack-ansible09:19
*** tosky has joined #openstack-ansible09:21
*** markvoelker has quit IRC09:24
*** markvoelker has joined #openstack-ansible09:33
*** markvoelker has quit IRC09:38
*** cshen has quit IRC09:51
*** also_stingrayza has joined #openstack-ansible10:03
*** jbadiapa has quit IRC10:04
*** stingrayza has quit IRC10:07
*** cshen has joined #openstack-ansible10:23
*** jbadiapa has joined #openstack-ansible10:26
*** cshen has quit IRC10:28
openstackgerritMerged openstack/openstack-ansible master: Rename mysql to mariadb for registered var  https://review.opendev.org/73726310:37
*** cshen has joined #openstack-ansible10:38
*** shyamb has joined #openstack-ansible10:41
*** sshnaidm_ has joined #openstack-ansible10:47
*** sshnaidm|off has quit IRC10:48
openstackgerritMerged openstack/openstack-ansible master: Bump SHAs for master  https://review.opendev.org/73835810:53
*** ysastri has quit IRC11:03
*** udesale_ has joined #openstack-ansible11:09
*** udesale has quit IRC11:11
*** shyamb has quit IRC11:41
noonedeadpunkysastri we have some notes here https://etherpad.openstack.org/p/osa-rocky-bionic-upgrade12:17
noonedeadpunkand also some doc is almost ready regarding the upgrade process https://review.opendev.org/#/c/710803/612:18
*** cshen has quit IRC12:36
*** cshen has joined #openstack-ansible13:12
*** cshen has quit IRC13:17
*** schwicht has joined #openstack-ansible13:17
*** cshen has joined #openstack-ansible13:30
*** markvoelker has joined #openstack-ansible13:53
*** markvoelker has quit IRC13:58
*** cshen has quit IRC14:40
admin0everytime i have to run neutron playbooks, it stops and starts the service which causes a service disruption in the compute nodes .. is there a way to tell the playbooks not to restart services like the l3 agents etc14:49
*** Goneri has joined #openstack-ansible14:54
*** cshen has joined #openstack-ansible14:55
kleinirestarted neutron agents should not cause any data plane outtage at all normally. what service disruption do you encounter?14:55
admin0i see the agents stopping and restarting ( rebuilding ) the security groups again15:10
admin0the platform where i see this is still on ubunt 16, rocky /18.1.815:10
*** sshnaidm_ is now known as sshnaidm|off15:11
*** luksky has quit IRC15:15
*** waxfire0 has joined #openstack-ansible15:25
*** waxfire has quit IRC15:25
*** waxfire0 is now known as waxfire15:25
kleiniI just tested in my production system running a mix of Pike/Queens and restarting there neutron agents just makes the agent check, whether everything is still in place on the data plane, but it does not change anything.15:27
kleiniDo you use Linux Bridge or OVS for data plane?15:27
*** spatel has joined #openstack-ansible15:38
*** spatel has quit IRC15:41
noonedeadpunkjrosser: don't you recall if we wrote down the way to use let's encrypt ssls into docs?15:47
noonedeadpunkjust can't find anything here https://docs.openstack.org/openstack-ansible/ussuri/user/security/ssl-certificates.html15:47
noonedeadpunkbtw, l3 restart can really make problems, if ha routers are not enabled15:48
noonedeadpunkbut there's no way to say not to restart some specific service in case you adjust configs15:48
*** luksky has joined #openstack-ansible15:50
arkanHi guys I ran ANSIBLE_ENABLE_TASK_DEBUGGER=True ansible-playbook os-octavia-install.yml15:51
arkanit's giving keystoneauth1.exceptions.http.Unauthorized for user service15:52
arkanbut It works from cli and from horizon for the same credentials (user service and pass)15:52
arkanonly I don't know why from ansible does not work15:52
arkanhttp://paste.openstack.org/show/795530/15:53
*** markvoelker has joined #openstack-ansible15:54
arkanif I use the same credentials as shown in the playbook debugger, then it will work from openstack client cli15:54
arkanif you have any idea please let me know, I already spent a few hours15:55
noonedeadpunkcan you paste task where it fails and error itself (out of stdout)15:57
arkanTASK [os_octavia : Create mgmt network]15:57
noonedeadpunkit's train?15:58
arkanyes15:58
arkanthe network is not created15:58
*** markvoelker has quit IRC15:59
arkanand I can created manually using the above credentials (cli & horizon)15:59
noonedeadpunkI can think only about that interface is wrong...16:00
noonedeadpunkhttps://opendev.org/openstack/openstack-ansible-os_octavia/src/branch/stable/train/tasks/octavia_mgmt_network.yml#L4216:00
noonedeadpunkwhat if you change that to public let's say?16:00
arkanif I change the task.args['auth']['username']=admin and task.args['auth']['password']=xxxxx and then redo, then it will work16:01
noonedeadpunkah, and eventually you probably will need to replace value for auth_url as well16:01
arkanno16:01
arkanit will work without changing it16:01
noonedeadpunkyes, but with admin account16:01
noonedeadpunkwhat I was suggesting to try using internal/public endpoints with octavia one16:02
arkanbut from cli it has the same auth_url for "service" user and it works16:02
arkanthis is the octavia rc http://paste.openstack.org/show/795531/16:04
jrossernoonedeadpunk: we did not do extra docs for letsecrypt, but we can add some16:04
arkanthe same username "octavia" for the project "service" from cli it works, but from the ansible it does not16:05
arkanthe internal and admin has the same url16:05
arkanonly the my public is different16:06
jrossernoonedeadpunk: I wonder if this Octavia stuff is about delegating to the service setup host16:06
jrosserbecause that’s not default in train iirc?16:06
noonedeadpunkarkan: btw you have a bit different OS_AUTH_URL in openrc and in debug oputput. But not sure that matters... as the only difference is v3 in the end which should be ok.16:07
noonedeadpunkI think we delegate by default in train...16:08
noonedeadpunkbut if connection is ok with keystone, see no reason why it should fail from there...16:08
arkanthis is from octavia container /etc/octavia/octavia.conf http://paste.openstack.org/show/795532/16:09
arkanit is using the same credentials16:09
noonedeadpunkhm....16:10
arkannoonedeadpunk: I can add v3 and redo and see if it works or not16:10
arkanin the playbook debugger16:11
noonedeadpunkarkan: we use domain_id for domain_name field https://opendev.org/openstack/openstack-ansible-os_octavia/src/branch/stable/train/tasks/octavia_mgmt_network.yml#L34-L3516:11
noonedeadpunkwould be good to check if it's not the root. I can't recall at the moment if plugin at this ansible version can apply id instead of name...16:12
arkannoonedeadpunk: do you want me to check something?16:13
noonedeadpunkyeeah, can you set octavia_service_user_domain_id: Default and octavia_service_project_domain_id: Default and check theat specific part?16:14
noonedeadpunkor just replace these variables with "Default" in the code16:15
noonedeadpunknot to break rest of the things16:15
arkando you want me to stop the debugger now ?16:15
arkanand apply the changes then rerun os-octavia-install ?16:15
arkanbtw I did not set any thing related to octavia_service_user_domain_* in my user_variables, so I used the defaults16:17
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_nova stable/ussuri: Add Centos-8 support  https://review.opendev.org/73925916:17
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_horizon stable/ussuri: Add Centos-8 support  https://review.opendev.org/73926016:17
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-os_heat stable/ussuri: Add Centos-8 support  https://review.opendev.org/73926116:17
arkannow I will make the changes that You've asked and rerun os-octavia-install16:17
noonedeadpunkprobably just hardcode "Default" then at these lines16:18
arkannoonedeadpunk: I set the vars and now I ran os-octavia-install again16:20
*** udesale has joined #openstack-ansible16:23
arkanit will fail again16:24
arkanafter that I will see what task.args has16:24
arkanhttp://paste.openstack.org/show/795533/16:25
*** udesale_ has quit IRC16:25
arkanand this is the error16:27
arkanhttp://paste.openstack.org/show/795535/16:27
jrosserarkan: can you please look at the topmost commit here https://github.com/openstack/octavia/commits/stable/train16:32
arkanok16:32
jrosserand check what the SHA of the octavia service you are deployinh16:33
jrosseryou will find that here https://github.com/openstack/openstack-ansible/blob/stable/train/playbooks/defaults/repo_packages/openstack_services.yml#L27416:33
jrosseralso can you tell me what "localhost" is here controller1_octavia_server_container-d8bd0abc -> localhost16:35
jrosserthat would be your deploy host, but what is that?16:35
arkanlocalhost I saw it also, but I don't know that is this16:36
arkanI'm deploying from the controller node16:36
*** jbadiapa has quit IRC16:37
arkanmy controller node is my deployer node16:37
jrosserarkan: it might be worth trying with these variables in your user_variables.yml https://github.com/openstack/openstack-ansible/blob/ed3691fd05a62cc71f05fafd7f3c5c1e7b107551/etc/openstack_deploy/user_variables.yml#L33-L3416:42
arkanmy openstack is using this commit commit 27884f5cfd1b5c3eb4e80cc1bc0d46b93656858a (HEAD, tag: 20.1.1)16:44
arkanMerge: f2a32b3df efae39ca016:44
arkanAuthor: Zuul <zuul@review.opendev.org>16:44
arkanDate:   Wed May 13 02:10:38 2020 +000016:44
arkanand for octavia16:46
arkancommit 7cbec4b075e1af1747b7c7909b57cea2efbcf6b6 (HEAD, origin/stable/train)16:46
arkanAuthor: OpenStack Proposal Bot <openstack-infra@lists.openstack.org>16:46
arkanDate:   Tue Mar 24 18:35:51 2020 +000016:46
arkan    Updated from OpenStack Ansible Tests16:46
arkan    Change-Id: If22140803c27cdf78480339511ae8131f5a4ffc016:46
arkanroot@controller1:/etc/ansible/roles/os_octavia# git remote -v16:46
arkanoriginhttps://opendev.org/openstack/openstack-ansible-os_octavia (fetch)16:46
arkanoriginhttps://opendev.org/openstack/openstack-ansible-os_octavia (push)16:46
arkanI can update octavia role16:47
arkanand rerun it16:47
arkanjrosser: also related to what you've pasted16:47
arkanwhat to put there ?16:47
arkan#openstack_service_setup_host: "{{ groups['utility_all'][0] }}"16:48
arkan#openstack_service_setup_host_python_interpreter: "/openstack/venvs/utility-{{ openstack_release }}/bin/python"16:48
arkanfirst I will try to update my octavia role and rerun it16:49
*** udesale has quit IRC16:49
*** spatel has joined #openstack-ansible16:49
spateljrosser: noonedeadpunk just for information i build centos8 with all current patches in my labs and all good. my build passed without any error16:52
noonedeadpunkcool!16:53
arkanthe repo that is using based on the https://docs.openstack.org/project-deploy-guide/openstack-ansible/train/deploymenthost.html16:55
arkanthat I have it https://opendev.org/openstack/openstack-ansible-os_octavia16:55
arkanand on the site https://opendev.org/openstack/openstack-ansible-os_octavia/src/branch/stable/train they have the same sha25616:56
arkanit's the same commit16:56
arkanjrosser: what openstack ansible version are you using ?17:01
*** cshen has quit IRC17:02
openstackgerritMerged openstack/ansible-config_template master: Update config_template to use the builtin _get_remote_user function  https://review.opendev.org/73805817:03
jrosserarkan: I highlighted two commentedout lines I. the example user_config. my suggestion is that you copy them, without the comments into your user variables17:05
*** spatel has quit IRC17:05
arkanok17:05
jrossermy question about sha was the sha of Octavia, not os_octavia17:05
jrosserread the comments in the example user config about those lines17:06
jrosserthe utility container has an openrc in it which is all set up for the internal endpoint17:06
jrosseryou will find it there if you look in the utility container17:06
arkanjrosser: yes I know that it has openrs, but I usually don't use it17:08
arkannow I set the vars17:08
arkanand re-run the os-octavia-install17:09
arkanjrosser: wow17:11
arkanit worked17:11
arkanthose 2 magic lines17:11
jrosserthe openrc in the utility container is set up very specifically for this17:12
jrosserit’s using the admin user and the internal endpoint17:12
jrosserso totally not like the one from horizon17:12
arkanyes17:13
jrosseryou should also pay attention to the commit in octavia I linked17:13
jrosserthat’s fixing a bug related to endpoints in octavia itself17:14
arkanyes, but how to pull the changes to my host ?17:14
arkanshould I use some scripts ? or using git pull, ...etc ?17:15
jrosserno :)17:16
arkan:))17:16
jrosseri linked you this https://github.com/openstack/openstack-ansible/blob/stable/train/playbooks/defaults/repo_packages/openstack_services.yml#L27417:16
jrosserthat is a file in the openstack-ansible repo17:16
arkanok17:17
jrosserit says "for this release of openstack-ansible these are the SHA of all the openstack services we are going to use"17:17
arkanok17:17
jrosserit's nothing to do with the versions of os_octavia or the ansible roles that deploy the services17:17
jrosserso check the sha you have17:17
jrosserif you have deployed tag 20.1.1 of openstack-ansible you will have a SHA of octavia which does not include that fix17:19
arkanhahah I was thinking about git17:20
arkan:)))17:20
arkanI found the file in my host17:20
jrosserwell ultimately it is done with git, but as part of building the venvs17:20
jrosserthis is how we 'pin' all of the services to a specific known state for each release17:21
jrosserbut as you can see in that file they are all ansible variables17:21
jrosserso everything there can be overridden17:21
arkanI have this one in my host17:21
arkanoctavia_git_install_branch: 7fa2f933a2c99e7ef0b2cb8d4cbcc1e9f9c0c79f17:21
arkanit's not the same17:21
arkanthe one you gave me is octavia_git_install_branch: 09e863eb539e29c8950ddfce12d602c0e77c1b7e17:21
jrosserok so now to understand how the git tags work17:22
jrosserevery 2 weeks or so we make a point release on each of the stable branches, and i think you have used 20.1.117:22
arkanyes17:22
jrosserwhat happens at the point release is all those SHA get updated, so any bugfixes from the upstream openstack services are pulled in17:23
arkangreat17:23
jrosserand there is now a 20.1.2 release of openstack-ansible which updates the SHA of the octavia service17:23
arkangreat17:24
arkanso I can pull it17:24
jrosseryou can do minor upgrade, yes17:24
arkanfrom 20.1.1 to 20.1.2 with patch 217:24
arkanand it will get updated17:25
arkanafter I git pull it, should I run scripts/bootstrap-ansible.sh17:25
arkan?17:25
jrosserthere are instructions https://docs.openstack.org/openstack-ansible/train/admin/upgrades/minor-upgrades.html17:26
jrosserreplace the tag with the one you actually need17:26
arkanyes17:26
jrosserwhen you get more familiar with this you can start to do things like just override the var with the octavia SHA17:27
arkandone17:27
arkanI git fetch and git checkout17:28
arkanroot@controller1:/opt/openstack-ansible/playbooks/defaults/repo_packages# git log -117:28
arkancommit edeca57bdc71360e06e393dbb9853b4405dc62f0 (HEAD, tag: 20.1.2)17:28
*** luksky has quit IRC17:29
arkangreat17:31
arkanjrosser: thanks a lot for your help17:31
jrosserno worries :) good luck!17:31
*** cshen has joined #openstack-ansible17:33
jrossernoonedeadpunk: we had a race condition with certbot fixed here https://review.opendev.org/#/c/734116/17:36
*** cshen has quit IRC17:38
noonedeadpunkjrosser: btw have you seen a comment in https://review.opendev.org/#/c/737221/2 ?17:40
jrosseryes i saw it17:41
jrosseri have not thought about what to do17:42
jrossermy aim was to try to get the batch size to be larger than the number of containers in the control plane17:42
jrosseras i think thats slowing us down17:42
noonedeadpunkbtw out of jobs I don't really see effect from https://review.opendev.org/#/c/737543 for some reason:(17:43
noonedeadpunkbut didn't have time to test it against some prod17:44
jrosserits odd isnt it, becasue i did all that testing on a cloud vm17:44
jrosser*but* that was all for the second run where few tasks are changing17:44
*** luksky has joined #openstack-ansible17:44
noonedeadpunkyeah, and out of what I saw that really should speedup17:44
jrosseras i was really interested in reducing overhead17:44
jrosseroh yes i thnk it's absolutely valid but yes also i have not tried in a prod environment17:45
jrosseri'd be interested about what you think of the ANSIBLE_FORKS thing17:45
noonedeadpunkmaybe we have some ram issue which doesn't allow to execute faster...17:45
jrosserif you watch the zuul console you can see it execute the tasks in blocks of 5 hosts17:45
jrosserthen a little pause whilst the batch completes, then the next batch17:46
noonedeadpunkyeah, I saw exactly the same in prod17:46
jrosserand again i measure a real improvement in a local vm17:46
noonedeadpunkeventually maxsessions thing also bothered me. I think I need to spawn a sandbox to see if we do adjust this17:48
jrosserhttps://github.com/openstack/openstack-ansible/blob/103f3d84c353ef33f8433e43697c140fd6d2f92d/tests/bootstrap-aio.yml#L6817:48
noonedeadpunkyes, but it's really only for aio17:49
noonedeadpunkAnd I'm not sure I see the best place to adjust this17:49
noonedeadpunkwe can use hardening or openstack_hosts for that17:50
jrosserthere is also a warning here https://github.com/openstack/openstack-ansible/blob/dbd711564a0e4751a13037e01ea198a9f7910b79/doc/source/admin/maintenance-tasks/ansible-modules.rst#L14017:50
jrosserbut i've never seen that myself17:50
jrosserunless it means that you run up against the sshd MaxSessions at that point17:50
noonedeadpunkeventually it's not so easy to hit that. you like need to run some command against all containers within controller host17:52
noonedeadpunkBut not sure it's good idea to leave that flaw as is, as one day we may face it or weird things might happen with connections17:53
openstackgerritMerged openstack/openstack-ansible-haproxy_server stable/ussuri: Add Centos-8 support  https://review.opendev.org/73910317:54
*** markvoelker has joined #openstack-ansible17:55
openstackgerritMerged openstack/openstack-ansible-lxc_hosts stable/ussuri: Remove machinectl from LXC cache prep  https://review.opendev.org/73911417:55
noonedeadpunkbut generally I think we need to extend number of forks from default value17:56
openstackgerritMerged openstack/openstack-ansible-haproxy_server master: Ensure there is no race between haproxy and certbot for initial cert  https://review.opendev.org/73411617:57
openstackgerritDmitriy Rabotyagov (noonedeadpunk) proposed openstack/openstack-ansible-haproxy_server stable/ussuri: Ensure there is no race between haproxy and certbot for initial cert  https://review.opendev.org/73926917:57
openstackgerritMerged openstack/openstack-ansible-lxc_hosts stable/ussuri: Do not install python2 or its dev package on Ubuntu Bionic hosts  https://review.opendev.org/73860717:58
*** markvoelker has quit IRC18:00
openstackgerritMerged openstack/openstack-ansible-plugins stable/stein: Optimise linear strategy access to physical host variables  https://review.opendev.org/73796518:09
openstackgerritMerged openstack/openstack-ansible-lxc_container_create stable/ussuri: Add Centos-8 support  https://review.opendev.org/73911818:10
*** markvoelker has joined #openstack-ansible18:13
*** markvoelker has quit IRC18:17
*** cshen has joined #openstack-ansible18:37
*** cshen has quit IRC18:42
*** markvoelker has joined #openstack-ansible18:43
*** cshen has joined #openstack-ansible18:44
*** markvoelker has quit IRC18:48
openstackgerritMerged openstack/openstack-ansible-os_placement stable/ussuri: Add Centos-8 support  https://review.opendev.org/73911319:11
openstackgerritMerged openstack/openstack-ansible-os_barbican stable/ussuri: Add Centos-8 support  https://review.opendev.org/73911519:11
openstackgerritMerged openstack/ansible-role-uwsgi stable/ussuri: Add Centos-8 support  https://review.opendev.org/73910419:13
openstackgerritMerged openstack/openstack-ansible-memcached_server stable/train: Reduce verbosity level when debug is true  https://review.opendev.org/73618019:32
openstackgerritMerged openstack/openstack-ansible-openstack_hosts stable/ussuri: Fix CentOS functional test  https://review.opendev.org/73912120:23
openstackgerritMerged openstack/openstack-ansible-openstack_hosts stable/ussuri: Enable PowerTools repository for distro installs  https://review.opendev.org/73911720:23
*** markvoelker has joined #openstack-ansible20:44
*** markvoelker has quit IRC20:48
*** markvoelker has joined #openstack-ansible21:09
*** cshen has quit IRC21:26
*** markvoelker has quit IRC21:48
*** cshen has joined #openstack-ansible22:01
*** cshen has quit IRC22:05
*** schwicht has quit IRC22:14
openstackgerritMerged openstack/openstack-ansible-haproxy_server stable/ussuri: Ensure there is no race between haproxy and certbot for initial cert  https://review.opendev.org/73926922:25
openstackgerritMerged openstack/openstack-ansible-lxc_container_create stable/ussuri: Fix tmpfiles-setup wait handler  https://review.opendev.org/73906722:36
openstackgerritMerged openstack/openstack-ansible-tests stable/ussuri: Do not set alternatives before package is present  https://review.opendev.org/73830922:49
*** tosky has quit IRC22:50
openstackgerritMerged openstack/openstack-ansible-os_rally stable/ussuri: Do not install development packages on the target  https://review.opendev.org/73867622:57
*** markvoelker has joined #openstack-ansible23:49
*** markvoelker has quit IRC23:53

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!