Tuesday, 2022-07-19

*** ysandeep|out is now known as ysandeep05:21
*** ysandeep is now known as ysandeep|lunch07:57
jrosseri always forget if we can depend-on in stable branches without a sha bump, for example on this https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/85023307:59
noonedeadpunkI think yes?08:47
noonedeadpunkeventually within a branch we likely don't need that as we pull always latest except upgrade job for N-108:48
noonedeadpunkso we should be able to just recheck once it merges08:49
opendevreviewJonathan Rosser proposed openstack/openstack-ansible stable/xena: Add mistra-extra repo  https://review.opendev.org/c/openstack/openstack-ansible/+/84957408:54
opendevreviewJonathan Rosser proposed openstack/openstack-ansible stable/xena: Return facts gathering to pre_tasks for masakari  https://review.opendev.org/c/openstack/openstack-ansible/+/84800608:54
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-os_magnum stable/xena: Fixed dest typo in config_template  https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/84693208:54
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-os_horizon stable/xena: Fix ALLOWED_HOSTS  https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/84800408:54
opendevreviewJonathan Rosser proposed openstack/openstack-ansible stable/xena: Fix facts gathering for zun  https://review.opendev.org/c/openstack/openstack-ansible/+/84963008:55
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-os_manila stable/xena: Create backends when running against manila_share  https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/84799308:55
opendevreviewJonathan Rosser proposed openstack/openstack-ansible stable/xena: Do not try to run etcd in serial mode  https://review.opendev.org/c/openstack/openstack-ansible/+/84800308:55
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-os_mistral stable/xena: Add mistral-extra in the mistral venv  https://review.opendev.org/c/openstack/openstack-ansible-os_mistral/+/84957508:56
opendevreviewJonathan Rosser proposed openstack/openstack-ansible stable/xena: Switch ARA report type from html to database for openstack-ansible jobs  https://review.opendev.org/c/openstack/openstack-ansible/+/84979708:56
opendevreviewJonathan Rosser proposed openstack/ansible-role-python_venv_build stable/xena: Gather build hosts facts once.  https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/84964108:56
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-os_tempest stable/xena: Drop tripleo c7 job definition  https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/84997708:57
opendevreviewJonathan Rosser proposed openstack/openstack-ansible-galera_server stable/xena: Bump mariadb version.  https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/84743508:57
opendevreviewMerged openstack/openstack-ansible-rabbitmq_server stable/xena: Sync RedHat erlang version  https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/85023309:28
*** ysandeep|lunch is now known as ysandeep09:30
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Bump ansible-core to 2.13.2  https://review.opendev.org/c/openstack/openstack-ansible/+/84238710:05
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Bump ansible-core to 2.13.2  https://review.opendev.org/c/openstack/openstack-ansible/+/84238710:05
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Bump ansible-core to 2.13.2  https://review.opendev.org/c/openstack/openstack-ansible/+/84238710:05
jrosser^ there was something truly odd happening with that10:09
jrosseriirc it was some error with rabbitmq installation and i was wondering if something to do with the templating of the config files came out different with a newer version of jinja210:18
noonedeadpunkWell, the issue I saw was about erlang version conflict.10:25
noonedeadpunkBasically on apt update10:25
noonedeadpunk(no sorry, instalaltion)10:25
noonedeadpunkAnyway have some time to waste on it now10:26
jrosseroh and this https://zuul.opendev.org/t/openstack/build/64e9c1c454cb447e88731e5b22386414/log/job-output.txt#1307910:39
noonedeadpunk^ thas sounds fun10:40
noonedeadpunkwell rabit issue is reproduced quite easily11:12
*** dviroel|out is now known as dviroel11:27
MichielPiscaer[m]Hi, I'm trying to install "only" the l3 agent on some nodes. I try to specify with neutron_l3_agent in openstack_user_config.yml but this get ignored. What is the bestway to do this?11:39
mgariepyMichielPiscaer[m], can you paste the relevant part of your config ?12:01
mgariepyon paste.openstack.org or similar12:01
*** ysandeep is now known as ysandeep|break12:02
mgariepyJust to be clear you want to have neutron-agent on only a subset of your infra node ? or you want to have them on separate host ?12:02
mgariepyMichielPiscaer[m], also : https://github.com/openstack/openstack-ansible/blob/master/etc/openstack_deploy/openstack_user_config.yml.example#L420-L45212:07
*** ysandeep|break is now known as ysandeep|mtg12:33
noonedeadpunkMichielPiscaer[m]: well, I guess you will at least need also ovs/lxb agent there in addition to l3?12:46
noonedeadpunkalso wondering about dhcp/metadata?12:46
noonedeadpunkbut eventually, in openstack_user_config you should place only hosts from physical_skel 12:47
mgariepyif you tried something that didn't work you might need to cleanup the opesntack_inventory.json file..12:47
MichielPiscaer[m]Sorry, that there is some radio silence, but I'm in a meeting. I will get to you all in a couple of minutes. 12:47
mgariepythis part might need a bit more documentation12:48
mgariepyMichielPiscaer[m], no worries, IRC doesn't need to be instant. :)12:49
noonedeadpunkas of today, in order to place only l3 separately, you will need to make new env.d file and define independant stanza comparing to default one https://opendev.org/openstack/openstack-ansible/src/branch/master/inventory/env.d/neutron.yml12:50
noonedeadpunkas this is smth that describes how components co-exist with each other12:51
*** ysandeep|mtg is now known as ysandeep12:58
mgariepyissue is bionic, rabbitmq and erlang not compatible ..  https://review.opendev.org/c/openstack/openstack-ansible-os_tempest/+/849861/13:10
mgariepyfor train.13:10
mgariepydid we had a fix for this?13:11
mgariepynot sure why erlang version do not respect the pin also.https://zuul.opendev.org/t/openstack/build/a4c18720342949ad835e061850ea1485/log/logs/etc/host/apt/preferences.d/rabbitmq.pref.txt13:12
noonedeadpunkwtf with ansible-core 2.13..... It feels like it explicitly provides version isntead of relying on system13:12
noonedeadpunkmgariepy: yeah, I think we had a fix for that13:12
mgariepyhttps://zuul.opendev.org/t/openstack/build/a4c18720342949ad835e061850ea1485/log/logs/host/apt/history.log.txt#6913:12
noonedeadpunkwe used https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/849225 for the fix13:14
mgariepyso we need to backport to U/S/T ?13:14
noonedeadpunkSo what confuses me for ansible-core 2.13 is that "msg": "'/usr/bin/apt-get -y -o \"Dpkg::Options::=--force-confdef\" -o \"Dpkg::Options::=--force-confold\"       install 'erlang-base=1:25.0.3-1' 'erlang-diameter=1:25.0.3-1' 'erlang-eldap=1:25.0.3-1' 'erlang-nox=1:25.0.3-1' 'socat=1.7.3.3-2''13:14
noonedeadpunkmgariepy: well... we can do that13:15
opendevreviewMarc Gariépy proposed openstack/openstack-ansible-rabbitmq_server stable/ussuri: Use cloudsmith repo for rabbit and erlang  https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/85035013:21
jrossernoonedeadpunk: you mean like 2.13 ignores apt pins somehow?13:41
noonedeadpunkit provides version explicitly somehow13:42
jrosseroh i see what you mean13:43
noonedeadpunkand yes, with that it seems ignores apt pins13:43
noonedeadpunkI never saw ansible passing erlang-base=1:25.0.3-1 for install command13:43
noonedeadpunkthough I don't see even with -vvv what command was in 2.1213:43
mgariepyhttps://github.com/ansible/ansible/commit/4a62c4e3e44b01a904aa86e9b87206a24bd41fbc13:44
admin1i followed the docs to upgrade to 25.0.0 .. i get this: https://paste.opendev.org/show/b7T9tsd6UptEOLtOBdtu/   something new that has to do with Generate an OpenSSH user certificate for keystone-container13:44
noonedeadpunkwtf13:45
mgariepynoonedeadpunk, indeed..13:45
* noonedeadpunk goes to ansible13:45
mgariepyfrom the comment: The version selection is handled by Apt itself, so things like the system policy, pinning, etc, are considered.13:46
MichielPiscaer[m]noonedeadpunk: I currently use linux bridge agent. When we also move the dhcp/metadata, that is fine. But I don't want a container for the neutron server on the new nodes.13:47
* noonedeadpunk don't trust that statement13:47
mgariepybut i think it's kinda wrong while reading quickly the change..13:47
noonedeadpunkMichielPiscaer[m]: then jsut define network_hosts for l3/dhcp/meta and network-agent_hosts for api in openstack_user_config13:48
mgariepyhttps://github.com/ansible/ansible/issues/7796913:51
jrosseradmin1: try `openstack-ansible playbooks/certificate-ssh-authority.yml`13:56
noonedeadpunklol, also erlang :D13:56
MichielPiscaer[m]noonedeadpunk & mgariepy https://paste.opendev.org/show/baX2mILNig6BMuz4Op8I/ I tryed using network-agent_hosts but it still generates a a container on the node.14:00
opendevreviewMerged openstack/openstack-ansible stable/victoria: Fix facts gathering for zun  https://review.opendev.org/c/openstack/openstack-ansible/+/84963214:00
jrosserMichielPiscaer[m]: i think it may have done what you asked it to?14:02
jrosserspecifically becasue of `network_hosts`14:03
noonedeadpunkMichielPiscaer[m]: one thing, is that hosts are cached in openstack_inventory.json once generated and not auto-removed from there for $reasons14:04
noonedeadpunkThough backup is present in archive nearby. So if you have misconfigured that part - container will still show up14:04
MichielPiscaer[m]I did a restore of /etc/openstack_deploy from this morning.14:05
noonedeadpunkah, and it still appears?14:05
noonedeadpunkhuh14:05
noonedeadpunkah. yes, jrosser is right14:05
noonedeadpunkit should be `network-agent_hosts` not `network_hosts`14:05
jrosseri am guessing because of cached stuff in the inventory it has remembered about the agents being defined there at some previous point14:06
noonedeadpunkso, tctrko1, tctrko2,  tctrko3 are  network-infra_hosts  and tcpuko1 is network-agent_hosts14:06
jrossersorry server i mean14:07
*** ysandeep is now known as ysandeep|dinner14:28
mgariepyhttps://zuul.opendev.org/t/openstack/build/45aad60bcb0d43618bac6ac1955cbcba/log/job-output.txt#1216914:28
mgariepyFailed to download key at http://repo.vexxhost.net/mariadb/key.gpg: Request failed:14:28
opendevreviewMerged openstack/ansible-role-python_venv_build stable/xena: Gather build hosts facts once.  https://review.opendev.org/c/openstack/ansible-role-python_venv_build/+/84964114:34
MichielPiscaer[m]noonedeadpunk: I try'ed network-agent_hosts, network-infra_hosts & network-server_hosts but all does not pick any changes. I'm using Ussuri & I don't have any files in the env.d & conf.d directory https://paste.opendev.org/show/bXuoPI8Vrh63p8vMa66D/ 14:35
noonedeadpunkoh, ussuri :)14:35
noonedeadpunkI should have started with that L)14:36
noonedeadpunkMichielPiscaer[m]: you would need to create custom env.d file then inside /etc/openstack_deploy/env.d directory14:36
opendevreviewMerged openstack/openstack-ansible-os_magnum stable/xena: Fixed dest typo in config_template  https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/84693214:37
MichielPiscaer[m]noonedeadpunk: sorry for that.14:38
noonedeadpunklet me search how it should look like14:39
* noonedeadpunk got side-pinged14:46
MichielPiscaer[m]noonedeadpunk: np14:48
noonedeadpunkMichielPiscaer[m]: smth like that should work https://paste.openstack.org/show/b20ep2WybbuJ6OQgtiyY/14:51
noonedeadpunk(I hope)14:52
MichielPiscaer[m]And then I define a network-agent_hosts for atleast the l3 agent?14:54
MichielPiscaer[m]network-agent_hosts is then the tcpuko nodes?14:54
noonedeadpunkyup15:02
*** ysandeep|dinner is now known as ysandeep15:02
opendevreviewMarc Gariépy proposed openstack/openstack-ansible-rabbitmq_server stable/ussuri: Use cloudsmith repo for rabbit and erlang  https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/85035015:03
damiandabrowskinoonedeadpunk: alarm failed again? :D 15:16
noonedeadpunkwhaaat15:19
noonedeadpunkdamn15:19
noonedeadpunk#startmeeting openstack_ansible_meeting15:20
opendevmeetMeeting started Tue Jul 19 15:20:16 2022 UTC and is due to finish in 60 minutes.  The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot.15:20
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:20
opendevmeetThe meeting name has been set to 'openstack_ansible_meeting'15:20
noonedeadpunk#topic office hours15:20
noonedeadpunksorry, skipping rollcall15:20
noonedeadpunkSo, I saw that Rocky Linux 9 has finally released :)15:23
noonedeadpunkanother news, that are not that great, is that ansible-croe2.13 has valid bug, so I'd discourage anybody from using it, as it simply ignores package pinning. And while for some cases you will fail, in others it can silently do wrong thing15:24
noonedeadpunk*ansible-core15:24
mgariepyhttps://github.com/ansible/ansible/issues/7796915:24
mgariepyfor logs :D15:24
noonedeadpunkYes, thanks!15:25
noonedeadpunk#link https://github.com/ansible/ansible/issues/7796915:25
noonedeadpunk:D15:25
noonedeadpunkI was just checking backports to Xena, and at very least horizon seems to be failing for bullseye pretty consistent, so I assume there's some valid issue in there15:26
noonedeadpunkAnd i've noticed that centos fails intermittently a lot especially on stable branches15:29
admin1anyone noticed any change in the cpu_allocation_ratio .. in one cluster which is on 24.3.0 , cpu_allocation_ratio is at 4.0, but it refuses to go above the physical threads .. giving There was a conflict when trying to complete your request. Unable to allocate inventory: Unable to create allocation for 'VCPU' on resource provider 'UUID '. The15:31
admin1requested amount would exceed the capacity.15:31
damiandabrowskiadmin1: maybe You have orphaned resource allocations in placement? I've seen it many times15:36
admin1sean from openstack-nova just replied it could be a mariadb bug 15:37
admin1" admin1: yep that version is apprently broken and its fixed in 10.6.8 i belive" 15:37
admin1doh ! 15:37
damiandabrowskiouh15:38
admin1https://lists.openstack.org/pipermail/openstack-discuss/2022-July/029536.html15:38
admin1spatel did a mariadb upgrade i think outside of the playbooks 15:39
admin1i try to stick to the playbooks to ensure i don't have any unicorn deployments15:39
noonedeadpunkI bet we backported mariadb bump recently but haven't released yet15:40
noonedeadpunkas waiting for other backports to land15:40
noonedeadpunkas eventually there're bunch of CVEs that's been released/covered recently15:40
noonedeadpunkand mariadb is always a powder barrel15:41
spateladmin1 no, i always stick to OSA version of db15:41
admin1hopefully that #tmp db fix also comes in 15:41
noonedeadpunkanyway, I don't have any more updates.15:41
noonedeadpunkI don't think we have backported it....15:42
admin1is there a cherry pick on the mysql i can apply 15:42
noonedeadpunkBut internally we applied 2 overrides to make it work15:42
noonedeadpunkso it's quite easy to get it15:42
admin1just apt upgrade to the new version ? 15:42
noonedeadpunkthat won't work :)15:43
noonedeadpunkoh, well, indeed we backported tmp fix15:43
noonedeadpunkhttps://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/84743515:43
admin1thanks 15:45
noonedeadpunk#topic bug triage15:46
noonedeadpunkI realized we have couple of new bugs15:46
noonedeadpunk#link https://bugs.launchpad.net/openstack-ansible/+bug/198146315:46
noonedeadpunkdamiandabrowski: you did quite a lot of work regarding pooling, and I'm not sure we included it in Xena, did we?15:47
* noonedeadpunk meories gets shorter and shorter with age15:47
noonedeadpunk*memories15:47
noonedeadpunkso maybe you can take a look at that, as sounds like smth we have already fighten with?15:48
noonedeadpunkalso15:53
noonedeadpunk#link https://bugs.launchpad.net/openstack-ansible/+bug/198022615:53
noonedeadpunkbut I think this one is just misconfigured openstack_user_config I guess15:54
noonedeadpunkI missed it for quite a while...15:54
noonedeadpunkWill answer it till EOD15:54
noonedeadpunk#endmeeting15:55
opendevmeetMeeting ended Tue Jul 19 15:55:27 2022 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:55
opendevmeetMinutes:        https://meetings.opendev.org/meetings/openstack_ansible_meeting/2022/openstack_ansible_meeting.2022-07-19-15.20.html15:55
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/openstack_ansible_meeting/2022/openstack_ansible_meeting.2022-07-19-15.20.txt15:55
opendevmeetLog:            https://meetings.opendev.org/meetings/openstack_ansible_meeting/2022/openstack_ansible_meeting.2022-07-19-15.20.log.html15:55
damiandabrowskinoonedeadpunk: yeah, db pooling changes are included in Xena, I'll have a look at this bug16:02
opendevreviewMerged openstack/openstack-ansible stable/xena: Switch ARA report type from html to database for openstack-ansible jobs  https://review.opendev.org/c/openstack/openstack-ansible/+/84979716:05
*** ysandeep is now known as ysandeep|afk16:18
*** ysandeep|afk is now known as ysandeep|out16:38
opendevreviewMerged openstack/openstack-ansible stable/xena: Set zuul post-timeout to 3 hours  https://review.opendev.org/c/openstack/openstack-ansible/+/84799018:06
opendevreviewMerged openstack/openstack-ansible stable/xena: Return facts gathering to pre_tasks for masakari  https://review.opendev.org/c/openstack/openstack-ansible/+/84800619:52
*** dviroel is now known as dviroel|dinner22:22
opendevreviewMerged openstack/openstack-ansible stable/xena: Add mistra-extra repo  https://review.opendev.org/c/openstack/openstack-ansible/+/84957423:18
*** dviroel|dinner is now known as dviroel|out23:20

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!