| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Switch to ceph-ansible stable-8.0 https://review.opendev.org/c/openstack/openstack-ansible/+/921976 | 07:29 |
|---|---|---|
| jrosser | noonedeadpunk: this is the next place to do some ansible 2.17 work https://github.com/noonedeadpunk/ansible-role-pacemaker_corosync | 08:37 |
| noonedeadpunk | ah, includes | 08:41 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/2024.1: Bump Keystone SHAs for 2024.1 RC3 https://review.opendev.org/c/openstack/openstack-ansible/+/921999 | 09:24 |
| noonedeadpunk | jrosser: this should fix it https://github.com/noonedeadpunk/ansible-role-pacemaker_corosync/pull/7 | 09:56 |
| jrosser | stable/2023.1 upgrade jobs are broken (zed branch name i think) | 10:51 |
| noonedeadpunk | have a question - does anyone experienced very poor udp performance in terms of networking, while TCP is being fine? | 10:58 |
| noonedeadpunk | Have that even when 2 VMs are on the same compute | 10:58 |
| noonedeadpunk | https://paste.openstack.org/show/bXD2UCAu75LU42nleFNh/ | 10:59 |
| noonedeadpunk | TCP is kinda 12gbit and no package loss | 10:59 |
| noonedeadpunk | nothing fancy - antelope, 22.04, ovs | 11:25 |
| jrosser | noonedeadpunk: thanks for the pacemaker fix - it gets a bit further now https://zuul.opendev.org/t/openstack/build/0ab12466fa5f4bbba6829cfd18e2513a/log/job-output.txt#14016 | 12:38 |
| jrosser | i wonder if there is some exception or stack trace there that gets lost in our wrapper | 12:39 |
| jrosser | oooh interesting `[ 1407.149138] Out of memory: Killed process 42924 (ansible-playboo) total-vm:32170160kB, anon-rss:31701744kB, file-rss:0kB, shmem-rss:8kB, UID:0 pgtables:62932kB oom_score_adj:0` | 19:10 |
| jrosser | thats why all the 2.17 jobs are failing now, there is a gigantic memory leak somewhere in ansible-playbook | 19:10 |
| jrosser | https://github.com/ansible/ansible/issues/83392 | 19:17 |
| frickler | wow, nice | 19:27 |
| jrosser | i just ran tip of stable-2.17 branch and its fixed there | 19:28 |
| jrosser | https://github.com/ansible/ansible/pull/83405/commits/db1689a44331fa8745f19d1c3e1450d530241784 | 19:29 |
| jrosser | but 2.17 release used up 32G of ram and OOM in just a few minutes runtime | 19:29 |
| frickler | actually looking closer I see ooms for the bump in kolla-ansible, too https://review.opendev.org/c/openstack/kolla-ansible/+/921743. so we'll wait for a fixed release I guess | 19:35 |
| jrosser | there was some talk of using ansble 10 for zuul stuff as well? that would include ansible-core 2.17.0 currently | 19:35 |
| frickler | not sure how far that is, but I'll mention it in the zuul channel, yes | 19:37 |
| jrosser | though apart from this bug, and having to fix a couple of deprecated remaining use of include: in our external dependancies it seems relatively trouble free so far | 19:39 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!