| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-tests master: Pin tox version to less than 4 https://review.opendev.org/c/openstack/openstack-ansible-tests/+/867197 | 08:35 |
|---|---|---|
| noonedeadpunk | yep. openstack-ansible-role-jobs were passing before tox4 stuff | 08:52 |
| noonedeadpunk | ah, I see the patch now :) | 08:53 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Remove support for calico ml2 driver. https://review.opendev.org/c/openstack/openstack-ansible/+/866119 | 08:56 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Do not try to install packages on each log_instance_info run https://review.opendev.org/c/openstack/openstack-ansible/+/866372 | 08:58 |
| noonedeadpunk | So, I was thinking. Maybe we can/should move isntallation of some "common" packages back to lxc_containers_create role to save-up execution time afterwards? For example, we always install git and libxml2, and some roles don't need anything except these: https://opendev.org/openstack/openstack-ansible-os_placement/src/branch/master/vars/debian.yml#L21-L23 | 09:01 |
| noonedeadpunk | Also we can skip update_cache when package_state != latest | 09:02 |
| moha7 | Is it optionally configurable to use Pacemaker instead of Keepalived? | 09:20 |
| noonedeadpunk | You can disable keepalived deployment and configure pacemaker if you want on your own | 09:21 |
| noonedeadpunk | I do actually maintain pacemaker role but it's more for computes and masakari rather then for VIPs management. Likely, it can be used for that as well, but I never tried | 09:22 |
| jrosser | moha7: you asked yesterday if the internal and external vip can be in the same range - yes they can | 09:31 |
| jrosser | the internal vip must be on or routable from the internal mgmt network, the external one does not matter and can be "external only" if you want it to be | 09:31 |
| moha7 | jrosser: :) Thanks | 09:32 |
| jrosser | noonedeadpunk: yes it would make sense to move things back to lxc_containers_create - the need for git is not totally clear, for when theres no repo server? | 09:33 |
| noonedeadpunk | For when build_wheels==false | 09:34 |
| jrosser | hmm yes ok i see | 09:34 |
| moha7 | Is my understanding correct? => "the unnumbered `br-vlan` interface is for the provider (external) network and the `br-vxlan` is used for the self-service (private) network" | 09:35 |
| jrosser | moha7: this is a "reference example" so you can do whatever you need | 09:39 |
| jrosser | you need some interface to give to neutron for the external provider network, it can be you choice what this is | 09:40 |
| moha7 | Aha! In the reference example, when you have choosed br-vlan to be unnumbered (without IP), you mean it has been considered for Provider network, right? | 09:46 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-lxc_hosts master: Add git and libxml2 to container base image https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/867203 | 09:50 |
| jrosser | moha7: in the reference example br-vlan does two things, it is the trunk interface where 'vlan' type networks are made (usually provider networks), and also untagged traffic on br-vlan becomes a flat interface, typically used in the all-in-one for the external network | 09:53 |
| moha7 | Got it. Thanks | 09:55 |
| jrosser | however for a real deployment i would recommend that you do not use flat type interfaces at all as they are inflexible and involve more configuration changes should you ever want to adjust anything lateer | 09:55 |
| jrosser | i would recommend making your external network be a vlan tag | 09:56 |
| opendevreview | Merged openstack/openstack-ansible-tests master: Pin tox version to less than 4 https://review.opendev.org/c/openstack/openstack-ansible-tests/+/867197 | 10:07 |
| opendevreview | Merged openstack/ansible-role-zookeeper master: Add envi to the list of allowed commands https://review.opendev.org/c/openstack/ansible-role-zookeeper/+/867049 | 11:10 |
| opendevreview | Merged openstack/ansible-role-uwsgi master: Drop bindep file https://review.opendev.org/c/openstack/ansible-role-uwsgi/+/865587 | 11:12 |
| opendevreview | Merged openstack/ansible-role-uwsgi master: Install OpenSSL development headers https://review.opendev.org/c/openstack/ansible-role-uwsgi/+/864783 | 11:12 |
| *** dviroel_ is now known as dviroel|rover | 11:20 | |
| *** anbanerj is now known as frenzy_friday | 11:35 | |
| noonedeadpunk | I wonder why this fails upgrade jobs https://review.opendev.org/c/openstack/openstack-ansible/+/866119 | 11:58 |
| noonedeadpunk | stacktrace is a bit weird https://zuul.opendev.org/t/openstack/build/12be955b18464fc39addfd220bf090ca/log/logs/host/neutron-metadata-agent.service.journal-10-43-52.log.txt#1501 | 12:04 |
| noonedeadpunk | or well https://zuul.opendev.org/t/openstack/build/12be955b18464fc39addfd220bf090ca/log/logs/host/neutron-l3-agent.service.journal-10-43-52.log.txt#3125 | 12:05 |
| noonedeadpunk | "arping: recvfrom: Network is down" | 12:05 |
| opendevreview | Merged openstack/openstack-ansible master: Disable octavia anti-affinity for AIO builds https://review.opendev.org/c/openstack/openstack-ansible/+/866061 | 12:05 |
| noonedeadpunk | but it looks like related to the patch itself. Except I don't see anything there | 12:06 |
| noonedeadpunk | as ^ just passed upgrade jobs as well | 12:07 |
| noonedeadpunk | damiandabrowski andrewbonney can you please take a look at https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/866058 ? It's not running zookeeper (as we're about to oom with octavia tempest) but I've tested locally and it seemed working | 12:09 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Remove support for calico ml2 driver. https://review.opendev.org/c/openstack/openstack-ansible/+/866119 | 12:13 |
| noonedeadpunk | as if it's not related to the patch it's quite bad | 12:14 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: DNM https://review.opendev.org/c/openstack/openstack-ansible/+/867214 | 12:15 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: DNM https://review.opendev.org/c/openstack/openstack-ansible/+/867214 | 12:15 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_ironic master: Allow user defined extra files to be added to ironic tftp directory https://review.opendev.org/c/openstack/openstack-ansible-os_ironic/+/867218 | 12:44 |
| admin1 | does sr-iov needs hugepages , 64gb of memory ? | 13:41 |
| admin1 | or is that only for dpdk ? | 13:41 |
| jrosser | i dont think hugepages is anything to do with sriov | 13:47 |
| mgariepy | good morning | 13:54 |
| admin1 | ok | 13:55 |
| jrosser | noonedeadpunk: wonder what is going on with /etc/hostname there - we passed tests on centos9 not so long ago in the tests repo | 14:24 |
| noonedeadpunk | yup... But it's "stream" | 14:24 |
| noonedeadpunk | I dunno though if centos use systemd-resolved | 14:26 |
| jrosser | ohhh of course :) | 14:27 |
| jrosser | what you see last time is irrelevant for next time /o\ | 14:27 |
| noonedeadpunk | NeilHanlon_: Hey! Do you have any idea if generic cloud image has a different set of packages rather then minimal ISO? | 14:37 |
| noonedeadpunk | Specifically talking about tar | 14:37 |
| noonedeadpunk | As I do see it in generic, but we've got a PR that claims it's not in minimal | 14:38 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-lxc_hosts master: Ensure tar is installed on LXC host https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/867252 | 14:57 |
| *** dviroel|rover is now known as dviroel|rover|lunch | 15:22 | |
| opendevreview | Merged openstack/openstack-ansible-os_octavia master: Add coordination to octavia https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/866058 | 15:34 |
| opendevreview | Merged openstack/openstack-ansible-ceph_client master: Ensure role not fail when secret_uuid is not part of ceph_extra_confs https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/866974 | 15:34 |
| opendevreview | Merged openstack/openstack-ansible-lxc_hosts master: Add git and libxml2 to container base image https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/867203 | 15:37 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Update AIO to use OVN-style provider network for Trove https://review.opendev.org/c/openstack/openstack-ansible/+/867256 | 15:56 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_trove master: DNM https://review.opendev.org/c/openstack/openstack-ansible-os_trove/+/867257 | 15:57 |
| *** dviroel|rover|lunch is now known as dviroel|rover | 16:10 | |
| jamesdenton_ | has anyone played with the 'queue_master_location' var for rabbitmq.conf? | 16:12 |
| *** jamesdenton_ is now known as jamesdenton | 16:12 | |
| noonedeadpunk | we're still small enough for synching queues, so not yet | 16:14 |
| noonedeadpunk | *syncing | 16:14 |
| jamesdenton | this environment has HA queues, but TIL there is a queue leader/master that any rabbitmq cluster member will forward the traffic to | 16:15 |
| jamesdenton | and the queue leader is heavily lopsided, likely due to 'queue_master_location' defaulting to 'client-local' | 16:16 |
| jamesdenton | there is a 'min-master' option, too, which would load balance across them on initial queue creation | 16:16 |
| cloudnull | what good ? | 16:20 |
| jamesdenton | it's monday and things on fire | 16:21 |
| noonedeadpunk | oh | 16:23 |
| noonedeadpunk | I didn't know that | 16:23 |
| jamesdenton | me either | 16:24 |
| noonedeadpunk | Eventually, our way of HA queues is about to be deprecated | 16:24 |
| noonedeadpunk | As they've created multiple better options, but they must be supported by clients | 16:24 |
| noonedeadpunk | And I'm not aware of oslo doing smth with this regards | 16:24 |
| noonedeadpunk | But I wasn't checking lately tbh | 16:24 |
| jamesdenton | "All operations for a given queue are first applied on the queue's leader node and then propagated to mirrors"... "Messages published to the queue are replicated to all mirrors. Consumers are connected to the leader regardless of which node they connect to, with mirrors dropping messages that have been acknowledged at the leader. Queue mirroring therefore enhances availability, but does not distribute load across nodes | 16:26 |
| jamesdenton | (all participating nodes each do all the work)."... "If leader node of a queue (the node running queue leader) is available, all queue operations (e.g. declaration, binding and consumer management, message routing to the queue) can be performed on any node. Cluster nodes will route operations to the leader node transparently to the clients." | 16:26 |
| jamesdenton | anyway, our queue leaders are heavily weighted to one or two rabbit nodes, as the connection ordering wasn't randomized | 16:26 |
| jamesdenton | we're testing a way to rebalance them, | 16:26 |
| noonedeadpunk | maybe we should switch that as default then? | 16:27 |
| jamesdenton | it could be worth a short | 16:27 |
| jamesdenton | *shot | 16:27 |
| noonedeadpunk | I think if you run rabbitmq-install.yml -e rabbit_upgrade=true - that will flush and re-create all queues, so with option enabled it will re-balance load eventually | 16:28 |
| jamesdenton | yes, it could | 16:29 |
| noonedeadpunk | a bit of service disrubtion is involved though | 16:29 |
| noonedeadpunk | *disruption | 16:29 |
| cloudnull | jamesdenton nice, fire keeps the cloud tears away | 16:30 |
| cloudnull | jamesdenton I remember this bug we used to have to restart rabbit to get the master queues rebalanced across the cluster. OSA used to distribute the queue balance across nodes using a deterministic sort, however, oslo introduced a random jitter so there's really nothing osa can do to ensure that the queues remained balanced across the cluster. Just | 16:34 |
| cloudnull | plan for restarts about every 6 months. IDK if that's still the case? | 16:34 |
| noonedeadpunk | dunno really - I haven't got issues with rabbit for a while now tbh | 16:37 |
| noonedeadpunk | I think it's rabbit that imporved a lot since 3.9 | 16:39 |
| noonedeadpunk | Out of current miracles is why in the world https://review.opendev.org/c/openstack/openstack-ansible/+/866119 fails on upgrade jobs | 16:41 |
| noonedeadpunk | And it's fails because of the patch | 16:42 |
| noonedeadpunk | As DNM feels great https://review.opendev.org/c/openstack/openstack-ansible/+/867214 | 16:42 |
| cloudnull | gremlins | 16:44 |
| noonedeadpunk | likely :D | 16:54 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Remove support for calico ml2 driver. https://review.opendev.org/c/openstack/openstack-ansible/+/866119 | 16:57 |
| noonedeadpunk | I'm waiting for 867218 to merge and will propose branching patch with SHAs | 17:13 |
| noonedeadpunk | https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/867252 is closing bug we were reported through PR. It can be backported later though | 17:14 |
| * jrosser finally boots rocky linux arm system with ironic | 17:24 | |
| NeilHanlon_ | noonedeadpunk: tar is not included in the "@minimal" target. GenericCloud is a different package set that is installed. https://git.resf.org/sig_core/kickstarts/src/branch/r9/cloud/rocky-cloud-base-packages.ks | 17:26 |
| *** NeilHanlon_ is now known as NeilHanlon | 17:28 | |
| * NeilHanlon should setup SASL | 17:28 | |
| opendevreview | Merged openstack/openstack-ansible-os_ironic master: Allow user defined extra files to be added to ironic tftp directory https://review.opendev.org/c/openstack/openstack-ansible-os_ironic/+/867218 | 17:55 |
| jrosser | \o/ woohoo | 17:56 |
| jrosser | NeilHanlon: DIB built me something that worked fine for rocky+ironic | 17:57 |
| NeilHanlon | 👀 nice! | 17:58 |
| jrosser | and on arm too :) | 17:58 |
| NeilHanlon | ! | 17:58 |
| opendevreview | Merged openstack/openstack-ansible-os_octavia master: Change defaults for octavia topology and affinity https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/866062 | 18:25 |
| noonedeadpunk | NeilHanlon: aha, thanks for pointing to that! | 19:51 |
| -opendevstatus- NOTICE: Gerrit will be unavailable for a short time as it is upgraded to the 3.6 release | 19:58 | |
| *** dviroel|rover is now known as dviroel|rover|out | 21:28 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!