Tuesday, 2023-12-05

noonedeadpunkFWIW, if you're running Ubuntu 22.04 with default kernel and have some Intel E5 (or something from Haswell family) along with Intel Golds of any generation, be careful with live migrations to E5 due to this: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/203667508:26
noonedeadpunkAnd better use HWE kernel....08:26
opendevreviewMarcus Klein proposed openstack/openstack-ansible-ops master: Upgrade Grafana deployment to Grafana collection  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/90261808:47
jrossergood morning08:58
noonedeadpunko/09:37
opendevreviewMerged openstack/openstack-ansible-rabbitmq_server master: Add ability to set environment variable RABBITMQ_USE_LONGNAMES  https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/90100313:04
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible master: Ensure tempest include and exclude lists all use unique names  https://review.opendev.org/c/openstack/openstack-ansible/+/89396814:01
opendevreviewJonathan Rosser proposed openstack/openstack-ansible master: Allow installing collections from repos containing more than one  https://review.opendev.org/c/openstack/openstack-ansible/+/90147114:04
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-os_octavia master: Remove obsoleted provider drivers  https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/88551914:06
jrosser^ new gerrit thing there i think "X on behalf of Y"14:24
noonedeadpunkyes....14:33
noonedeadpunkNot sure I like it or hate it14:33
noonedeadpunkBut it looks weird and scary14:33
NeilHanlonoh that's... interesting14:37
jrosseralso.... today 8-O at permissions like `0o755` and more confusingly `0O755`14:40
NeilHanlonoOoOoO0O00O014:40
NeilHanlonspooky14:40
NeilHanloni thought we were past haloween14:40
jrosseri thought it was a typo given how close the 0 and o keys are14:41
jrosserNeilHanlon: i guess you saw we had a massive bunch of rocky jobs failing on repo stuff?14:42
NeilHanlonyeah i did see that14:50
NeilHanlonit doesn't really make sense, though, as there are plenty of mirrors that should be working just fine. in any case, I know I promised a change to override the repo URLs, I just haven't got a chance to work on it yet :\14:53
NeilHanlonok, i think i see.14:54
* NeilHanlon will investigate14:55
noonedeadpunk#startmeeting openstack_ansible_meeting15:01
opendevmeetMeeting started Tue Dec  5 15:01:01 2023 UTC and is due to finish in 60 minutes.  The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot.15:01
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:01
opendevmeetThe meeting name has been set to 'openstack_ansible_meeting'15:01
noonedeadpunk#topic rollcall15:01
noonedeadpunko/15:01
jrossero/15:01
damiandabrowskihi! (i just got back from a vacation)15:02
NeilHanlono/ 15:02
NeilHanlonwelcome back damiandabrowski :) hope it was a good holiday15:02
damiandabrowskiyeah thanks, finally i had some rest :D 15:04
noonedeadpunkthat is really sweet :)15:05
noonedeadpunk#topic office hours15:05
noonedeadpunkSo, I wanna re-raise one of recent bug reports, as we had chance to triage it on the production :D15:05
noonedeadpunk#link https://bugs.launchpad.net/openstack-ansible/+bug/203881815:06
spatelDo you know how to delete cinder-volume services ? 15:06
noonedeadpunkMy idea is not great, but well.... I was thinking, that maybe we should jsut enable all flags once rabbitmq_upgrade is finished15:06
spatelhttps://platform9.com/kb/openstack/deleting-a-cinder-service-from-host15:06
noonedeadpunkspatel: https://docs.openstack.org/cinder/latest/cli/cinder-manage.html#cinder-service15:07
spateldo we need cinder-manager? 15:07
spatelHow do i install cinder-manager? 15:08
noonedeadpunkI know that flags for rabbitmq must be enabled BEFORE the upgrade... Though, if we will backport and enable them after each one - we should be in a good shape...15:09
NeilHanlonthat seems at least logical to me15:09
noonedeadpunkAs it's kinda scary to enable flags before, since cluster might be in kinda unknown state consisting of multiple versions15:09
NeilHanlonright...15:10
noonedeadpunkor hell know what15:10
noonedeadpunklooking at table of when flags introduced/become required - there's quite some margin there15:10
jrosserandrewbonney: rabbitmq feature flags ^^ if you're around15:11
damiandabrowskilooks good, but it may still leave some users with a broken rabbitmq after upgrade, so maybe it's worth to mention in releasenotes what to do with this15:12
NeilHanlonis it "safer" if it's done stepwise? i.e., backup, reset, upgrade, restore?15:12
damiandabrowskiin my case I just removed mnesia DB on second and third rabbitmq node15:12
NeilHanlonor at least mention in release notes how to recover, and to make backups if needed15:12
noonedeadpunkdamiandabrowski: yes, totally worth having a release note with that15:13
noonedeadpunkI'm not kinda afraid of Bobcat upgrades due to that15:13
noonedeadpunkas quite some flags are required from 2.3.1215:13
noonedeadpunk*3.1215:13
noonedeadpunkand totally affecting 2023.1 with 3.11 requirements15:15
noonedeadpunkother then that - I have proposed role branching already. so any potential fix will come as a backport to 2023.215:16
noonedeadpunkI hope roles will be branched really soonish and we can have tagged before deadline...15:17
noonedeadpunkDespite we didn't have plenty of work - that made me feel relaxed and still we're on deadline :( So this time it's totally my fault and meantal gap kinda15:18
NeilHanlonthanks for doing all that. Appreciate you!15:21
noonedeadpunkIn terms of outstanding patches that we need to land for 2023.215:23
noonedeadpunkI think we might have missed some important bits actually15:24
noonedeadpunkLike Ironic might be failing when deployed in LXC without this: https://review.opendev.org/c/openstack/openstack-ansible-os_ironic/+/89825315:24
opendevreviewDmitriy Rabotyagov proposed openstack/openstack-ansible-haproxy_server master: Convert haproxy_service_config_external.yml to a symlink  https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/89784115:24
noonedeadpunkthen this would also made sense to appear for 2023.2 in octavia role: https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/88551915:25
noonedeadpunkcompatability matrix: https://review.opendev.org/c/openstack/openstack-ansible/+/90119115:25
noonedeadpunkAnd this thing: https://review.opendev.org/c/openstack/openstack-ansible/+/90153415:27
noonedeadpunkI'm not 100% sure about it frankly speaking. It sounds reasonable and help handle CMQv2 logic15:27
noonedeadpunkBut resulting readability is not perfect15:28
noonedeadpunkAnd I guess it should be somehow documented separately along with quorum queues15:28
noonedeadpunkI think that this is pretty much everything I had on the table for today15:35
NeilHanlonmay not be a lot of _stuff_ this round, but it's complicated... that's for sure15:35
NeilHanlon#endmeeting16:03
opendevmeetMeeting ended Tue Dec  5 16:03:24 2023 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)16:03
opendevmeetMinutes:        https://meetings.opendev.org/meetings/openstack_ansible_meeting/2023/openstack_ansible_meeting.2023-12-05-15.01.html16:03
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/openstack_ansible_meeting/2023/openstack_ansible_meeting.2023-12-05-15.01.txt16:03
opendevmeetLog:            https://meetings.opendev.org/meetings/openstack_ansible_meeting/2023/openstack_ansible_meeting.2023-12-05-15.01.log.html16:03
NeilHanlonthanks folks <316:03
noonedeadpunkah, thanks NeilHanlon :)16:09
opendevreviewMerged openstack/openstack-ansible master: Add compatability matrix for 2023.2 (Bobcat)  https://review.opendev.org/c/openstack/openstack-ansible/+/90119117:01
opendevreviewMerged openstack/openstack-ansible-haproxy_server stable/zed: Fix haproxy_stats SSL path defenition  https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/90253318:14
opendevreviewMerged openstack/openstack-ansible-ops master: Upgrade Grafana deployment to Grafana collection  https://review.opendev.org/c/openstack/openstack-ansible-ops/+/90261818:35
opendevreviewMerged openstack/openstack-ansible master: Allow installing collections from repos containing more than one  https://review.opendev.org/c/openstack/openstack-ansible/+/90147121:18

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!