| opendevreview | Pierre Riteau proposed openstack/kayobe master: CI: Add ansible-role-jobs template https://review.opendev.org/c/openstack/kayobe/+/951011 | 06:01 |
|---|---|---|
| opendevreview | Joachim de Groot proposed openstack/kolla master: Fix glance-api: add missing lsscsi package https://review.opendev.org/c/openstack/kolla/+/951090 | 07:51 |
| opendevreview | Rafal Lewandowski proposed openstack/kolla-ansible master: Enable ML2/OVN by default https://review.opendev.org/c/openstack/kolla-ansible/+/904959 | 09:04 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [rabbitmq] Use stop instead of drain on single-node https://review.opendev.org/c/openstack/kolla-ansible/+/951100 | 09:09 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: Move to CentOS Stream 10 https://review.opendev.org/c/openstack/kolla/+/950392 | 09:34 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [rabbitmq] Use stop instead of drain on single-node https://review.opendev.org/c/openstack/kolla-ansible/+/951100 | 10:24 |
| opendevreview | Merged openstack/kayobe master: Fix syntax error in drac-pxe role https://review.opendev.org/c/openstack/kayobe/+/951001 | 10:33 |
| opendevreview | Merged openstack/kayobe master: Configure ansible-lint and fix issues found https://review.opendev.org/c/openstack/kayobe/+/951006 | 10:49 |
| opendevreview | Merged openstack/kayobe master: CI: Add ansible-role-jobs template https://review.opendev.org/c/openstack/kayobe/+/951011 | 10:49 |
| opendevreview | Piotr Milewski proposed openstack/kolla-ansible master: masakari: Add support for using uWSGI https://review.opendev.org/c/openstack/kolla-ansible/+/951115 | 10:57 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/2025.1: Fix syntax error in drac-pxe role https://review.opendev.org/c/openstack/kayobe/+/951118 | 11:08 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/2024.2: Fix syntax error in drac-pxe role https://review.opendev.org/c/openstack/kayobe/+/951120 | 11:10 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/2024.1: Fix syntax error in drac-pxe role https://review.opendev.org/c/openstack/kayobe/+/951122 | 11:11 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/2023.2: Fix syntax error in drac-pxe role https://review.opendev.org/c/openstack/kayobe/+/951123 | 11:11 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: Move to CentOS Stream 10 https://review.opendev.org/c/openstack/kolla/+/950392 | 11:26 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: Move to CentOS Stream 10 https://review.opendev.org/c/openstack/kolla/+/950392 | 11:27 |
| opendevreview | Joachim de Groot proposed openstack/kolla master: Fix glance-api: add missing lsscsi package https://review.opendev.org/c/openstack/kolla/+/951090 | 11:28 |
| opendevreview | Joachim de Groot proposed openstack/kolla master: Fix glance-api: add missing lsscsi package https://review.opendev.org/c/openstack/kolla/+/951090 | 11:33 |
| kevko | mnasiadka: I've fixed kolla-ansible-debian-upgrade :) | 11:40 |
| mnasiadka | by? | 11:40 |
| kevko | mnasiadka: https://review.opendev.org/c/openstack/kolla-ansible/+/951100 | 11:40 |
| mnasiadka | kevko: open a bug, I doubt it's only on master ;) | 11:41 |
| kevko | mnasiadka: I will... | 11:42 |
| kevko | mnasiadka: btw, we are not waiting for drain ..should I open two separate bugs or one bug and two patches with partial and closes ? what do you mean ? | 11:43 |
| mnasiadka | it's up to you, I just think we need A BUG to do backports ;-) | 11:43 |
| kevko | mnasiadka: okay, cool, understand | 11:44 |
| kevko | mnasiadka: just one question ... | 11:51 |
| kevko | mnasiadka: should I wait for check if drained in container until it will pass .... | 11:52 |
| kevko | mnasiadka: or run script from ansible with delay and retry and until ... | 11:52 |
| mnasiadka | I guess wait should time out after X minutes, right? | 11:52 |
| kevko | mnasiadka: point is that in first approach it take time to complete ...and operator is waiting .... | 11:52 |
| kevko | mnasiadka: and operator has a chance to go to container and verify manually ..even during upgrade ... | 11:53 |
| kevko | mnasiadka: second approach will fail after some retires ...right ... | 11:53 |
| kevko | mnasiadka: so operator can rerun upgrade playbook ... | 11:53 |
| kevko | mnasiadka: in both cases you can ctlr+c and solve ...or rerun playbook right ... | 11:53 |
| kevko | mnasiadka: Yeah, but drain can be seconds ...but can be minutes or half hour on busy big clouds ..as i have for example ... | 11:54 |
| mnasiadka | huh, half an hour | 11:55 |
| mnasiadka | it would be good to have some output that the drain has still not finished - maybe with block/rescue or block/always? | 11:55 |
| kevko | mnasiadka: why not ...hundreds of computes ? | 11:55 |
| kevko | mnasiadka: Of course, I'm thinking in theoretical terms , actually I can't really imagine how long it could take to move leader... on my empty cloud it always succeeds on the first try, without needing a retry. | 11:57 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: Move to CentOS Stream 10 https://review.opendev.org/c/openstack/kolla/+/950392 | 11:59 |
| kevko | mnasiadka: Maybe I could implement a heuristic inside the script — check the output of the previous run in the loop and compare it with the current run, to see how many leaders have moved... and for example, if no leaders move for 5 consecutive runs, then fail. That way Ansible will get proper feedback and fail, and I won’t have to rely on | 11:59 |
| kevko | retry, delay, and until... yeah, that would be the cleanest solution | 11:59 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: Move to CentOS Stream 10 https://review.opendev.org/c/openstack/kolla/+/950392 | 12:11 |
| kevko | mnasiadka: is it possible somehow configure in CI that "if you are doing upgrade from previous release ...build image from depends-on which is previous release new patch in kolla" and do it ... | 12:32 |
| mnasiadka | nothing is impossible ;) | 12:32 |
| kevko | mnasiadka: point is that if i want to fix "wait for drain" ..i can't do it in ansible ...or...I can actually ...but i need to copy script to destination host ...and then i need to copy that script inside container ..and then i need to run it in current not restarted container ... | 12:33 |
| kevko | mnasiadka: but nicer is to add script to images ...wait for build at night ...and then just test fix with kolla-ansible ...did you got it ? ... | 12:33 |
| kevko | maybe i am not exlaining good :D | 12:34 |
| kevko | by proper way .. | 12:34 |
| opendevreview | Michal Arbet proposed openstack/kolla master: [rabbitmq] Add script for check if node is drained https://review.opendev.org/c/openstack/kolla/+/951126 | 12:35 |
| kevko | mnasiadka: well, I can override image to the url where I will manually build image with change :D ... | 12:36 |
| opendevreview | Piotr Milewski proposed openstack/kolla-ansible stable/2025.1: Unnecessary comma in kolla-tolbox configuration - external rabbitmq https://review.opendev.org/c/openstack/kolla-ansible/+/951127 | 12:36 |
| kevko | i mean full_image variable | 12:36 |
| kevko | mnasiadka: aaa, nevermind ...i will do it in kolla-ansible way ..but don't like it | 12:37 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: Move to CentOS Stream 10 https://review.opendev.org/c/openstack/kolla/+/950392 | 12:44 |
| mnasiadka | mnasiadka bbezak frickler kevko mmalchuk gkoper jovial mattcrees dougszu darmach pabloclsn ravlew amir58118 - meeting in 9 minutes | 12:51 |
| kevko | \o/ | 12:59 |
| mnasiadka | #startmeeting kolla | 13:00 |
| opendevmeet | Meeting started Wed May 28 13:00:07 2025 UTC and is due to finish in 60 minutes. The chair is mnasiadka. Information about MeetBot at http://wiki.debian.org/MeetBot. | 13:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 13:00 |
| opendevmeet | The meeting name has been set to 'kolla' | 13:00 |
| mnasiadka | #topic rollcall | 13:00 |
| mnasiadka | kevko: now ;) | 13:00 |
| mnasiadka | o/ | 13:00 |
| bbezak | o/ | 13:00 |
| ravlew | o7 | 13:01 |
| jhorstmann | o/ | 13:02 |
| mmalchuk | \o | 13:02 |
| *** ravlew is now known as Guest16917 | 13:02 | |
| mnasiadka | https://www.irccloud.com/pastebin/68Nzqjsh/ | 13:02 |
| mnasiadka | oops | 13:02 |
| mnasiadka | wrong button | 13:02 |
| mnasiadka | * CI status | 13:02 |
| mnasiadka | * Release tasks | 13:02 |
| mnasiadka | * Regular stable releases (first meeting in a month) | 13:02 |
| mnasiadka | * Current cycle planning | 13:02 |
| mnasiadka | * Additional agenda (from whiteboard) | 13:02 |
| mnasiadka | * Open discussion | 13:02 |
| mnasiadka | #topic CI status | 13:02 |
| mnasiadka | I assume green | 13:02 |
| mnasiadka | kevko: found some issues with Debian upgrades | 13:03 |
| mnasiadka | but I think they just manifest strong on Debian ;-) | 13:03 |
| mnasiadka | and we don't have any CentOS/Rocky testing at the moment. | 13:03 |
| mnasiadka | #topic Release tasks | 13:03 |
| mnasiadka | It's R-18 | 13:04 |
| mnasiadka | We already switched master to master | 13:04 |
| mnasiadka | So I think that's it | 13:04 |
| mnasiadka | #topic Current cycle planning | 13:04 |
| mnasiadka | Anybody wants to talk about any new features or other work for this cycle? | 13:04 |
| darmach | \o | 13:05 |
| mnasiadka | Oh well | 13:05 |
| darmach | (technical difficulties) | 13:05 |
| mnasiadka | I've done some CS10 related work, some packages are missing in EL10 world, like mod_auth_mellon for keystone - pursuing that to be added to EPEL | 13:06 |
| mnasiadka | Additional agenda is empty | 13:06 |
| mnasiadka | #topic Open discussion | 13:06 |
| mnasiadka | Anybody anything? | 13:06 |
| kevko | mnasiadka: what issues ? | 13:07 |
| mnasiadka | Well, I only mentioned you found some issues :) | 13:07 |
| mnasiadka | with RMQ | 13:07 |
| kevko | aaa ...there is an ':' | 13:07 |
| kevko | :D | 13:07 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Check drain while upgrading rabbitmq https://review.opendev.org/c/openstack/kolla-ansible/+/951133 | 13:07 |
| kevko | I would like to merge several patches today ...because now CI is blocked ... | 13:08 |
| kevko | because of debian-upgrade ... (but it's global problem ...but in debian it's visible ..don't know why ) | 13:08 |
| kevko | https://review.opendev.org/c/openstack/kolla-ansible/+/951100 | 13:09 |
| mnasiadka | we'll keep an eye out on these changes | 13:10 |
| mnasiadka | right bbezak ? | 13:10 |
| bbezak | sure ;) | 13:11 |
| mnasiadka | ok then, I see no other topics | 13:12 |
| kevko | Thank you... but even if not... I'm enough of a spammer, sorry in advance :) | 13:12 |
| mnasiadka | let's close for today - thanks for coming! | 13:12 |
| mnasiadka | #endmeeting | 13:12 |
| opendevmeet | Meeting ended Wed May 28 13:12:21 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 13:12 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/kolla/2025/kolla.2025-05-28-13.00.html | 13:12 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/kolla/2025/kolla.2025-05-28-13.00.txt | 13:12 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/kolla/2025/kolla.2025-05-28-13.00.log.html | 13:12 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Check drain while upgrading rabbitmq https://review.opendev.org/c/openstack/kolla-ansible/+/951133 | 13:12 |
| mmalchuk | mnasiadka thanks | 13:14 |
| Guest16917 | thanks | 13:14 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: opensearch: Switch to recover_after_data_nodes and expected_data_nodes https://review.opendev.org/c/openstack/kolla-ansible/+/949478 | 13:17 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Update OpenSearch and OpenSearch Dashboards to version 3.x https://review.opendev.org/c/openstack/kolla/+/949444 | 13:18 |
| mnasiadka | Ok, let's see if that passes now. | 13:18 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [rabbitmq] Use stop instead of drain on single-node https://review.opendev.org/c/openstack/kolla-ansible/+/951100 | 13:26 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Check drain while upgrading rabbitmq https://review.opendev.org/c/openstack/kolla-ansible/+/951133 | 13:26 |
| kevko | bbezak, mnasiadka: https://review.opendev.org/c/openstack/kolla-ansible/+/951100 << gate unblocker, it will pass :) , added reno and closes-bug | 13:27 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Check drain while upgrading rabbitmq https://review.opendev.org/c/openstack/kolla-ansible/+/951133 | 13:32 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [rabbitmq] Use stop instead of drain on single-node https://review.opendev.org/c/openstack/kolla-ansible/+/951100 | 14:13 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Check drain while upgrading rabbitmq https://review.opendev.org/c/openstack/kolla-ansible/+/951133 | 14:13 |
| fungi | greetings kolla folks! i would like to draw your attention to this announcement from opendev about allowing gerrit's hashtags feature globally for all users on all projects: https://lists.opendev.org/archives/list/service-announce@lists.opendev.org/thread/SVXT6X4WRYE6XQPB5PTWFUKUATICLO56/ | 16:22 |
| fungi | https://review.opendev.org/c/openstack/project-config/+/802744 set it up restricted to core reviewers in kolla repositories several years ago, so i wanted to make sure there weren't concerns or a need to make any preparation before that goes into effect next week | 16:23 |
| kevko | mnasiadka bbezak : so can we move on ? https://review.opendev.org/c/openstack/kolla-ansible/+/951100 << passed | 16:26 |
| opendevreview | Merged openstack/kolla-ansible master: opensearch: Switch to recover_after_data_nodes and expected_data_nodes https://review.opendev.org/c/openstack/kolla-ansible/+/949478 | 17:40 |
| mnasiadka | fungi: that’s fine :) | 18:03 |
| fungi | thanks for checking! lots of openstack teams already allow the hashtags feature for all users, and seem not to have run into any problems with it, but i just wanted to make sure you're aware since kolla was really the only project team with restrictive rules for it in their acl | 18:04 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Check drain while upgrading rabbitmq https://review.opendev.org/c/openstack/kolla-ansible/+/951133 | 18:09 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [DNM] Just test https://review.opendev.org/c/openstack/kolla-ansible/+/951193 | 18:50 |
| sean-k-mooney | o/ just an fyi i noticed that kolla is one of the future proejct that had ara_reportign configured in its jobs but i checked and it also seam to be not working anymore | 18:54 |
| sean-k-mooney | was that intentionall or did it break at some point and no one noticed bcasue it was not being used | 18:55 |
| sean-k-mooney | oh | 18:55 |
| sean-k-mooney | https://opendev.org/openstack/kolla-ansible/src/branch/master/tests/ara-readme.md.j2#L33 | 18:55 |
| sean-k-mooney | Yes! If the commit message contains the string `#ara_verbose` then Zuul will | 18:56 |
| sean-k-mooney | render the html file and collect it in the logs. Beware: this adds hundreds of | 18:56 |
| sean-k-mooney | MiB to the log files | 18:56 |
| sean-k-mooney | i see | 18:56 |
| sean-k-mooney | so the jobs can produce them but i guess ye dont do that by default ebcause well 100s of MBs of extra data per run is not great | 18:57 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: [rabbitmq] Use stop instead of drain on single-node https://review.opendev.org/c/openstack/kolla-ansible/+/951100 | 18:57 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Check drain while upgrading rabbitmq https://review.opendev.org/c/openstack/kolla-ansible/+/951133 | 18:57 |
| mnasiadka | sean-k-mooney: I think it would be better if OpenDev would have a central ara server - but nobody did that work | 18:57 |
| sean-k-mooney | zuul kidn of gained the ablity to analses it for you too | 18:59 |
| sean-k-mooney | i never use the console tab but it kind of does the teask breakdown for you | 18:59 |
| sean-k-mooney | so like https://zuul.opendev.org/t/openstack/build/8bae983c4f7a4ce99920a85775ae056b/console | 19:00 |
| sean-k-mooney | you can even link to sepcific tasks https://zuul.opendev.org/t/openstack/build/8bae983c4f7a4ce99920a85775ae056b/console#2/1/42/primary | 19:01 |
| sean-k-mooney | the reason i was lookign into this is our dwonstream was wondering about how to do an analasy of tasks that fail often and were considrign using the ansble xml plugin for that | 19:02 |
| sean-k-mooney | i was wondering if ara woudl be better | 19:02 |
| mnasiadka | Haven’t personally used that a long time, so can’t tell | 19:08 |
| kevko | mnasiadka: neverending story with CI ...now docker package is broken :( | 19:15 |
| kevko | podman is OK | 19:15 |
| kevko | mnasiadka: aaaaa | 19:16 |
| kevko | mnasiadka: check zull online status for current running change 951193 (which is actually test: me in defaults to trigger all tests ) | 19:17 |
| kevko | no change | 19:17 |
| frickler | docker-ce 5:28.2.0-1~debian.12~bookworm, hot off the press, didn't even make it into renos yet https://docs.docker.com/engine/release-notes/28/ | 19:26 |
| kevko | frickler: but it's already in repo :D right ? | 19:31 |
| kevko | frickler: package is not creating docker group :( | 19:32 |
| mnasiadka | Lol, we need to pin something again?:) | 19:33 |
| kevko | mnasiadka: docker-ce is already in repo ..and it's broken :( | 19:33 |
| kevko | mnasiadka: tested now on my localhost :( /var/log/syslog:2025-05-28T21:30:05.877259+02:00 pixla delgroup[94676]: Removing group `docker' ... | 19:33 |
| kevko | /var/log/syslog:2025-05-28T21:30:40.207214+02:00 pixla (sd-chown)[95140]: docker.socket: Failed to resolve group docker: No such process | 19:34 |
| kevko | mnasiadka: cloning docker-ce packaging :D | 19:34 |
| kevko | https://github.com/docker/docker-ce-packaging/commit/8c5e99fd383b461151ef0f1e1875d1801a175c80 | 19:35 |
| mnasiadka | systemd-sysusers? Let me guess - Debian doesn’t use it?;) | 19:35 |
| kevko | mnasiadka: ubuntu same :) | 19:36 |
| kevko | mnasiadka: need to fix in a-c-k | 19:40 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: Move to CentOS Stream 10 https://review.opendev.org/c/openstack/kolla/+/950392 | 19:43 |
| kevko | mnasiadka frickler: ah, ok ... it looks like it will be fixed in short time | 19:44 |
| kevko | https://github.com/docker/docker-ce-packaging/issues/1209 | 19:44 |
| kevko | Even if they fix the package ..i am wondering if it's not needed also to fix in kolla-ansible and a-c-k | 19:51 |
| mnasiadka | Seems they just need to actually run systemd-sysusers to create the user… | 19:52 |
| kevko | mnasiadka: yeah, | 20:23 |
| kevko | mnasiadka: but actually it's also our bug :) | 20:26 |
| kevko | we are not masking docker.socket | 20:26 |
| opendevreview | Pierre Riteau proposed openstack/kayobe master: CI: Bump ansible-lint and prune skip list https://review.opendev.org/c/openstack/kayobe/+/951204 | 20:56 |
| -opendevstatus- NOTICE: The Gerrit service on review.opendev.org is temporarily unreachable due to an ongoing issue in the hosting provider where it resides | 22:07 | |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!