| Core1553 | Should we use consider the possibility of adding vector for log forwarding along with fluentd? VRL parser seems so initiative for parsing logs on the edge. | 05:54 |
|---|---|---|
| SvenKieske | Core1553: is "vector" another log shipping product? If you want to add it, propose a patch, I would say. I don't use or know it :) | 07:18 |
| SvenKieske | cores, reviews for this trivial bugfix for keystone federation would be appreciated: https://review.opendev.org/c/openstack/kolla-ansible/+/913908 (see also the link on whiteboard) | 07:18 |
| bbezak | kevko, frickler: please take a look - to unblock CI in antelope/zed/yoga https://review.opendev.org/c/openstack/kolla/+/921280 | 09:10 |
| SvenKieske | bbezak, kevko, frickler, mnasiadka: maybe we should stop merging patches as well? not sure this affects us: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/ORQB3WFFA67UUAF5JNRKGYWB2U2MXHGJ/ | 09:16 |
| SvenKieske | tl;dr: gate pipeline doesn't run all jobs it should be running, at least for neutron, need to look into it if this affects us | 09:16 |
| SvenKieske | ok it doesn't look like we use "negate:" anywhere in kolla or kolla-ansible | 09:21 |
| SvenKieske | looking into ack now | 09:21 |
| SvenKieske | also clean | 09:22 |
| opendevreview | Pierre Riteau proposed openstack/kayobe master: CI: Build default images in experimental jobs https://review.opendev.org/c/openstack/kayobe/+/921065 | 09:34 |
| SvenKieske | mhm, locally I get "pyflakes" failed during execution due to "'FlakesChecker' object has no attribute 'NAMEDEXPR'" when running "tox -e pep8" | 10:32 |
| SvenKieske | anybody seen this? | 10:33 |
| opendevreview | Sven Kieske proposed openstack/kolla-ansible master: hardening horizon: don't mount hosts /tmp https://review.opendev.org/c/openstack/kolla-ansible/+/921371 | 10:34 |
| opendevreview | Merged openstack/kolla stable/2023.2: Drop packaging requirement https://review.opendev.org/c/openstack/kolla/+/921272 | 10:49 |
| opendevreview | Merged openstack/kolla stable/2023.1: Pin requests to <2.32 for docker sdk (2023.1 and older releases) https://review.opendev.org/c/openstack/kolla/+/921280 | 10:51 |
| opendevreview | Merged openstack/kayobe master: Add ironic config files clean up tasks to overcloud deprovision https://review.opendev.org/c/openstack/kayobe/+/920546 | 10:51 |
| opendevreview | Maksim Malchuk proposed openstack/kolla stable/2023.2: swift-base: fix rsync missing for rpm installs https://review.opendev.org/c/openstack/kolla/+/916770 | 10:52 |
| opendevreview | Maksim Malchuk proposed openstack/kolla stable/2023.1: swift-base: fix rsync missing for rpm installs https://review.opendev.org/c/openstack/kolla/+/916771 | 10:52 |
| opendevreview | Rafal Lewandowski proposed openstack/kayobe stable/2023.2: Add ironic config files clean up tasks to overcloud deprovision https://review.opendev.org/c/openstack/kayobe/+/921375 | 11:33 |
| opendevreview | Merged openstack/kayobe master: Add seed service destroy https://review.opendev.org/c/openstack/kayobe/+/911702 | 12:54 |
| opendevreview | Merged openstack/kayobe master: Update instructions to migrate from CentOS Stream 8 https://review.opendev.org/c/openstack/kayobe/+/921014 | 12:54 |
| bbezak | mgoddard mnasiadka bbezak frickler kevko SvenKieske mmalchuk gkoper jangutter jsuazo jovial osmanlicilegi mattcrees dougszu kolla meeting in 5! | 12:55 |
| bbezak | #startmeeting kolla | 13:00 |
| opendevmeet | Meeting started Wed Jun 5 13:00:20 2024 UTC and is due to finish in 60 minutes. The chair is bbezak. Information about MeetBot at http://wiki.debian.org/MeetBot. | 13:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 13:00 |
| opendevmeet | The meeting name has been set to 'kolla' | 13:00 |
| bbezak | #topic rollcall | 13:00 |
| SvenKieske | o/ | 13:00 |
| jovial[m] | o/ | 13:00 |
| mmalchuk | o/ | 13:00 |
| frickler | \o | 13:00 |
| mhiner | o/ | 13:00 |
| jhorstmann | o/ | 13:01 |
| r-krcek | 0/ | 13:01 |
| bbezak | #topic agenda | 13:02 |
| bbezak | * Roll-call | 13:02 |
| bbezak | * Agenda | 13:02 |
| bbezak | * Announcements | 13:02 |
| bbezak | * Review action items from the last meeting | 13:02 |
| bbezak | * CI status | 13:02 |
| bbezak | * Release tasks | 13:02 |
| bbezak | * Regular stable releases (first meeting in a month) | 13:03 |
| bbezak | * Current cycle planning | 13:03 |
| bbezak | * Additional agenda (from whiteboard) | 13:03 |
| bbezak | * Open discussion | 13:03 |
| bbezak | #topic CI status | 13:03 |
| bbezak | periodics jobs are failing | 13:04 |
| bbezak | possible fix - https://review.opendev.org/c/openstack/kolla/+/921032. I'm working on bumping community.docker in CI | 13:04 |
| bbezak | test change of periodics jobs will follow after the meeting | 13:04 |
| jovial[m] | I think we are all green in kayobe land | 13:05 |
| bbezak | today we fixed bobcat and antelope for kolla | 13:05 |
| bbezak | great | 13:06 |
| bbezak | I think except periodics we're OK | 13:06 |
| SvenKieske | why do the periodic jobs need another fix? I don't understand the comment in your commit message | 13:06 |
| SvenKieske | https://opendev.org/openstack/kolla/src/commit/968711579fbe73316227bd2b1993c5656adad9f4/tests/playbooks/publish.yml#L8 this explictly tells that it's using docker sdk (I guess they mean docker-py?) from kolla venv | 13:07 |
| bbezak | publish.yml is the only place where we're using community.docker collection | 13:07 |
| bbezak | and apparently it also needs to be bumped | 13:07 |
| bbezak | and publish.yml is being used only in periodics | 13:07 |
| bbezak | docker_host_info: | 13:08 |
| bbezak | for example | 13:08 |
| SvenKieske | okay, unfortunate | 13:08 |
| frickler | maybe creating a dedicated venv for the publish job might be a better solution though? | 13:09 |
| bbezak | maybe | 13:09 |
| bbezak | as ansible version (and default community.docker) is coming from opendev base jobs | 13:10 |
| bbezak | I'll look into it | 13:10 |
| SvenKieske | great | 13:10 |
| bbezak | #topic Release tasks | 13:11 |
| bbezak | how we're doing on Kayobe release jovial[m] ? | 13:11 |
| jovial[m] | Just trying to merge the last few patches and then I was going to propose a release candidate | 13:12 |
| bbezak | ok, let me know if you need to push some changes forward | 13:13 |
| jovial[m] | Any further patches people would like to see in the release? | 13:13 |
| jovial[m] | bbezak, will do - thanks | 13:13 |
| bbezak | I think we should just merge what we can and do a release at this point | 13:15 |
| jovial[m] | Right, we are cutting it fine already :) | 13:15 |
| bbezak | :) | 13:15 |
| bbezak | #topic Regular stable releases (first meeting in a month) | 13:15 |
| bbezak | I'd say let's move stable release when we have kayobe as well | 13:16 |
| bbezak | as we recently released kolla/k-a | 13:16 |
| bbezak | #topic Current cycle planning | 13:17 |
| opendevreview | Will Szumski proposed openstack/kayobe master: Add proxy support to Vagrantfile https://review.opendev.org/c/openstack/kayobe/+/703666 | 13:17 |
| bbezak | I think we should add Dalmatian release and move the priorities there | 13:17 |
| bbezak | I'll look into that | 13:18 |
| bbezak | (on the whiteboard) | 13:18 |
| bbezak | #topic Additional agenda (from whiteboard) | 13:20 |
| bbezak | speaking of federation bug - I somehow don't have this problem in antelope deployment, however I didn't dig into that why is that | 13:22 |
| bbezak | https://etherpad.opendev.org/p/KollaWhiteBoard#L72 | 13:22 |
| bbezak | I added a comment in the change | 13:22 |
| SvenKieske | okay, thanks, I just wanted to get some reviews, I didn't really reproduce it locally (no federation available locally on my laptop), but it seemed like an important bugfix :) | 13:24 |
| SvenKieske | and as far as I could tell the fixed syntax looks more correct to me, I think I even consulted the haproxy docs for that | 13:24 |
| bbezak | yeah it maybe makes sense to update the syntax too | 13:25 |
| SvenKieske | maybe some ansible/jinja shenanigans | 13:26 |
| bbezak | there are two comments of r-krcek | 13:27 |
| bbezak | https://etherpad.opendev.org/p/KollaWhiteBoard#L74 | 13:27 |
| SvenKieske | I guess those didn't merge because backportvoting was not done when workflow was set to +1 | 13:28 |
| SvenKieske | I hoped me doing backport +1 would suffice | 13:28 |
| frickler | the first one only needed another W+1 after backport flag was set | 13:28 |
| SvenKieske | maybe toggle that w+1 switch again | 13:28 |
| SvenKieske | that trove thing, does this need a backport? I really like the new switch, honestly. | 13:29 |
| frickler | https://review.opendev.org/c/openstack/kolla-ansible/+/920714 is a big change, might need some more discussion. but I only took a quick look just now | 13:31 |
| r-krcek | Definitely, I'd be glad for any feedback/discussion .) | 13:32 |
| SvenKieske | maybe I can squeeze that in my ever growing list, I'm on the reviewers list already anyway. | 13:33 |
| bbezak | I added some more reviewers | 13:33 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Add certificates for RabbitMQ internode TLS https://review.opendev.org/c/openstack/kolla-ansible/+/921380 | 13:34 |
| SvenKieske | the bug description and the commit message don't really seem to well connected and might need an update as well, but I just took a quick look | 13:35 |
| SvenKieske | and yes, I read both bug reports. | 13:35 |
| SvenKieske | my initial reaction is, this should be split into at least two parts: the refactoring (converting scripts into playbooks) and the bugfixing (removing all datadirs) just going by the commit message and the size of the patch | 13:37 |
| SvenKieske | unless there is a very good reason why both must be done at the same time, of course. | 13:37 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Add certificates for RabbitMQ internode TLS https://review.opendev.org/c/openstack/kolla-ansible/+/921380 | 13:38 |
| r-krcek | SvenKieske: Since the list of containers that will be destroyed is taken from the config, the playbooks wont remove any containers that were not deployed directly by kolla-anisble. | 13:38 |
| SvenKieske | mhm, is that a bug, or a feature? ;) | 13:38 |
| r-krcek | I see, okay, I will split it into multiple. I dont belive there is a direct need to do it at the same time | 13:38 |
| r-krcek | Huh, did not think of if was a feature :O | 13:39 |
| SvenKieske | I at least know of some deployments who run totally separate playbooks (with distinct inventory files) for kolla and "other stuff". I know that's not good, but that's what people seem to do.. | 13:39 |
| SvenKieske | so there might very well be containers not deployed by k-a that you surely don't want k-a to delete :) | 13:39 |
| jovial[m] | I like the move to use playbooks - finally we will be able to use --tags :) | 13:40 |
| SvenKieske | as long as there is an option to turn that feature/bug off, I guess it's fine | 13:40 |
| SvenKieske | ftr: I also like the general idea and the idea to "own" all containers on kolla deployed hosts, but I guess the history of deployments needs some escape switches :) | 13:40 |
| bbezak | we've also got some ihalomi changes on whiteboard | 13:41 |
| SvenKieske | I guess we should write the rest directly into gerrit for preservation of review history :) | 13:41 |
| bbezak | https://etherpad.opendev.org/p/KollaWhiteBoard#L78 | 13:41 |
| ihalomi | the most important is to merge the first patchset https://review.opendev.org/c/openstack/ansible-collection-kolla/+/916258 | 13:44 |
| ihalomi | so debian tests will pass on the docker worker refactor patchset | 13:44 |
| bbezak | ok I'll try to look into that | 13:46 |
| bbezak | and some changes of mhiner as well | 13:47 |
| mhiner | yes, both ready for review | 13:48 |
| bbezak | ok, let's try to review them in following week | 13:49 |
| bbezak | commas one is pretty simple :) | 13:49 |
| bbezak | #topic Open discussion | 13:50 |
| bbezak | anything to discuss? | 13:51 |
| frickler | next PTG dates announced: 2024-10-21 - -25 https://openinfra.dev/ptg/ | 13:51 |
| frickler | just if ppl like to plan early | 13:51 |
| bbezak | thx! | 13:51 |
| mmalchuk | thanks for info | 13:51 |
| SvenKieske | only 4 months to go | 13:52 |
| opendevreview | Matúš Jenča proposed openstack/kolla-ansible master: Add support for RabbitMQ internode tls https://review.opendev.org/c/openstack/kolla-ansible/+/921381 | 13:52 |
| bbezak | :) | 13:52 |
| bbezak | ok let's finish for today then | 13:52 |
| bbezak | #endmeeting | 13:52 |
| opendevmeet | Meeting ended Wed Jun 5 13:52:53 2024 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 13:52 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-06-05-13.00.html | 13:52 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-06-05-13.00.txt | 13:52 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-06-05-13.00.log.html | 13:52 |
| bbezak | thx all! | 13:52 |
| mmalchuk | bbezak thanks | 13:52 |
| SvenKieske | thx | 13:54 |
| opendevreview | Merged openstack/kolla stable/2023.1: swift-base: fix rsync missing for rpm installs https://review.opendev.org/c/openstack/kolla/+/916771 | 14:31 |
| opendevreview | Merged openstack/kolla-ansible master: Fix: Update Skyline stop task https://review.opendev.org/c/openstack/kolla-ansible/+/919697 | 15:03 |
| opendevreview | Verification of a change to openstack/kolla-ansible master failed: Fix trove module imports https://review.opendev.org/c/openstack/kolla-ansible/+/912378 | 15:10 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/2023.2: Update instructions to migrate from CentOS Stream 8 https://review.opendev.org/c/openstack/kayobe/+/921391 | 15:21 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/2023.1: Update instructions to migrate from CentOS Stream 8 https://review.opendev.org/c/openstack/kayobe/+/921392 | 15:22 |
| opendevreview | Merged openstack/kayobe master: CI: Build default images in experimental jobs https://review.opendev.org/c/openstack/kayobe/+/921065 | 15:57 |
| opendevreview | Will Szumski proposed openstack/kayobe master: Add vagrant-libvirt support https://review.opendev.org/c/openstack/kayobe/+/703667 | 16:00 |
| tafkamax | <SvenKieske> "I at least know of some deployme..." <- I am deploying keycloak to a same physical host as kolla-ansible runs on for OIDC auth. | 16:03 |
| SvenKieske | yeah, that is a usual setup, for example :) | 16:04 |
| SvenKieske | it's still difficult, because you have effectively two kinds of cfgmgmt solutions competing for the same host. | 16:04 |
| SvenKieske | I would say the best practice for that would be to wrap k-a inside your own playbook, so you only ever execute one playbook and your outer playbook controls what k-a sees. this is still not bulletproof of course. | 16:05 |
| opendevreview | Will Szumski proposed openstack/kayobe-config master: Sync with kayobe for 2024.1 release https://review.opendev.org/c/openstack/kayobe-config/+/921397 | 16:11 |
| tafkamax | yes it is. Currently docker.json is managed by both, I need to fix it. As we are still not yet in prod. | 16:11 |
| tafkamax | but docker.json is added during bootstrapping | 16:11 |
| tafkamax | Thanks to kolla-ansible i already merged the role in my playbook docker repo setup to deb822 version for futureproofing | 16:12 |
| opendevreview | Will Szumski proposed openstack/kayobe-config-dev master: Sync with kayobe for 2024.1 release https://review.opendev.org/c/openstack/kayobe-config-dev/+/921398 | 16:14 |
| opendevreview | Merged openstack/kayobe-config master: Sync with kayobe for 2024.1 release https://review.opendev.org/c/openstack/kayobe-config/+/921397 | 16:28 |
| greatgatsby | Hello. We performed upgrades on our controllers and rebooted them one at a time. Our rabbitmq logs are now flooded with queue.declare channel exception not_found errors for all the non-durable queues and our services are not functioning. Did we miss something we should do when we need to restart a controller? We do use kolla-ansible to stop all the containers, and confirm RMQ is in maintenance | 18:50 |
| greatgatsby | mode and drain it before the reboot. | 18:50 |
| opendevreview | Merged openstack/kayobe-config-dev master: Sync with kayobe for 2024.1 release https://review.opendev.org/c/openstack/kayobe-config-dev/+/921398 | 18:55 |
| opendevreview | Verification of a change to openstack/kolla stable/2023.2 failed: swift-base: fix rsync missing for rpm installs https://review.opendev.org/c/openstack/kolla/+/916770 | 19:06 |
| opendevreview | Verification of a change to openstack/kolla-ansible master failed: Fix trove module imports https://review.opendev.org/c/openstack/kolla-ansible/+/912378 | 19:24 |
| opendevreview | Merged openstack/kolla-ansible master: Fix trove module imports https://review.opendev.org/c/openstack/kolla-ansible/+/912378 | 22:23 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!