| opendevreview | Verification of a change to openstack/kolla-ansible stable/2024.1 failed: CI: Use libpod/registry for registry https://review.opendev.org/c/openstack/kolla-ansible/+/935994 | 00:34 |
|---|---|---|
| opendevreview | Merged openstack/kolla-ansible master: Fix TLS settings when letsencrypt turned on https://review.opendev.org/c/openstack/kolla-ansible/+/925971 | 01:44 |
| opendevreview | Merged openstack/kolla-ansible stable/2024.1: CI: Use libpod/registry for registry https://review.opendev.org/c/openstack/kolla-ansible/+/935994 | 03:14 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: nova: Add support for using uWSGI https://review.opendev.org/c/openstack/kolla-ansible/+/935975 | 05:18 |
| *** MikeCTZA_ is now known as MikeCTZA | 06:44 | |
| SvenKieske | this is an unlucky patch, 7 rechecks already :( https://review.opendev.org/c/openstack/kolla-ansible/+/924145 | 09:01 |
| kevko | SvenKieske: haha, well yeah .. | 09:35 |
| parasitid | hi all, i'm not very used to kolla so maybe i'm doing something wrong. when i use kolla 19.0.0, i hit an issue i dont have with 17.5.0 | 10:07 |
| parasitid | INFO:kolla.common.utils.openstack-base:Building started at 2024-11-27 10:03:37.611800 ERROR:kolla.common.utils.openstack-base:Unsafe filenames in archive ./kolla-tmp/docker/openstack-base/openstack-base-plugin-pycadf-archive | 10:07 |
| parasitid | ERROR:kolla.common.utils.openstack-base:Failed turning any plugins into a plugins archiv | 10:07 |
| parasitid | is this a known issue ? thanks a lot | 10:07 |
| parasitid | my kolla build conf is pretty simple and simply contains a profile to build designate images on ubuntu 24.04 | 10:09 |
| kevko | parasitid: try to build from git sources | 10:13 |
| kevko | parasitid: what distro you are trying to build | 10:15 |
| kevko | lemme try | 10:17 |
| parasitid | kevko: what do you mean by from sources ? | 10:24 |
| parasitid | kevko: https://pastebin.com/twcKLn6g | 10:25 |
| parasitid | here's my whole kollabuild.conf | 10:26 |
| parasitid | kevko: pip freeze | grep -i kolla | 10:26 |
| parasitid | kolla==19.0.0 | 10:26 |
| opendevreview | Merged openstack/kolla-ansible stable/2023.2: Fix IPv6 address format in ProxySQL configuration https://review.opendev.org/c/openstack/kolla-ansible/+/936243 | 10:28 |
| opendevreview | Merged openstack/kolla-ansible stable/2024.2: Fix external ceph cinder keyring https://review.opendev.org/c/openstack/kolla-ansible/+/936278 | 10:28 |
| parasitid | kevko: it's weird as when i try to tar tvf the pycadf archive file, it works and doesnt seem at first glance to contain any suspicious filename | 10:28 |
| kevko | parasitid: i've just build openstack-base and it's working without problem | 10:29 |
| kevko | parasitid: try to remove docker build cache and all kolla images and try from the scratch | 10:30 |
| parasitid | kevko: still failing | 10:38 |
| kevko | let me try with ubuntu | 10:40 |
| kevko | i've build debian | 10:40 |
| parasitid | https://pastebin.com/dw3sfSKN | 10:40 |
| parasitid | kevko: trying debian 12 | 10:41 |
| parasitid | kevko: i hit the same issue with debian12 | 10:43 |
| kevko | SvenKieske: implemented letsencrypt for proxysql internal VIP :) | 10:43 |
| kevko | SvenKieske: and working :) | 10:43 |
| parasitid | kevko: PRETTY_NAME="Debian GNU/Linux 12 (bookworm)" | 10:49 |
| SvenKieske | kevko: maybe you got an idea why let's encrypt job is failing in https://review.opendev.org/c/openstack/kolla-ansible/+/933924 ? I have a suspicion we have a bug in our nova-cell bootstrap logic, but not sure as I'm not that familiar with nova-cell architecture just yet | 10:49 |
| parasitid | i'm trying to build debian12 on a debian12 | 10:49 |
| kevko | SvenKieske: of course | 10:52 |
| parasitid | kevko: i think i spot it | 10:54 |
| parasitid | i have a 'kolla-tmp' dir in CWD | 10:54 |
| parasitid | and the .venv/lib/python3.11/site-packages/kolla/image/tasks.py:def _test_malicious_tarball(archive, path): | 10:55 |
| kevko | what :D ? | 10:55 |
| parasitid | seems to be broken | 10:55 |
| parasitid | /home/debian/dyonisos/kolla-tmp/docker/openstack-base/plugins/pycadf-3.1.1 | 10:56 |
| parasitid | ./kolla-tmp/docker/openstack-base/plugins | 10:56 |
| parasitid | ERROR:kolla.common.utils.openstack-base:Unsafe filenames in archive ./kolla-tmp/docker/openstack-base/openstack-base-plugin-pycadf-archive | 10:56 |
| kevko | parasitid: i still can't reproduce .. | 10:56 |
| parasitid | kolla-build --work-dir ./kolla-tmp/ | 10:57 |
| parasitid | kevko: that was it | 11:00 |
| parasitid | kevko: kolla-build --work-dir ./kolla-tmp/ fails, whereas kolla-build --work-dir $PWD/kolla-tmp/ works | 11:01 |
| parasitid | kevko: in 17.5.0, there was no such issue | 11:01 |
| kevko | parasitid: hmm, i am not using work dir :/ | 11:02 |
| kevko | parasitid: for what is it ? | 11:02 |
| parasitid | __ /tmp part is too small :) | 11:03 |
| SvenKieske | that's a security function (in tasks.py) which is correct. | 11:03 |
| SvenKieske | I guess you need to just supply an absolute path then to --work-dir, right? | 11:04 |
| parasitid | i think maybe: | 11:04 |
| parasitid | if not os.path.abspath(os.path.join(path, n)).startswith(path): | 11:04 |
| parasitid | should be | 11:04 |
| parasitid | if not os.path.abspath(os.path.join(path, n)).startswith(os.path.abspath(path)): | 11:05 |
| parasitid | no ? | 11:05 |
| SvenKieske | I don't think so, afaik this code is copied from pythons upstream checking code for insecure tarballs, but I would have to check | 11:05 |
| SvenKieske | introduced by: https://review.opendev.org/c/openstack/kolla/+/877611 | 11:07 |
| parasitid | kevko: ok. then maybe the doc should stipulate: --work-dir WORK_DIR Path to be used as working directory. By default, a temporary dir is created | 11:07 |
| parasitid | to use an abspath | 11:07 |
| parasitid | and enforce it is | 11:07 |
| SvenKieske | most of the information is in the original CVE bugreport: https://bugs.launchpad.net/kolla/+bug/1990432 | 11:08 |
| parasitid | kevko: anyway, thanks a lot for the support, i managed to build my images:) | 11:09 |
| SvenKieske | kevko: the second link in https://docs.openstack.org/kolla-ansible/latest/reference/compute/nova-cells-guide.html is dead btw as well (it's redirecting to the first link), but I'm not sure if there is an appropriate replacement. | 11:12 |
| kevko | SvenKieske: i probably know what triggered permissions check fail | 11:12 |
| kevko | SvenKieske: point is that without TLS to mariadb there is a container for bootstrap cell right ? ...it's done on servers[0] .... BUT i've added support to completly switch to TLS ...so i needed to import copy-config role (or what is the name) ... which produced /etc/kolla/nova-cell-bootstrap.... on every host ...but then task which changing | 11:15 |
| kevko | permissions touch only the servers[0] ... so ..before the script was checking all for 0770 .. and when it wasn't on other hosts ..it's ok ...but now it's on every host ...but kolla-ansible change only on first ... | 11:15 |
| kevko | SvenKieske: so i think the solution is fix all permissions ..or import role and add when servers[0] | 11:15 |
| SvenKieske | yep | 11:15 |
| kevko | so the bootstrap folder under /etc/kolla will be created only on server[0] ..as before ... | 11:15 |
| SvenKieske | But I was wondering: that cert directory will end up on all servers anyway, no? so what is fixing permissions before your change? or don't we have certs distributed on all servers before your change? | 11:16 |
| kevko | (but my opinion is that it should be everywhere ...as host[0] is bad from the beginning) | 11:16 |
| SvenKieske | not sure I understand 100% the situation before your patch :D | 11:16 |
| SvenKieske | yep, host[0] is somehow an antipattern I think, don | 11:16 |
| SvenKieske | 't want to grep where we have that everywhere.. | 11:17 |
| kevko | SvenKieske: okay, again ...there is nova-cell-bootstrap container ...which done something OK ? | 11:17 |
| kevko | SvenKieske: but there is only config.json ..nova.conf and that's it ...right ? | 11:17 |
| kevko | SvenKieske: so there is simple task to ensure that files exist ..and then run container which do some work ... | 11:17 |
| kevko | SvenKieske: BUT ... if you are using TLS ... you need to have root.crt imported into nova-cell-bootstrap container when it's boostraping .. | 11:18 |
| kevko | SvenKieske: so I took the common role to achieve cert copy inside /etc/kolla/nova-cell-boostrap -> https://review.opendev.org/c/openstack/kolla-ansible/+/933924/15/ansible/roles/nova-cell/tasks/config_bootstrap.yml | 11:19 |
| kevko | SvenKieske: after this it's working ... but problem is that I missed when: inventory_hostname == groups[nova_cell_conductor_group][0] during the import ... | 11:19 |
| kevko | SvenKieske: so it actually created folder and ca-certificates as it is on my testing stack -> https://paste.openstack.org/show/bkP8JVvqTDvW8EmemRzg/ | 11:20 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add TLS support for MariaDB Connections https://review.opendev.org/c/openstack/kolla-ansible/+/933924 | 11:23 |
| kevko | SvenKieske: TLDR , simply said ...this should fix :D ^^ | 11:23 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: nova: Add support for using uWSGI https://review.opendev.org/c/openstack/kolla-ansible/+/935975 | 11:25 |
| SvenKieske | thx! | 11:39 |
| kevko | hmm, what are we going to do with that dockerhub limits ? :P | 12:04 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: nova: Add support for using uWSGI https://review.opendev.org/c/openstack/kolla-ansible/+/935975 | 12:16 |
| opendevreview | Bartosz Bezak proposed openstack/kayobe master: CI: use quay.io/podman/hello due too dockerhub rate limits https://review.opendev.org/c/openstack/kayobe/+/936130 | 12:29 |
| SvenKieske | kevko: we could try to apply for the OSS dockerhub program (no quotas there it seems) or just move everything somehow to quay.io (no quotas there for OSS stuff as well) | 13:28 |
| SvenKieske | my quay.io account is currently locked due to other stuff, but I should get resolved that soon, so if everything breaks I could try to publish the official ubuntu and debian images there as well | 13:28 |
| SvenKieske | ideally we would have our own caching registry (harbor or something) inside opendev infra | 13:29 |
| kevko | ack | 13:36 |
| kevko | do anybody know if i can 1. have external public network with dns_domain set - shared, 2. zone for that network => Can another tenant create VM on this network and it will be automatically DNS created ? | 13:38 |
| kevko | working / not working ? | 13:38 |
| kevko | i thought that it's | 13:38 |
| kevko | SvenKieske: ?? ^^ | 13:39 |
| SvenKieske | honestly I always need to look up the docs for this stuff :D | 13:40 |
| SvenKieske | this failed like the sixt time with docker limits, giving up for now on rechecks: https://review.opendev.org/c/openstack/kolla-ansible/+/924145 | 13:40 |
| kevko | SvenKieske: so you don't remember if it is working ? | 13:42 |
| SvenKieske | I don't remember if this is the correct config for this. I _guess_ if this is the correct config combination it should work? I'm lucky to not need configure dns settings on a daily basis :D | 13:44 |
| SvenKieske | so purely from your description it should work imho. did it not work, or what is the problem? | 13:45 |
| SvenKieske | kevko: what exactly do you mean with "shared"? you mean the public network has "dns_domain" set to a valid domain? | 13:48 |
| SvenKieske | kevko: I think you want this behaviour? https://docs.openstack.org/neutron/latest/admin/config-dns-int-ext-serv.html#use-case-3c-the-dns-extension | 13:49 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: nova: Add support for using uWSGI https://review.opendev.org/c/openstack/kolla-ansible/+/935975 | 13:57 |
| mnasiadka | mnasiadka bbezak frickler kevko SvenKieske mmalchuk gkoper jangutter jsuazo jovial osmanlicilegi mattcrees dougszu darmach - meeting in 3 minutes | 13:57 |
| kevko | SvenKieske: well, it semiworks ...in log i can see it want to create an record ..but cant access as the zone is in different project ... | 13:59 |
| kevko | SvenKieske: so - don't work | 13:59 |
| mnasiadka | #startmeeting kolla | 14:00 |
| opendevmeet | Meeting started Wed Nov 27 14:00:01 2024 UTC and is due to finish in 60 minutes. The chair is mnasiadka. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| opendevmeet | The meeting name has been set to 'kolla' | 14:00 |
| mnasiadka | #topic rollcall | 14:00 |
| mnasiadka | o/ | 14:00 |
| SvenKieske | ah yeah, if you have created a zone in a different project you can have zone highjacks.. | 14:00 |
| SvenKieske | o/ | 14:00 |
| mmalchuk | o/ | 14:00 |
| kevko | \o/ | 14:00 |
| kevko | SvenKieske: but i thought that for this reason in neutron conf is designate user configured ...for this integration | 14:00 |
| bbezak | \o | 14:01 |
| SvenKieske | let's discuss this later :) | 14:01 |
| mnasiadka | #topic agenda | 14:01 |
| kevko | yep | 14:01 |
| mnasiadka | * CI status | 14:01 |
| mnasiadka | * Release tasks | 14:01 |
| mnasiadka | * Current cycle planning | 14:01 |
| mnasiadka | * Additional agenda (from whiteboard) | 14:01 |
| mnasiadka | * Open discussion | 14:02 |
| frickler | o/ | 14:02 |
| mnasiadka | #topic CI status | 14:02 |
| mnasiadka | So... Docker doesn't like us | 14:02 |
| mnasiadka | for Kolla - there are now only two images that we fetch from docker hub - debian and ubuntu base images | 14:02 |
| SvenKieske | just mentioning this: https://www.docker.com/community/open-source/application/ | 14:02 |
| SvenKieske | other solutions are using quay.io or self hosting/building of course. not sure what's the least amount of work | 14:03 |
| mnasiadka | I see two options of trying to fix this - 1) docker login 2) mirror those images to something like quay.io | 14:03 |
| mnasiadka | No, I'm not going to beg Mirantis to give me access | 14:03 |
| frickler | there is a role being developed in zuul to perform 2) https://review.opendev.org/c/zuul/zuul-jobs/+/935574 | 14:03 |
| mnasiadka | If somebody else wants to go the extra mile of paperwork - fine by me | 14:03 |
| mnasiadka | Any volunteers for the paperwork? :) | 14:04 |
| mnasiadka | Guess not | 14:05 |
| mnasiadka | frickler: that's why we're probably going to wait until that merges and we can use that for mirroring debian/ubuntu | 14:05 |
| frickler | yes, I'd second that | 14:05 |
| SvenKieske | and which quay.io account are we going to use? I think there was a kolla account, right? | 14:06 |
| mnasiadka | Ok then, any other CI failures not related to DockerHub? | 14:06 |
| mnasiadka | SvenKieske: we have openstack.kolla org, I'd say we use that. | 14:06 |
| SvenKieske | good :) | 14:06 |
| frickler | mnasiadka: yes, jobs failing on raxflex | 14:06 |
| mnasiadka | frickler: how are they failing? | 14:06 |
| frickler | example at https://zuul.opendev.org/t/openstack/build/d7a108adc1c8491a92362bcb40b37670 I hope i can look into this later this week | 14:07 |
| frickler | essentially: Error mounting /var/lib/docker: mount: /var/lib/docker: can't find LABEL=kolla. | 14:07 |
| mnasiadka | Ah, I've seen that | 14:07 |
| mnasiadka | thought it's some kernel bug | 14:07 |
| SvenKieske | yep, I had roughly 5-10 of those the last days | 14:07 |
| frickler | so something being different about the ephemeral volume. maybe there isn't any in that cloud | 14:07 |
| frickler | usually these are retried because the failure happens in pre | 14:08 |
| mnasiadka | well, we check if there is one - https://github.com/openstack/kolla/blob/master/roles/configure-ephemeral/tasks/main.yml | 14:08 |
| frickler | if someone finds the cause for this, I won't mind, too ;) | 14:08 |
| mnasiadka | either /dev/xvde or a disk with "ephemeral" label | 14:08 |
| frickler | ah, it might be that /dev/xvde is swap instead | 14:09 |
| mnasiadka | that makes things complicated | 14:09 |
| mnasiadka | but we can always check for raxflex ;-) | 14:09 |
| frickler | anyway, the data to debug should be in the logs | 14:09 |
| SvenKieske | we really should not rely on special device names, labels are more stable | 14:09 |
| mnasiadka | darmach68: want to have a look? | 14:10 |
| frickler | if not, I can also hold a node if needed | 14:10 |
| mnasiadka | SvenKieske: some cloud does /dev/xvde but no labels :) | 14:10 |
| SvenKieske | but I never looked at that file | 14:10 |
| darmach688 | mnasiadka sure! | 14:10 |
| mnasiadka | frickler: we have a volunteer | 14:10 |
| frickler | cool | 14:10 |
| SvenKieske | nice | 14:11 |
| mnasiadka | ok then, let's move on | 14:12 |
| mnasiadka | #topic Release tasks | 14:12 |
| mnasiadka | bbezak: I think we have around two weeks to switch to Epoxy sources? | 14:12 |
| *** darmach688 is now known as darmach | 14:12 | |
| bbezak | let me see | 14:12 |
| mnasiadka | yup, R-17 | 14:13 |
| bbezak | R-17: Switch source images to current release¶ | 14:13 |
| bbezak | yeap | 14:13 |
| opendevreview | Dawud proposed openstack/kolla-ansible master: Add size limits to Fluentd buffers https://review.opendev.org/c/openstack/kolla-ansible/+/924359 | 14:13 |
| bbezak | Dec 02 - Dec 06 | 14:13 |
| mnasiadka | frickler: /dev/vdb has ephemeral0 label on raxflex | 14:14 |
| mnasiadka | but for some reason when we create a filesystem with kolla label - it's not there for mounting | 14:15 |
| mnasiadka | bbezak: willing to raise those next week? or do we need another volunteer? | 14:16 |
| bbezak | source image changes mnasiadka ? | 14:18 |
| mnasiadka | yup | 14:18 |
| bbezak | willdo | 14:18 |
| mnasiadka | there are three changes listed :) | 14:18 |
| mnasiadka | goodie | 14:18 |
| mnasiadka | #topic Current cycle planning | 14:18 |
| mnasiadka | So, I started our WSGI mission with Gunicorn, but that's hard to implement because it requires changes in at least Nova - so probably not a good idea | 14:19 |
| mnasiadka | so I went with uWSGI, although it's in maintenance mode - but Devstack and others are using it | 14:20 |
| mnasiadka | And it seems to work now even with backend TLS | 14:20 |
| mnasiadka | So the question is | 14:20 |
| mnasiadka | do we want to support both apache/mod_wsgi and uwsgi for a cycle? | 14:20 |
| SvenKieske | is it worth it to publish your findings wrt to WSGI to the ML as well? to get broader feedback, e.g. from nova folks? | 14:20 |
| bbezak | I think we should? at least for keystone/horizon | 14:20 |
| mnasiadka | SvenKieske: that's my plan | 14:21 |
| mnasiadka | well, apache (or nginx) and uwsgi - yes | 14:21 |
| mnasiadka | I'm asking if we should replace apache/mod_wsgi with uwsgi in Epoxy | 14:21 |
| mnasiadka | or support both for one cycle | 14:21 |
| frickler | there is some discussion to happen in the scope of the eventlet-rm team about which wsgi/asgi solution to use in the future | 14:21 |
| frickler | so we might need to change again later | 14:21 |
| frickler | support both for transition seems better to me | 14:22 |
| parasitid | hi again, one simple question: is there a ref doc where i can find what are the conf options i can setup in a kolla-build.conf ? i didnt manage to find any on the docs.openstack.org site. thanks a lot | 14:22 |
| SvenKieske | https://github.com/pallets/werkzeug is afaik a still maintained wsgi server, not sure if it's "better" though. | 14:22 |
| mnasiadka | parasitid: we have a meeting now, please ask later | 14:22 |
| mnasiadka | SvenKieske: I think I prefer to go with what others are doing, at least we see the same problems | 14:22 |
| mnasiadka | At least for now :) | 14:23 |
| mnasiadka | frickler: yeah, seen that, but it doesn't seem like it will be changed in Epoxy | 14:23 |
| SvenKieske | sure, absolutely, would be nice if openinfra could converge on the same solution! but I would like something alive upstream :) | 14:23 |
| SvenKieske | that said I don't know if uwsgi is just simply "finished", maybe? :) | 14:24 |
| mnasiadka | anyway, so we go with both apache/mod_wsgi and uwsgi in Epoxy and deprecate apache/mod_wsgi approach so it's removed in F (unless we see some issues) | 14:24 |
| mnasiadka | SvenKieske: You're trying to use the funny term "feature complete"? :) | 14:24 |
| SvenKieske | yeah :) but looking at the c code of uwsgi I'm not so certain I would bet my life on it. | 14:25 |
| mnasiadka | yes, I went with gunicorn since Skyline uses it, but then it has problems passing CLI args - I'll write about my adventure on ML and we can take it from there | 14:26 |
| SvenKieske | nice | 14:26 |
| mnasiadka | Any other new features/changes we need to discuss? | 14:26 |
| SvenKieske | frickler: is there a link to the eventlet-rm team discussion about wsgi/asgi solution? was that only on the ML? | 14:27 |
| mnasiadka | SvenKieske: there's a channel - #openstack-eventlet-removal | 14:27 |
| frickler | SvenKieske: no, there is a (biweekly?) meeting | 14:27 |
| mnasiadka | and meeting logs should be in the usual place | 14:27 |
| SvenKieske | ah ty | 14:28 |
| frickler | also some notes in https://etherpad.opendev.org/p/epoxy-eventlet-tracking | 14:28 |
| mnasiadka | Ok, from another front - I think it would be good if somebody would focus on getting RMQ QueueManager stuff tested and validated | 14:28 |
| mnasiadka | Matt Crees: alive? | 14:28 |
| mnasiadka | Ok, I'll chase him internally - maybe he can do some work in that area | 14:29 |
| frickler | that new ovs exporter project also looked interesting | 14:30 |
| kevko | mnasiadka: there is need to share /dev/shm | 14:30 |
| SvenKieske | yes, that would be nice, it should work now properly in containers | 14:30 |
| kevko | Quemanager | 14:30 |
| SvenKieske | if we have the above device^^ | 14:30 |
| kevko | for oslo services | 14:30 |
| kevko | then ..it can work nice .. | 14:30 |
| mnasiadka | kevko: I know, I'm just trying to find somebody else than you to test it out thoroughly :) | 14:30 |
| mnasiadka | And I assume that's needed for the fanout queues in new RMQ world? | 14:31 |
| kevko | mnasiadka: like i am not testing well ? :D | 14:31 |
| kevko | mnasiadka: yep, correct ... fanouts .. | 14:31 |
| mnasiadka | kevko: I'd prefer that we don't blame everything on you :) | 14:31 |
| SvenKieske | yes, pretty certain we need the queue manager to get this correctly working | 14:32 |
| SvenKieske | there was a ML report having issues with quorum queues, but there was not much information what the actual issue was - could've been just misconfiguration | 14:32 |
| mnasiadka | Well, I've seen issues with quorum queues and ironic-neutron-agent | 14:33 |
| mnasiadka | I don't know if that's rather a fanout queue, or not - in classical world it had autodelete on | 14:33 |
| kevko | Haha, I don't recall you ever having to blame me for anything :D, as far as I know, everything works :) | 14:33 |
| mnasiadka | and now it doesn't, so it's piling up queues and messages unless you delete those old queues without consumers | 14:33 |
| mnasiadka | kevko: nobody's perfect :) | 14:34 |
| mnasiadka | Ok then, let's move on | 14:35 |
| mnasiadka | #topic Additional agenda (from whiteboard) | 14:35 |
| mnasiadka | (SvenKieske 2024-11-13) reviews would be nice: | 14:36 |
| mnasiadka | https://review.opendev.org/c/openstack/kolla-ansible/+/928949 | 14:36 |
| mnasiadka | https://review.opendev.org/q/+status:open+-is:WIP+uploader:kieske@osism.tech+label:Verified%252B1+-label:Workflow-1+is:mergeable | 14:36 |
| mnasiadka | If anybody got some time, especially our quickstartguide needs to be changed since our arguments are reversed now (good first issue, if you are new to kolla): https://bugs.launchpad.net/kolla-ansible/+bug/2087920 | 14:37 |
| mnasiadka | darmach68: I know you complained over the quickstart guide recently, right? | 14:37 |
| SvenKieske | ah I probably get around to fix the quick start guide today | 14:37 |
| SvenKieske | but I won't complain if someone is faster :D | 14:37 |
| SvenKieske | that list above is mostly backports I think | 14:37 |
| mnasiadka | ok then, I'll just leave it there - I reapplied my +2 on the first patch from the list | 14:38 |
| frickler | the issue darmach found was with the deploy guide? different bug | 14:38 |
| mnasiadka | Ah, that's another thing | 14:38 |
| mnasiadka | We were discussing that today, that probably those variables that we use in docs conf.py should land in deploy-guide conf.py as well | 14:38 |
| mnasiadka | so maybe we need to move that to some common place and source it? | 14:39 |
| frickler | +1 | 14:39 |
| mnasiadka | darmach688: you've got your answer | 14:39 |
| mnasiadka | Next one from the whiteboard | 14:39 |
| mnasiadka | (frickler 2024-11-27) transition stable/2023.1 to unmaintained? | 14:39 |
| mnasiadka | https://review.opendev.org/c/openstack/releases/+/934490 | 14:39 |
| mnasiadka | I think all non-deployment projects already moved to unmaintained? | 14:40 |
| frickler | well almost all, some still lagging | 14:40 |
| mnasiadka | Do we need to do anything in Kolla like switch to use unmaintained sources? | 14:40 |
| frickler | but a lot of stable/2023.1 branches are already gone | 14:40 |
| frickler | so yes, those references would need to get switched likely like was done for zed? | 14:41 |
| SvenKieske | yes, I think so, also in the docs, no? | 14:41 |
| mnasiadka | Yes, so any volunteer to do the switch? | 14:41 |
| frickler | a related question would be whether some of the older branches can be eoled? or who is volunteering to take care of them? | 14:42 |
| mnasiadka | I'll do that again... darmach68 ? | 14:42 |
| SvenKieske | #link https://review.opendev.org/c/openstack/kolla-ansible/+/927022 | 14:42 |
| mnasiadka | frickler: from my perspective yoga and zed can be EOLed | 14:42 |
| mnasiadka | if anybody is there - it's his own fault ;-) | 14:42 |
| SvenKieske | that's an example what needs to be fixed in the docs part at least | 14:42 |
| SvenKieske | that as well: https://review.opendev.org/c/openstack/kolla-ansible/+/917421 (update .gitreview) | 14:43 |
| SvenKieske | if you search for unmaintained you find quite some stuff :) | 14:43 |
| mnasiadka | yup | 14:44 |
| darmach | Yep, let it be me :D | 14:44 |
| mnasiadka | ok then | 14:44 |
| frickler | the gitreview update patch will be auto-created | 14:44 |
| SvenKieske | drop upgrade testing as well | 14:44 |
| SvenKieske | ah right | 14:44 |
| frickler | so if there are no complaints, we could just go ahead with the release patch first | 14:44 |
| mnasiadka | ok then, once we fix antelope and transition it to unmaintained - we can EOL yoga and zed | 14:45 |
| frickler | and amend the unmaintained branches as needed afterwards? | 14:45 |
| mnasiadka | frickler: fine by me | 14:45 |
| mnasiadka | commented on the patch | 14:45 |
| mnasiadka | ok then, let's go to open discussion | 14:46 |
| mnasiadka | #topic Open discussion | 14:46 |
| mnasiadka | Anybody anything? | 14:46 |
| frickler | ah, you were faster than me ;) | 14:46 |
| frickler | yes, one more thing | 14:46 |
| mmalchuk | Certificates defaults | 14:46 |
| mmalchuk | https://review.opendev.org/c/openstack/kolla-ansible/+/934514 | 14:46 |
| mmalchuk | please review | 14:46 |
| frickler | cberendt (fellow kolla core) is running for foundation board, maybe someone wants to support his nomination https://openinfra.dev/a/community/members/7040 | 14:47 |
| mnasiadka | mmalchuk: I'm rather thinking about -1 than merging - I don't think people should be using that role for anything except development envs and testing | 14:47 |
| mmalchuk | mnasiadka nice) why? | 14:48 |
| mnasiadka | frickler, SvenKieske, bbezak, kevko what do you think? | 14:48 |
| mnasiadka | because Kolla-Ansible should not be in the certificate management business :) | 14:48 |
| SvenKieske | it _might_ be useful if you need to test some specific cert/CA settings | 14:48 |
| mmalchuk | peoples already use this for production for a years) | 14:48 |
| SvenKieske | but yeah, maybe there are better tools for that | 14:48 |
| SvenKieske | you really shouldn't use this for production, honestly | 14:48 |
| SvenKieske | use a proper CA at least, there are some projects on github etc which do this | 14:49 |
| mmalchuk | SvenKieske but there is no other tools in Kolla for production) | 14:49 |
| SvenKieske | you can always push your CA cert with external cert role, no? | 14:50 |
| SvenKieske | e.g. something like this (no endorsement, there are others as well): https://github.com/smallstep/certificates | 14:50 |
| kevko | mnasiadka: kolla-ansible should be cert manager ...but still ...everyone is using it :D :D :D | 14:50 |
| kevko | *shouldn't be | 14:50 |
| mmalchuk | I do that, only CA and use role to generate endlevel certs | 14:51 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: nova: Add support for using uWSGI https://review.opendev.org/c/openstack/kolla-ansible/+/935975 | 14:51 |
| kevko | I am ok with it ...it's just movement values to variables ..that's all | 14:51 |
| mnasiadka | I think we discussed some PTGs ago to support things like smallstep | 14:52 |
| mmalchuk | yep | 14:52 |
| SvenKieske | yes, I mean of course people will be using test CAs in prod, it doesn't matter how many warnings you put into docs :D | 14:52 |
| mnasiadka | Yes, I don't know if we should make it easier though :) | 14:52 |
| mnasiadka | And I don't know if I like the command: style of that role ;-) | 14:53 |
| mmalchuk | the bugreport is open, this change only PartialBug so lets merg it and add smalstep or whatever and close the bug | 14:53 |
| mnasiadka | It's not a bug | 14:53 |
| mnasiadka | It's by design | 14:53 |
| mmalchuk | design with bug))) | 14:54 |
| kevko | Well, if it will be under ansible/roles ...from my perspective it can be used by user ...we also providing kolla-ansible certificates command ...so if we don't want from users to use it ...remove it and place into kolla collections or somewhere ...or something similar as we did for ceph | 14:54 |
| mmalchuk | yes, it in the K-A but can't be used and configured | 14:55 |
| mnasiadka | That was my comment long time ago when we added that certificates role - but obviously it makes life easier when doing a development/test deployment with TLS | 14:55 |
| mmalchuk | so this is an issue | 14:55 |
| mnasiadka | Issue why? | 14:55 |
| mnasiadka | Tell us why do you need to set those variables to do a K-A deployment for testing | 14:56 |
| mmalchuk | certs generated with wrong defaults | 14:56 |
| mmalchuk | at least days | 14:56 |
| SvenKieske | from a users perspective I can see why you would want it. from a dev perspective I would argue to make it harder to use and replace by proper CA integration. you can use your CA api for configuring all this stuff, it's outside of kollas domain imho. | 14:56 |
| mmalchuk | at least days of validity | 14:56 |
| SvenKieske | isn't the current validity a year or something? | 14:57 |
| SvenKieske | maybe I don't understand the validity issue. | 14:57 |
| mnasiadka | Me neither | 14:58 |
| mmalchuk | after an year peoples forgot about self-signed cert and get an issue in api/browser again | 14:58 |
| frickler | after a year a test environment is broken anyway | 14:58 |
| mmalchuk | some of them want certs in dev for 10 years for example | 14:58 |
| mnasiadka | you've got year long development environments? | 14:58 |
| mmalchuk | yep | 14:58 |
| mmalchuk | we still have ussuri ))) | 14:59 |
| SvenKieske | so either you automate your CA, (you need to control client CA deployment anyway I guess) | 14:59 |
| mmalchuk | on Centos7 ))) | 14:59 |
| SvenKieske | I mean changing validity is orthogonal to make it configurable | 14:59 |
| darmach | I'd assume that long existing dev/test env should already be rebuilt ;) | 14:59 |
| SvenKieske | we could propose to set validity to seven days only, and hardcode that, to make clear this is only for dev mode? ;) | 14:59 |
| frickler | SvenKieske: +1 | 15:00 |
| mmalchuk | lol | 15:00 |
| darmach | SvenKieske +1 for harcore solutions! | 15:00 |
| mnasiadka | Yeah, probably that's what we should rather do | 15:00 |
| SvenKieske | poor users using this in prod xD | 15:00 |
| mnasiadka | maybe 14 or 30 so we're not that hard on people - but that shouldn't be a year | 15:01 |
| mmalchuk | lets drop the certificates role at all! and ask users to use their own CA | 15:01 |
| mmalchuk | this would be hardcore) | 15:01 |
| mnasiadka | We do that in the docs, don't we? | 15:01 |
| SvenKieske | I think it's really good to have a cert stub like this for easy testing | 15:01 |
| mmalchuk | no one reads the docs) | 15:01 |
| mnasiadka | I'm happy to move that role outside of kolla-ansible delivered roles and just give a bash script to create certs for dev env with TLS | 15:02 |
| SvenKieske | or we go full in and implement a proper CA even for testing, that would be better, but nobody got time for that I guess? | 15:02 |
| SvenKieske | we at least need something to test TLS in CI | 15:02 |
| mmalchuk | oh cool. lets rewrite ansible to bash) | 15:02 |
| kevko | haha ..let's do that :D 7 days is fine :D | 15:03 |
| darmach | mmalchuk I started recently, and found out that half is outdated, the other half doesn't template/render properly ;) | 15:03 |
| kevko | we will see how many bugreports we will receive :D | 15:03 |
| SvenKieske | anyway we are over time for the meeting | 15:03 |
| mnasiadka | yup | 15:03 |
| kevko | Okay, can anybody advise me I've asked before meeting ? | 15:03 |
| kevko | do anybody know if i can 1. have external public network with dns_domain set - shared, 2. zone for that network => Can another tenant create VM on this network and it will be automatically DNS created ? | 15:04 |
| mnasiadka | so the consensus is let's leave it as it is or make it harder for users to use ;-) | 15:04 |
| frickler | kevko: no, external networks are intentionally excluded | 15:04 |
| mmalchuk | be prepare for bugreports) | 15:05 |
| kevko | frickler: Hmm, I had that feeling ... | 15:05 |
| kevko | frickler: what about zone transfer request ? | 15:05 |
| mnasiadka | ok then, let's finish the meeting | 15:06 |
| mnasiadka | #endmeeting | 15:06 |
| opendevmeet | Meeting ended Wed Nov 27 15:06:30 2024 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:06 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-11-27-14.00.html | 15:06 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-11-27-14.00.txt | 15:06 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-11-27-14.00.log.html | 15:06 |
| kevko | or zone share ? | 15:06 |
| mmalchuk | mnasiadka thanks | 15:06 |
| frickler | kevko: the restriction is on the neutron side | 15:07 |
| SvenKieske | thx guys | 15:07 |
| kevko | frickler: if i am creating as admin ..it is creating records in my zone .. | 15:10 |
| kevko | frickler: only if I use another tenant it's not workng | 15:11 |
| SvenKieske | is anybody from stackhpc still interested in this, given that Mark moved on to other stuff? https://review.opendev.org/c/openstack/kolla-ansible/+/899615 | 15:30 |
| SvenKieske | mnasiadka, kevko: any non osism core have time for a +2? :) https://review.opendev.org/c/openstack/kolla-ansible/+/916972 | 15:35 |
| mnasiadka | SvenKieske: done, need more details, this looks a bit vague :) | 15:49 |
| SvenKieske | okay, maybe someone else knows why, I don't know much Skyline tbh | 15:53 |
| mnasiadka | Me neither, but since it's a feature behind a variable - I would like to understand why it can't work like Horizon. | 15:54 |
| kevko | skyline is crap :D | 15:54 |
| kevko | btw - designate , shared public network, shared zone ... user a create vm ..record is present ...user b create another vm ...recordset is there ....if user b want to remove or update user a record ..it's forbidden ..so ..that what i've needed | 15:56 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: nova: Add support for using uWSGI https://review.opendev.org/c/openstack/kolla-ansible/+/935975 | 16:00 |
| opendevreview | Merged openstack/kolla-ansible stable/2024.1: Do not remove inventory file if placed in /etc/kolla https://review.opendev.org/c/openstack/kolla-ansible/+/934440 | 16:15 |
| SvenKieske | mnasiadka: what about https://review.opendev.org/c/openstack/kolla-ansible/+/927096 is that sort of abandoned? was it superseeded by something in the relation chain? | 16:17 |
| SvenKieske | it's about the wsrep_provider plugin | 16:17 |
| mnasiadka | Yeah, I need to get back to mariadb bump :) | 16:18 |
| mnasiadka | https://review.opendev.org/c/openstack/kolla-ansible/+/927039 - that should be good to merge though | 16:19 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: mariadb: switch to use mariadb flavored commands https://review.opendev.org/c/openstack/kolla-ansible/+/927039 | 16:20 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: mariadb: Disable wsrep_provider plugin https://review.opendev.org/c/openstack/kolla-ansible/+/927096 | 16:20 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: mariadb: Switch to rsync sst_method on upgrade https://review.opendev.org/c/openstack/kolla-ansible/+/928487 | 16:20 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: mariadb: Bump to 11.4 https://review.opendev.org/c/openstack/kolla/+/926981 | 16:23 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: mariadb: Bump to 11.4 https://review.opendev.org/c/openstack/kolla/+/926981 | 16:24 |
| mnasiadka | SvenKieske: thanks for reminding :) | 16:24 |
| SvenKieske | no problem, going over all the open stuff right now (well as far as I can get) | 16:31 |
| opendevreview | yosef proposed openstack/kolla-ansible master: Add election-timer option to ovn-db role https://review.opendev.org/c/openstack/kolla-ansible/+/934106 | 16:31 |
| ishanwar | Hi Team, I wanted to discuss about rabbitmq issue I have been facing during maintainence of the openstack cluster. I have started a thread in the openstack discuss mailing list | 16:34 |
| parasitid | hi again. is there a way to refenrence a review.opendev.org patchset in the git location of a kolla component ? | 16:38 |
| parasitid | when i put the reference = refs/changes/x/xxxxx/x kolla fails to find it | 16:39 |
| SvenKieske | you mean if you want to build a container? | 16:40 |
| SvenKieske | depends a bit, but in general, you can find the commit id from the gerrit web ui | 16:41 |
| SvenKieske | e.g. https://review.opendev.org/c/openstack/kolla-ansible/+/927039 here you have a link to "gitea" on the left mid side | 16:41 |
| SvenKieske | which is https://opendev.org/openstack/kolla-ansible/commit/065bc8028035cb10840e055b2bed227bf97a2ac3 | 16:41 |
| SvenKieske | but I'm not sure I understood correctly what you want to do with the git location in kolla | 16:42 |
| parasitid | build a container | 16:43 |
| parasitid | from a patchset which isnt merged | 16:43 |
| parasitid | i dont want to clone the sources locally if possible, i guess that "source type git" is made for this right ? | 16:44 |
| SvenKieske | yes, see the docs: https://docs.openstack.org/kolla/latest/admin/image-building.html#build-openstack-from-source | 16:46 |
| parasitid | SvenKieske: cmdline: git checkout a26bd59e34246bd102dbcffbdefca1bf9cfe0df5 | 16:48 |
| parasitid | stderr: 'fatal: reference is not a tree: a26bd59e34246bd102dbcffbdefca1bf9cfe0df5' | 16:48 |
| opendevreview | Merged openstack/kayobe master: CI: use quay.io/podman/hello due too dockerhub rate limits https://review.opendev.org/c/openstack/kayobe/+/936130 | 16:49 |
| parasitid | SvenKieske: i cant specify a commit id in the reference option ? | 16:49 |
| kevko | parasitid: vvvvv | 16:50 |
| kevko | michalarbet@pixla:/tmp/kolla$ git show 11f65c6c1 | grep Change-Id: | 16:50 |
| kevko | Change-Id: I61d0790c5d4d070b7ea9e8c99c0a76ff5d22bf9d | 16:50 |
| kevko | copy that change id | 16:50 |
| kevko | and paste into text area on review.openstack.org | 16:50 |
| kevko | or ..checkout some change from review | 16:52 |
| kevko | for https://review.opendev.org/c/openstack/kolla/+/934417 ...it's git fetch https://review.opendev.org/openstack/kolla refs/changes/17/934417/1 && git checkout FETCH_HEAD | 16:52 |
| kevko | i never used gitea :D | 16:53 |
| parasitid | kevko: i tried to put the following stuff | 16:54 |
| parasitid | ! 12 [designate-base] | 16:54 |
| opendevmeet | parasitid: Error: "designate-base" is not a valid command. | 16:54 |
| parasitid | ! 13 enabled = True | 16:54 |
| opendevmeet | parasitid: Error: "13" is not a valid command. | 16:54 |
| parasitid | ! 14 type = git | 16:54 |
| opendevmeet | parasitid: Error: "14" is not a valid command. | 16:54 |
| parasitid | ! 15 location = https://opendev.org/openstack/designate | 16:54 |
| opendevmeet | parasitid: Error: "15" is not a valid command. | 16:54 |
| parasitid | ! 16 reference = "a26bd59e34246bd102dbcffbdefca1bf9cfe0df5" | 16:54 |
| opendevmeet | parasitid: Error: "16" is not a valid command. | 16:54 |
| parasitid | damn sorry for the paste command | 16:54 |
| kevko | :D] | 16:55 |
| parasitid | missclicked | 16:55 |
| SvenKieske | kevko: they try to do something with kolla build customization, not sure what exactly, so this is not about locating changes in gerrit.. | 16:55 |
| parasitid | i dont know what to paste into the reference field (refs/changes/... commit id, etc) | 16:55 |
| SvenKieske | it would be good to get a complete description though, e.g. your kolla-build.conf | 16:56 |
| SvenKieske | but I'm out soon, so might be better to post to the ML instead | 16:56 |
| parasitid | https://pastebin.com/DM8uErir | 16:56 |
| SvenKieske | unless kevko does not sleep, as usual :) | 16:57 |
| kevko | parasitid: https://paste.openstack.org/show/bdzSoOiUHXnQO8Jt3fTs/ | 16:57 |
| kevko | parasitid: aaaaa, you want to build kolla images with changed designate source right ? | 16:58 |
| parasitid | yes | 16:58 |
| kevko | master kolla branch ? | 16:58 |
| parasitid | 19.0.0 | 16:58 |
| kevko | parasitid: you can fork designate ...apply change you are missing ... publish to git ..or publish to some tarball ...and configure your kolla-build.conf to get designate from different source | 16:59 |
| kevko | SvenKieske: what else I can do :D ... sleeping is boring :D | 16:59 |
| parasitid | kevko: the change is already on opendev.org | 16:59 |
| kevko | parasitid: but not merged :) | 17:00 |
| parasitid | no it's not. so you confirm i cant target a commit id ? it has to be merged ? | 17:00 |
| kevko | parasitid: you can, you have to prepare your own git with designate (or tarball) ...and place config into kolla-build conf ...to download from different location | 17:01 |
| parasitid | at first i used the tarball link from the review.opendev.org patchset. but it didnot work either because the tarbal is not structured like expected by kolla | 17:02 |
| kevko | parasitid: https://paste.openstack.org/show/bKh2BdlSamO7p0pmJ579/ | 17:02 |
| parasitid | ok thanks, will do | 17:02 |
| kevko | parasitid: which version do you need ? | 17:02 |
| kevko | 2024.2 ? | 17:03 |
| kevko | i mean openstack ? | 17:03 |
| kevko | parasitid: place this into kolla-build.conf and be happy :) | 17:13 |
| kevko | https://paste.openstack.org/show/bS8WsiwFa2ovJCTMnn5g/ | 17:13 |
| parasitid | kevko: thanks a lot. i've pushed it on my own github account. cheers. | 18:41 |
| parasitid | actually there's a bug with designate master's branch | 20:35 |
| parasitid | https://opendev.org/openstack/designate/commit/3462a55177e208090ae571f95037e3d0ff35cf89 | 20:35 |
| parasitid | they renamed the rootwrap.conf.sample files, which breaks kolla designate dockerfile ;( | 20:36 |
| deflated | Evening all, attempting to deploy cloudkitty on 2024.2 with prometheus/opensearch and the cloudkitty-processing container is unhealthy, logs show no errors but restarting the service on the host shows me this: https://paste.openstack.org/show/b84zNBqM5UScWZoLzsOU/ Anyone any ideas? Also telegraf exits with no errors, any chance it relates to this bug: https://bugs.launchpad.net/kolla-ansible/+bug/2077785 | 20:42 |
| opendevreview | para sitid proposed openstack/kolla master: Fix 2089792: designate's rootwrap.conf renamed https://review.opendev.org/c/openstack/kolla/+/936413 | 20:44 |
| opendevreview | Merged openstack/kolla-ansible master: Skyline: use an external object store (Swift) in the dashboard https://review.opendev.org/c/openstack/kolla-ansible/+/916972 | 20:52 |
| opendevreview | Bartosz Bezak proposed openstack/kayobe stable/2024.2: CI: use quay.io/podman/hello due too dockerhub rate limits https://review.opendev.org/c/openstack/kayobe/+/936416 | 21:52 |
| opendevreview | Bartosz Bezak proposed openstack/kayobe stable/2024.1: CI: use quay.io/podman/hello due too dockerhub rate limits https://review.opendev.org/c/openstack/kayobe/+/936417 | 21:52 |
| opendevreview | Bartosz Bezak proposed openstack/kayobe stable/2023.2: CI: use quay.io/podman/hello due too dockerhub rate limits https://review.opendev.org/c/openstack/kayobe/+/936418 | 21:53 |
| opendevreview | Bartosz Bezak proposed openstack/kayobe stable/2023.1: CI: use quay.io/podman/hello due too dockerhub rate limits https://review.opendev.org/c/openstack/kayobe/+/936419 | 21:54 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!