| gokhan | hello folks, I am trying to access public container on swift, but I am getting no such bucket error. I am using swift radosgw implementation. I followed this document > https://docs.cleura.cloud/howto/object-storage/swift/public-container/. My configs and rgw logs are in https://paste.openstack.org/show/b8YbglwOxZp6wK7oJIOz/. may be this known issue. Do you have any idea for this ? | 10:01 |
|---|---|---|
| * noonedeadpunk participated in writing this doc lol | 10:03 | |
| noonedeadpunk | that totally works for us with osa and rgw just in case, as otherwise won't be wirting it... | 10:04 |
| noonedeadpunk | so, using swift cli does the container exist? | 10:06 |
| noonedeadpunk | and how are you trying to access it, just in case? | 10:06 |
| gokhan | thanks noonedeadpunk for this doc :) May be it is about ceph version. we are using ceph reef 18.2.2 | 10:06 |
| gokhan | noonedeadpunk, I am trying to access with curl : curl http://10.13.0.10:8080/swift/v1/AUTH_99fe423d80344a6fa1aa3bcd5767e07a/public-container/testobj.txt | 10:07 |
| gokhan | with openstack swift client using openstack token it is ok. | 10:08 |
| gokhan | url -H "X-Auth-Token: gAAAAABn0qKf0oYrh5YbzOjX3t2uyChfmUZbbW00hmYcY7nbBpdzvzWbTF-ofvGOX_0SpqE1u3CkUzNb5gEmCgtCsoZiH90YYVSJzek3IotNdx3Yr0_xF3PwmGWcLO_F32gsIaNhjaw56m2w4rJ5FwtfEenw9UwWeREc65qZ340YpkT4LWCT224" http://10.13.0.10:8080/swift/v1/AUTH_99fe423d80344a6fa1aa3bcd5767e07a/public-container/testobj.txt | 10:09 |
| gokhan | hello world | 10:09 |
| gokhan | root@test-infra01-utility-container-e5858c68:~# curl http://10.13.0.10:8080/swift/v1/AUTH_99fe423d80344a6fa1aa3bcd5767e07a/public-container/testobj.txt | 10:09 |
| gokhan | NoSuchBucketroot@test-infra01-utility-container-e5858c68:~# | 10:09 |
| gokhan | its curl response is NoSuchBucket | 10:09 |
| noonedeadpunk | huh, ok, that is totally weird and not expected I'd say... Let me to follow the guide on one of our regions just in case | 10:10 |
| gokhan | ok thank you :) it worked on my previously created deployments on antelope and ceph is quincy or pacific. | 10:11 |
| noonedeadpunk | We're running caracal and for ceph - need to double check | 10:14 |
| opendevreview | Marcus Klein proposed openstack/openstack-ansible-ops master: traefik_common for Grafana deployment needs to use Python 3 https://review.opendev.org/c/openstack/openstack-ansible-ops/+/944202 | 10:15 |
| gokhan | We are now running caracal (29.0.2) and ceph reef 18.2.2 | 10:16 |
| noonedeadpunk | ok, so I have exactly same environment on my hands right now | 10:18 |
| opendevreview | Marcus Klein proposed openstack/openstack-ansible-ops master: upgraded prometheus.prometheus collection does not need ansible fact vars anymore https://review.opendev.org/c/openstack/openstack-ansible-ops/+/944203 | 10:18 |
| noonedeadpunk | gokhan: well. seems we have the same issue there | 10:20 |
| noonedeadpunk | damiandabrowski: are you aware of this? ^ | 10:20 |
| gokhan | noonedeadpunk, on rgw logs: debug 2025-03-13T09:08:03.904+0000 7ee658000700 1 ====== starting new request req=0x7ee7224a6730 ===== | 10:22 |
| gokhan | debug 2025-03-13T09:08:03.904+0000 7ee658000700 1 req 12852707387194772156 0.000000000s op->ERRORHANDLER: err_no=-2002 new_err_no=-2002 | 10:22 |
| gokhan | debug 2025-03-13T09:08:03.905+0000 7ee658000700 1 ====== req done req=0x7ee7224a6730 op status=0 http_status=404 latency=0.001000031s ====== | 10:22 |
| gokhan | debug 2025-03-13T09:08:03.905+0000 7ee658000700 1 beast: 0x7ee7224a6730: 10.13.15.208 - 99fe423d80344a6fa1aa3bcd5767e07a$anonymous [13/Mar/2025:09:08:03.904 +0000] "GET /swift/v1/AUTH_99fe423d80344a6fa1aa3bcd5767e07a/public-container/testobj.txt HTTP/1.1" 404 12 - "curl/7.81.0" - latency=0.001000031s | 10:22 |
| noonedeadpunk | yeah, was able to reproduce it... | 10:22 |
| noonedeadpunk | I'd guess it must be rgw thing tbh | 10:23 |
| damiandabrowski | ouh....interesting... | 10:23 |
| gokhan | yeah I aggree with you. | 10:24 |
| noonedeadpunk | jrosser: looking at https://review.opendev.org/c/openstack/openstack-ansible-ops/+/944136 - that sounds like despite things are not very well reviewed, now on top we also are unable to contribute either due to rust requirement? | 10:52 |
| noonedeadpunk | sounds that I'd need to get time to look at a different driver... | 10:56 |
| noonedeadpunk | gokhan: it's pretty much possible that you can do smth like that only through S3 API now... | 10:58 |
| gokhan | noonedeadpunk, ohh I can curl http://10.13.0.10:8080/public-container/testobj.txt | 11:02 |
| kleini | I am newly deploying my staging environment with Antelope 27.5.1 and stumbling over Heat and Magnum service setup: https://paste.opendev.org/show/bzMqwfvHlrcmlXYn393P/ | 11:06 |
| opendevreview | Merged openstack/openstack-ansible-ops master: traefik_common for Grafana deployment needs to use Python 3 https://review.opendev.org/c/openstack/openstack-ansible-ops/+/944202 | 11:07 |
| kleini | Is that fixed in a later openstack.osa collection service_setup role? | 11:07 |
| noonedeadpunk | I'm pretty sure it must be fixed.. I'm a little bit suprised that it's broken on Antelope | 11:11 |
| noonedeadpunk | but it could be actually an ansible-collection-openstack thing as well | 11:13 |
| kleini | preparing upgrade to Caracal and report, whether caracal fails or not. | 11:19 |
| noonedeadpunk | I can recall that issue and that we dealt with it somehow, but was a while ago... | 11:19 |
| kleini | short different question without having searched anywhere: due to moving nodes to a different data center, is there a way to disable auto start of LXC containers temporarily? So I can start Galeras and stuff first before starting all OpenStack services. I also need to verify all VLANs are working as expected. | 11:20 |
| noonedeadpunk | I think you need to `sysctemctl disable lxc` | 11:25 |
| noonedeadpunk | as it's who's responsible for containers autostart | 11:25 |
| kleini | commenting `lxc.start.auto = 1` will not work? | 11:27 |
| noonedeadpunk | it will as well afaik | 11:27 |
| noonedeadpunk | but you'd need to to that for all containers I assume? | 11:28 |
| noonedeadpunk | while disabling lxc service seems better solution for the usecase.. | 11:28 |
| kleini | I would like to start one Galera first, check it, and so on | 11:28 |
| noonedeadpunk | yeah, but you can do lxc-start -n <container> anyway? | 11:28 |
| kleini | right, I can still use lxc-start | 11:28 |
| kleini | thanks! I am curious. Will be busy weekend... | 11:29 |
| noonedeadpunk | sounds like it | 11:29 |
| kleini | https://review.opendev.org/c/openstack/openstack-ansible-ops/+/944203 <- is it possible to get this merged, too? | 12:58 |
| sykebenX | Hello, is there a way, using openstack-ansible Nova role to safely not use SSH CA authentication between compute hosts OR to somehow prevent the creation of the authorized principals file? It is conflicting with my own implementation of SSH CA auth using another authorized principal mechanism | 15:22 |
| mossblaser | well this is funny timing; I have just today been thinking about the same thing | 15:59 |
| mossblaser | the difficulty here is that OpenSSH's config design doesn't provide a sensible way to have multiple non-coordinated things configure certificates and principals | 15:59 |
| sykebenX | Yeah that's what I'm finding now too :/ | 16:00 |
| mossblaser | the existing role part-works-around this by having a convention of putting all the certs in a directory and then assembling them into a combined file | 16:00 |
| mossblaser | unfortunately there isn't a similar process in place for the principals (which, when using AuthorizedPrincipalsFile doesn't support breaking things down depending on the certificate either) | 16:01 |
| sykebenX | I almost thought I hacked it by setting AuthorizedPrincipalsCommand /usr/bin/awk -F: '($3>=1000)&&($1!="nobody"){print $1}' /etc/passwd -- this for obvious reasons is NOT something you should do lol | 16:01 |
| sykebenX | Yeah I figured out how to get the certs part working, but yeah still stuck on the auth principals | 16:02 |
| mossblaser | what I've been thinking is that it should define an AuthorizedPrincipalsCommand /bin/cat /etc/ssh/authorized_principals/%F/%u_principals.txt | 16:02 |
| mossblaser | then have a directory per certificate fingerprint with a per-user principals file | 16:02 |
| mossblaser | this way at the very least if the principals for each certificate are separately managed you can avoid stepping on eachothers toes | 16:03 |
| mossblaser | (you still have to remember to assemble the certs file of course!) | 16:03 |
| mossblaser | (sorry, just going afk for a moment) | 16:03 |
| sykebenX | That's an interesting idea - it's still not ideal for my use-case, however. I am using Hashicorp vault as CA and I want to use that as the way to limit which principals can be attached to the cert. This way I get the same control over which users each person is allowed to authenticate as, but I don't have to maintain an authorized principals file on each host | 16:06 |
| sykebenX | (no worries - going for lunch now myself . will check back later) | 16:06 |
| sykebenX | It seems though that openssh will use any authorized principals file as an authoritative source if one is specified and this will override the behaviour I want for my own cert authentication scheme | 16:08 |
| mossblaser | ah right; we're also using Vault to issue certs, though our principals are more task-oriented rather than user-oriented | 16:12 |
| mossblaser | in the case, perhaps it might be worth writing a very small script in place of cat in the proposal above which allows you to choose between an explicit principals mapping (as deployed by OSA) and the implicit principals=users behaviour | 16:13 |
| opendevreview | Merged openstack/openstack-ansible-ops master: upgraded prometheus.prometheus collection does not need ansible fact vars anymore https://review.opendev.org/c/openstack/openstack-ansible-ops/+/944203 | 16:19 |
| mossblaser | I've been thinking about different approaches to this for the past day or two and was just about to start putting together a patch | 16:20 |
| mossblaser | my instinct is that the current assembled TrustedUserCAKeys is probably the best you can do for the list of trusted certs (but again I'd be enthusiastic to hear of any alternative suggestions) | 16:22 |
| mossblaser | for the principals situation, clearly different people are going to have different needs so perhaps the most flexible option is to deploy a small script which on a per-CA basis has one of three behaviours: an explicit set of principal files (like OSA does), the implicit principal=user mapping or a user-provided script to implement as an escape-hatch for alternative logic | 16:23 |
| opendevreview | Vincent Legoll proposed openstack/openstack-ansible-os_nova master: Fix ansible `difference()` filter use https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/944283 | 16:36 |
| noonedeadpunk | hey | 17:06 |
| * noonedeadpunk reeding back | 17:06 | |
| noonedeadpunk | sykebenX mossblaser: I think there's actually a way of doing that | 17:06 |
| noonedeadpunk | oh | 17:07 |
| noonedeadpunk | you're probably right... | 17:08 |
| noonedeadpunk | I kind of wonder if such principal even makes sense in case of having vault, rather then just ask a vault for issuing a certificate for nova/keystone directly? | 17:10 |
| noonedeadpunk | not 100% sure it's how it works though... | 17:10 |
| sykebenX | noonedeadpunk: I think that's why I was thinking of a way to either disable the authorized principal validation for nova (since you still have to have a key signed by the OSA CA). I think to implement HC Vault signing for Nova SSH keys might be a bit challenging since you'd have to have Nova authenticate with vault and make a request to sign the key somehow. Not impossible, but might be more effort and introduces a new dependency as well | 17:19 |
| opendevreview | Vincent Legoll proposed openstack/openstack-ansible-os_nova master: Fix ansible `difference()` filter use https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/944283 | 17:25 |
| noonedeadpunk | and actually, I think that we had somebody in our org looking into vary simmilar topic last week as well... | 17:27 |
| noonedeadpunk | But I wasn't involved too much | 17:27 |
| mossblaser | yeah, the difficulty is that ultimately there are two quite independent systems at play: OpenStack and then whatever else you're doing to provision the underlying infrastructure. The CA setup is intimately tied to both and having one environment set up things for both would be non-ideal | 17:27 |
| mossblaser | hah, what are the odds! If they came to a good solution -- or even have more insight into the requirements which exist out there -- it'd be great if you could find out | 17:28 |
| noonedeadpunk | I'm really not sure they did exactly the same thing, as it sounded like placing an extra file to `/etc/ssh/trusted_ca.d/` just worked for them... | 17:29 |
| noonedeadpunk | But I'd need to double check that tbh | 17:29 |
| mossblaser | thanks! I think the most interesting thing is how you handle the authorised principals mapping as different people/systems will have completely different approaches | 17:32 |
| noonedeadpunk | yeah, that is smth I don't know so far... as it's all WIP right now. | 17:34 |
| noonedeadpunk | but at the very least there's an interest in better support for this | 17:34 |
| mossblaser | jolly good; hopefully I'll be able to put a patch together tomorrow | 18:03 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!