| *** mhen_ is now known as mhen | 01:57 | |
| Callum027 | Hi there. I was wondering if anyone here (maybe core members who have been around a while) knew why the container_format and disk_format fields are nullable in Glance's APIs? What does it mean when they are null? I'm trying to resolve an issue in Ceilometer where images with container_format/image_format set to null cannot be published to Gnocchi | 09:07 |
|---|---|---|
| Callum027 | because the attributes are marked as "required" there, and I'm trying to determine the context/validity of those values being null before proposing a patch there. Thanks! | 09:07 |
| Callum027 | The Container and Disk Formats page in the docs don't help much, as they seem to imply that they are not in fact nullable: https://docs.openstack.org/glance/latest/user/formats.html | 09:09 |
| Callum027 | To give context, in our OpenStack deployment we have some Glance images that have container_format and disk_format both set to null in the database (and they return null when queried via the API as well), and I'm trying to figure out if that is correct or not. My initial assumption was that they should be using 'bare' container format and 'raw' | 09:15 |
| Callum027 | disk format, but now that I can see that images are nullable in the API I'm not sure. | 09:15 |
| dansmith | Callum027: that's news to me too.. absolutely should not be nullable | 13:25 |
| dansmith | Callum027: I don't think you can draw any such conclusions about images you have, and would highly recommend you probe them to determine their format before changing in the DB | 13:26 |
| dansmith | your operational security depends on it | 13:26 |
| dansmith | croeland1: job result looks pretty good right? | 13:31 |
| dansmith | croeland1: I have a conflict for the meeting time today, FYI | 13:38 |
| croeland1 | dansmith: yeah, it does look amazing | 13:56 |
| *** croeland1 is now known as croelandt | 13:56 | |
| croelandt | the grenade job also fails on master | 13:56 |
| croelandt | functional tests are expected t ofail | 13:56 |
| dansmith | yeah | 13:56 |
| croelandt | so this is pretty amazing, considering I just ripped off most of the eventlet stuff just hoping for the best | 13:57 |
| croelandt | we still lose the posthook functionality (do we care about it?) and the sqlite driver thing is annoying | 13:57 |
| croelandt | apart from that... | 13:57 |
| croelandt | Regarding the meeting I totally forgot :D | 13:57 |
| croelandt | mhen: did you have anything for today? We cannot merge encryption as this state unfortunately | 13:58 |
| mhen | if dansmith is around, I'd like to discuss possible ways forward | 14:02 |
| croelandt | he is not :-( | 14:02 |
| croelandt | rosmaita: I don't suppose you got much for the meeting? | 14:03 |
| rosmaita | i do not | 14:03 |
| croelandt | I think we might as well cancel as I had no specific topics and both abhishek and dansmith are away | 14:03 |
| croelandt | dansmith: do you think after ytour meeting you will have time to talk to mhen? | 14:03 |
| mhen | I can only be around for another hour or so for today | 14:08 |
| rajiv | Hi, is there glance meeting today ? | 14:11 |
| rosmaita | rajiv: i believe it has been cancelled due to lack of participants | 14:11 |
| rajiv | Hi rosmaita , i see, are there any known issues with application credentials and glance ? i am observing a strange issue wrt listing images | 14:12 |
| rajiv | 2-3 older images dont show up | 14:12 |
| croelandt | is there a launchpad for that? A reproducer? | 14:13 |
| rosmaita | how are you listing? | 14:13 |
| rosmaita | and you will kill me for saying this, but are you looking at the second page of results? | 14:14 |
| rajiv | i found this https://bugs.launchpad.net/keystone/+bug/2030061/comments/6 but i use custom policy hence it doesnt work ? | 14:14 |
| rajiv | i wanted to discuss it here before creating a launchpad bug | 14:14 |
| rajiv | i have several openstack domains, and in 1 custom domain where the IDP is using application credentials, | 14:15 |
| rosmaita | i wouldn't think that something like that would be affected by the age of the images | 14:15 |
| rajiv | 2-3 older images dont show up in openstack image list, however if i set --project-id it works | 14:16 |
| croelandt | hm | 14:16 |
| rajiv | yes, i doubted the same, hence wondering if there is something else | 14:16 |
| croelandt | Interesting | 14:16 |
| croelandt | Do you have a full reproducer? Can you create an image that won't be listed? | 14:16 |
| rajiv | os image list (doesnt show older images) ; os image list --project <project_id> lists all images | 14:16 |
| croelandt | yeah but can I create an image that won't be listed? | 14:17 |
| rajiv | yes | 14:18 |
| rosmaita | also, what are the credentials of the person making the call? admin or the project that owns the images | 14:18 |
| rajiv | yes, its admin, & image_admin | 14:18 |
| rajiv | owner of the images | 14:19 |
| croelandt | so what's the difference between the non-listed images and the listed ones? | 14:19 |
| rajiv | https://paste.opendev.org/show/blFtXQp9SiOf7T4SMuGA/ | 14:19 |
| rosmaita | can you do an image-show on each of those so we can see the full list of image properties? | 14:20 |
| croelandt | can you "openstack image show $ID" for rajiv-test-ubuntu-24-04-01 and cirros? | 14:20 |
| rosmaita | jinx! | 14:20 |
| croelandt | yeah what rosmaita said | 14:20 |
| rajiv | any particular info or value your looking for ? | 14:21 |
| rajiv | i will need to mask few details | 14:21 |
| rajiv | ah ok, few secs plz | 14:21 |
| rosmaita | feel free to mask anything sensitive, but mainly interested in os_hidden and visibility | 14:23 |
| rosmaita | and probably owner | 14:23 |
| rajiv | https://paste.opendev.org/show/boqhc9uQSRqjuJWYsehG/ | 14:23 |
| rajiv | os_hidden is false by default | 14:24 |
| rajiv | visibility is private for all iamges | 14:24 |
| rajiv | owner is the same project id for all images | 14:24 |
| croelandt | openstack image list --all ? | 14:27 |
| croelandt | I think it's --all, not sure | 14:27 |
| rosmaita | ok, if you add --debug when you use openstackclient, i believe it will print out the API requests it makes ... would be interesting to see what it does for os image list vs. os image list --project ### | 14:27 |
| rajiv | croelandt: it works will in non application credentials domain without --all as well | 14:27 |
| rajiv | rosmaita: yes, i did compare this but dint find any difference | 14:28 |
| rajiv | strangely. | 14:28 |
| rosmaita | that is strange indeed | 14:28 |
| rosmaita | it that's the case, i would say it's not a glance problem, it is probably that osc is doing some kind of filtering or something | 14:29 |
| rajiv | any idea why all images show up with "--project" flag ? | 14:29 |
| croelandt | rosmaita: well the glance cli has the same issue, right? | 14:29 |
| rosmaita | does it? | 14:30 |
| rajiv | let me check quickly | 14:30 |
| rajiv | doesnt work well with application creds which work with cred id and secret | 14:33 |
| rajiv | ➜ glance image-list You must provide a username via either --os-username or env[OS_USERNAME] | 14:33 |
| croelandt | what if you pass --os-username | 14:34 |
| rajiv | I tried it now, next is --os-password | 14:35 |
| rajiv | ➜ glance image-list 'Namespace' object has no attribute 'os_password' | 14:36 |
| croelandt | just export OS_* | 14:37 |
| rajiv | but application credentials doesnt support this right ? | 14:38 |
| croelandt | hm I'm not sure | 14:39 |
| croelandt | let's just try to useglanceclient here | 14:39 |
| croelandt | it's not a definitive solution, it's just debugging | 14:39 |
| rajiv | np, i guess i will raise a bug in glance or osc ? | 14:40 |
| croelandt | well if you could use glanceclient we'd know whether it's osc-related | 14:40 |
| rajiv | ah ok ok | 14:41 |
| rosmaita | even better would be to make API requests directly | 14:43 |
| rajiv | okay, thanks for the info, i will check and revert in next week's meeting | 14:44 |
| croelandt | I think with --debug --verbose you should see the API requests | 14:45 |
| opendevreview | Dan Smith proposed openstack/glance master: DNM: Test removing glance standalone mode https://review.opendev.org/c/openstack/glance/+/960620 | 15:30 |
| Callum027 | dansmith: I did some testing and I was able to create an "empty" image with container_format and disk_format set to null, but when I tried to upload an image using the PUT endpoint it gave me an error (in poorly formatted in HTML) saying that they needed to be set. Here is the Glance API response for one of the images I am looking into, it has no | 21:12 |
| Callum027 | container_format or disk_format but is in 'active' state... it appears to have been created from a snapshot using a block device mapping, and I'm unsure how this could have been done or whether it's valid or not. We only use RBD to back Glance, we don't enable any other backends. https://paste.openstack.org/show/brfK1PvWCtnHXTrnv8xM/ | 21:12 |
| dansmith | Callum027: yeah, I dunno if there's any reason for allowing that in the API other than oversight (which I hope is the case) | 21:19 |
| dansmith | but I think it's worth a bug report and hopefully we can just fix that either with defaults or a rejection | 21:19 |
| Callum027 | There's a change from a long time ago that enabled them being nullable in the APIs: https://review.opendev.org/c/openstack/glance/+/138183 | 21:19 |
| dansmith | Callum027: oh jeez.. well, I don't have anything nice to say about that, but at least some context | 21:20 |
| dansmith | I wonder if that's just "meh they're nullable in the DB schema" or what.. if so, that might've just been extending the oversight to the API, but who knows | 21:21 |
| dansmith | doesn't much make sense to me though, so I still think it's worth changing | 21:21 |
| Callum027 | I agree, I'm happy to create a bug report for this so it can be investigated | 21:21 |
| Callum027 | Is there anything you can tell me about the image in the paste I linked above? I'm a bit stumped as to how it was created | 21:22 |
| dansmith | er, what do you mean? | 21:22 |
| Callum027 | So we only use RBD to back our images, which have completely different metadata to the image I linked above, which appears to have been created from a snapshot using a block device mapping? But I have no idea how it was done | 21:23 |
| Callum027 | I'm trying to determine the origin of these images so if there's a problem there it can be fixed as well | 21:23 |
| Callum027 | I know you can create an image from a volume, but in our setup that results in a normal-looking image backed by a separate RBD block device | 21:24 |
| dansmith | I think since it has "snapshot_id" that it came either from cinder itself or from nova, of a cinder volume | 21:24 |
| dansmith | yeah, I really don't know.. nothing about it that tells me anything specific | 21:26 |
| dansmith | I assume you might be able to follow the snapshot_id into cinder and find the volume which might offer clues | 21:29 |
| Callum027 | In our pre-prod environment the snapshots these were created from no longer exist, but we have plenty of these in our prod that might still exist | 21:30 |
| Callum027 | Is it actually possible to create an image directly from a snapshot using the Cinder/Glance APIs? The volume_id field in the block device mapping metadata for these images are unset | 21:31 |
| Callum027 | My understanding was that you could only create them from a volume | 21:31 |
| dansmith | from an instance you mean? | 21:32 |
| Callum027 | No, a volume snapshot | 21:32 |
| dansmith | idk | 21:32 |
| dansmith | there's enough of a BDM here that it sure seems like it must have come from nova | 21:33 |
| Callum027 | Interesting, looks like you're right, I created an image from a server and it has much of the same metadata (and more, including an RBD block device because presumably this one is not bugged): https://paste.openstack.org/show/b5QcmCz0yaPG1a5qI2mt/ | 21:56 |
| Callum027 | I can also see it has the container and disk format set on it properly | 21:56 |
| Callum027 | I've got another one here in my testing env that's still in queued state, this one doesn't have them set yet (but as already established the API *does* allow this while the image is not actually uploaded yet): https://paste.openstack.org/show/boO6WWKgkXfPsUcazIWa/ | 21:58 |
| dansmith | yeah and recent (from this year) | 22:13 |
| dansmith | that one is created from osc I'm pretty sure because of the "owner_specified" stuff | 22:14 |
| dansmith | super bizarre, I didn't know they were even nullable until you brought it up, but I would have bet money that at least the clients had defaults | 22:14 |
| Callum027 | The client does have defaults, I'm hitting the API directly to set them to null explicitly and the API is accepting it | 22:19 |
| Callum027 | Though that last paste is one I created via openstack server image create | 22:19 |
| Callum027 | I'm unsure if Nova is explicitly setting it to null or it's just not setting it when it creates the image in Glance before uploading it. That makes me worried if it would cause compatibility issues making it non-nullable | 22:24 |
| Callum027 | So I'll make a bug on Launchpad for this issue and we can look into the implications of making the change | 22:26 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!