| *** mhen_ is now known as mhen | 02:04 | |
| mhen | sorry, I will not be able to attend today's meeting | 13:59 |
|---|---|---|
| croelandt | #startmeeting glance | 14:01 |
| opendevmeet | Meeting started Thu Sep 4 14:01:45 2025 UTC and is due to finish in 60 minutes. The chair is croelandt. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:01 |
| opendevmeet | The meeting name has been set to 'glance' | 14:01 |
| croelandt | #topic roll call | 14:01 |
| croelandt | o/ | 14:01 |
| dansmith | o/ | 14:02 |
| croelandt | abhishek_: around? | 14:02 |
| croelandt | #link https://etherpad.openstack.org/p/glance-team-meeting-agenda | 14:03 |
| abhishek_ | o/ | 14:03 |
| croelandt | Not much on today's agenda | 14:03 |
| croelandt | #topic Release/periodic job updates | 14:03 |
| croelandt | glance-multistore-cinder-import-fips fails... differently :) | 14:03 |
| croelandt | we finally moved to CentOS 10 to have a recent version of Python | 14:03 |
| abhishek_ | ack | 14:04 |
| croelandt | Now the jobs fails while setting up fips | 14:04 |
| croelandt | [Errno 2] No such file or directory: b'fips-mode-setup'" | 14:04 |
| dansmith | awesome | 14:04 |
| croelandt | I kind of need to figure out how ansible handles this but I cannot find the role for fips enablement | 14:04 |
| croelandt | I need to dig a bit deeper | 14:04 |
| croelandt | #topic Open Discussion | 14:05 |
| croelandt | Abhishek is on PTO starting next week, so I encourage everybody to throw stuff to review at him, because we probably will merge way fewer patches in the next three weeks as we usually do | 14:06 |
| abhishek_ | I have one, related to s3 checksum validation | 14:06 |
| croelandt | do you have a link? | 14:06 |
| abhishek_ | I think right now we should add a release note stating set the environment variables | 14:06 |
| croelandt | ok | 14:07 |
| abhishek_ | and next cycle fix it properly | 14:07 |
| croelandt | and later release a proper fix? | 14:07 |
| abhishek_ | let me share the idea | 14:07 |
| croelandt | Hm, why not fix in this cycle? | 14:07 |
| abhishek_ | https://paste.openstack.org/show/bIn61aqEtUVII8nnij9S/ | 14:08 |
| croelandt | oh you're turning this into a spec :) | 14:09 |
| abhishek_ | yes | 14:09 |
| croelandt | with very fine-grained control over the behaviour | 14:09 |
| croelandt | yes in that case it's a discussion for G | 14:09 |
| dansmith | so this is new boto change, which has an envar workaround for the time being, and we can address it later correct? | 14:09 |
| abhishek_ | right | 14:10 |
| croelandt | yes | 14:10 |
| croelandt | the issue is that for some users things are going to start failing | 14:10 |
| croelandt | which is really a boto issue, to be fair :) | 14:10 |
| abhishek_ | and only ceph rgw is having issue with this, with swift it is working | 14:10 |
| croelandt | I think Cinder *might* have issues with it | 14:10 |
| croelandt | so I am going to try and talk to ratailor Rajat or rosmaita about this, see if we can check | 14:11 |
| abhishek_ | I will submit the spec when I will be back and we can discuss this during PTG | 14:11 |
| croelandt | I also wonder how easily this fix can be backported | 14:11 |
| croelandt | ok | 14:11 |
| dansmith | yeah, so envar workaround seems fine to make sure we handle it properly | 14:11 |
| croelandt | yeah at first | 14:11 |
| abhishek_ | ack | 14:11 |
| croelandt | ideally things would be a bit smoother than "wow, it's failing now? OK gotta set some vars" | 14:12 |
| croelandt | this is so annoying | 14:12 |
| dansmith | we could also just pin boto for the time being right? | 14:12 |
| croelandt | hm, interesting | 14:13 |
| abhishek_ | hmm, but that might not allow us to use other features if there are any? | 14:13 |
| croelandt | no but we pin it to <1.36.0 for F | 14:13 |
| croelandt | and unpin it for G | 14:13 |
| abhishek_ | either env workaround or pin boto I am ok for anything | 14:13 |
| croelandt | but I think that is a change we'd have to implement in openstack/requirements | 14:14 |
| croelandt | so by pinning, we make sure the users never run into the issue | 14:14 |
| abhishek_ | yes | 14:14 |
| croelandt | and once they upgrade to G, they get glance_store with the fix and an up-to-date boto | 14:14 |
| croelandt | so it's worth considering | 14:14 |
| abhishek_ | ack | 14:14 |
| croelandt | ok I'll consult first with Cinder and see if they have the same issue | 14:14 |
| croelandt | because it would be nice for the both of us to implement the same fix for F | 14:14 |
| abhishek_ | Ok | 14:15 |
| dansmith | I mean.. this is not unusual for us to do | 14:15 |
| dansmith | unless there's some burning reason for us to have the newer thing | 14:15 |
| croelandt | dansmith: yeah, makes sense | 14:15 |
| croelandt | especially if we pin it temporarily | 14:15 |
| croelandt | I don't want this to turn into "the year is 2037, boto3 is still pinned to 1.35" | 14:15 |
| croelandt | :D | 14:15 |
| abhishek_ | :P | 14:16 |
| croelandt | OK, moving on | 14:16 |
| croelandt | #topic https://etherpad.opendev.org/p/2026.1-ptg-glance-planning | 14:16 |
| dansmith | we should be so lucky to still be relevant in 2037 :) | 14:16 |
| croelandt | You may start adding topics for the pTG at the end of october | 14:16 |
| croelandt | dansmith: the industry moves *slowly* :D | 14:16 |
| croelandt | nothing's urgent, but it would be nice not to fill that Etherpad in the last week :) | 14:17 |
| croelandt | #topic https://review.opendev.org/c/openstack/glance/+/958715 | 14:17 |
| croelandt | Last but not least, we should keep an eye on this patch | 14:17 |
| croelandt | there is a Cinder counterpart | 14:17 |
| abhishek_ | you can merge it once cinder is good to go | 14:18 |
| croelandt | yeah waiting on Cinder :) | 14:18 |
| croelandt | Anything else? | 14:18 |
| abhishek_ | nope, I am good | 14:19 |
| abhishek_ | I will be back in October 1st week, will be checking mail frequently | 14:19 |
| dansmith | abhishek_: have a good one! | 14:19 |
| croelandt | please don't check email too frequently | 14:20 |
| abhishek_ | thank you :D | 14:20 |
| croelandt | Please go to places with no 4G | 14:20 |
| abhishek_ | croelandt: ack :P | 14:20 |
| croelandt | #endmeeting | 14:20 |
| opendevmeet | Meeting ended Thu Sep 4 14:20:52 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:20 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/glance/2025/glance.2025-09-04-14.01.html | 14:20 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/glance/2025/glance.2025-09-04-14.01.txt | 14:20 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/glance/2025/glance.2025-09-04-14.01.log.html | 14:20 |
| abhishek_ | croelandt: so are you going to look at pinning 1.36 boto ? | 15:15 |
| abhishek_ | < 1.36 i mean | 15:15 |
| abhishek_ | In that case we need to do it for master and then for epoxy as well | 15:16 |
| croelandt | abhishek_: trying to get Cinder's attention right now so both projects do the same thing | 15:34 |
| croelandt | I don't want users to have to do different things for different projects | 15:35 |
| dansmith | you can't.. | 15:35 |
| dansmith | you need to pin it in u-c AFAIK, which will affect everyone | 15:35 |
| clarkb | yes upper constraints should win during installations in the gate | 15:36 |
| croelandt | dansmith: yes but if we do that, once we unpin it, Cinder starts failing | 15:46 |
| croelandt | I'm trying to figure out whether they have the same issue (quite likely) and how we end up fixing it (leaving the user to set env variables? Providing config options?) | 15:47 |
| dansmith | croelandt: that's how this always works.. the unpin helps us find out if anything is broken and we don't unpin until it's fixed | 15:47 |
| croelandt | ok wait a minute | 15:49 |
| croelandt | patch incoming | 15:49 |
| croelandt | https://review.opendev.org/c/openstack/requirements/+/959560 | 16:00 |
| dansmith | croelandt: yeah so unless that causes cinder heartache it seems like that's a good plan to me | 16:09 |
| croelandt | yeah, let's get the requirements patch merged | 16:09 |
| clarkb | do we use any problematic backends in the gate? (like maybe swift's s3 backend or cephs?) It would probably be helpful to specify those details too so we know why we need to modify thinsg for the gate | 16:11 |
| clarkb | makes it easier to unpin later as it will be more clear what the original conflict was | 16:12 |
| dansmith | clarkb: u-c is not just for the gate right? | 16:12 |
| croelandt | clarkb: I don't think we broke the gate, but as dansmith says, u-c is not just for the gate? Or do I need to amend that patch? | 16:13 |
| clarkb | dansmith: that is what it is for aiui. To ensure we test all the services with a consistent set of dependencies and to reduce dep solving time | 16:13 |
| clarkb | fungi is constantly trying to get the deployment projects to stop relying on it or manage their own version of it | 16:14 |
| dansmith | clarkb: I thought the only reason we really still have it is because of the distros not knowing what their bounds are | 16:14 |
| clarkb | u-c exists because once or twice a week a python dep would update and break all of openstack's gate and untangling that was a mess so now we update things in lockstep as they are made to work to keep the gate working | 16:14 |
| croelandt | so should I set < 1.36.0 directly in requirements.txt for glance? | 16:14 |
| clarkb | dansmith: requirements.txt is supposed to set the bounds | 16:14 |
| clarkb | constraints is for the gate. That is my interpretation anyway | 16:15 |
| croelandt | clarkb: should I ask for clarification on #openstack-infra? | 16:15 |
| clarkb | croelandt: no we don't manage the constraints. The requirements team can weigh in on it and can probably do so via the change | 16:15 |
| croelandt | I mean, is there a channel where I can ask whether your interpretation is the right one? | 16:16 |
| clarkb | I think they have an irc channel too yes | 16:16 |
| dansmith | okay, well, if that's the case, then we should pin in requirements at least | 16:16 |
| dansmith | and I guess it seems weird to have u-c be higher than the upper of requirement | 16:16 |
| clarkb | dansmith: no u-c should be within the requirements bounds | 16:17 |
| croelandt | yeah so we need to change in both places | 16:17 |
| clarkb | but all of this is sort of tertiary to what I was trying to convey | 16:17 |
| clarkb | the change as proposed does not provide sufficient detail to know when to unpin things | 16:17 |
| clarkb | that is a bug | 16:17 |
| croelandt | well | 16:17 |
| clarkb | especially if the gate doesn't give us the signal for "things are ok now" | 16:17 |
| croelandt | we unpin when glance_store is fixed? :) | 16:18 |
| croelandt | I have no better answer for now | 16:18 |
| clarkb | croelandt: yes but 12 months from now the people looking at this may not be us and not have this context anymore | 16:18 |
| croelandt | yeah I tried to convey context in the commit log | 16:18 |
| croelandt | there is no test that is going to go from red to green | 16:18 |
| clarkb | right and I suggested that a critical piece of info was missing | 16:18 |
| croelandt | because I don't think we have that kind of backend in the gate | 16:18 |
| clarkb | specifically which backends are affected | 16:19 |
| croelandt | oh the S3 backend | 16:19 |
| clarkb | so that they can be tested one way or another or support can be dropped for them etc | 16:19 |
| croelandt | isn't that clear from the commit log? | 16:19 |
| croelandt | I thought it was | 16:19 |
| croelandt | and *possibly* the Cinder S3 storage backend, but we're still trying to figure this out | 16:19 |
| clarkb | croelandt: its not just the s3 backend | 16:19 |
| clarkb | its the s3 backend with implementations that are not s3. Which implementations? | 16:20 |
| clarkb | if you use the s3 backend with boto and s3 everything should work fine | 16:20 |
| croelandt | yes | 16:20 |
| croelandt | so Ceph RGW would be one of them | 16:20 |
| dansmith | clarkb: the commit explains the reason, to me | 16:20 |
| croelandt | but I don't really have a list of S3 implementations so it's hard to say | 16:20 |
| croelandt | it's basically "whatever S3-looking backend people might want to use" | 16:21 |
| clarkb | croelandt: ya so i would list ceph rgw s3 api as that is something specific that can be tested | 16:21 |
| croelandt | I see | 16:21 |
| clarkb | then 12 months from now when we want to unpin someone can test that and if it works make a better educated decision on whether or not it is safe to unpin | 16:21 |
| croelandt | Yes, I wish it were simpler and just a matter of running a test :/ | 16:22 |
| fungi | including an adjacent todo comment linking to a bug url or fixing commit that needs to appear in a new release is preferable, just to help future-you not need to fall back on git blame | 16:22 |
| clarkb | it is easy for these nebulous "some backends don't work" pins to never get unpinned out of fear for breaking something. Then when a situation makes it so that we really want to update (api changes or security fixes whatever) its tough to make a good decision on which break is more acceptable | 16:23 |
| clarkb | so I was merely trying to convey that providing as much detail as possible including the currently known broken backends may be helpful in the future | 16:23 |
| croelandt | ok, updated with a Launchpad link | 16:25 |
| croelandt | and example of failing backend | 16:25 |
| clarkb | thanks! | 16:25 |
| clarkb | python version updates are another common reason we end up needing to bump things | 16:25 |
| croelandt | another layer of complexity | 16:30 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!