21:00:35 <timburke> #startmeeting swift
21:00:35 <opendevmeet> Meeting started Wed Oct 30 21:00:35 2024 UTC and is due to finish in 60 minutes.  The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:35 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:35 <opendevmeet> The meeting name has been set to 'swift'
21:00:43 <timburke> who's here for the swift meeting?
21:02:19 <indianwhocodes> o/
21:04:00 <timburke> first up
21:04:05 <timburke> #topic ptg
21:04:13 <timburke> it was last week!
21:04:16 <mattoliver> o/
21:04:26 <mattoliver> (sorry internet issues)
21:04:40 <timburke> thanks to everybody for coming -- i know it was early/late a lot of days for a lot of people
21:04:54 <timburke> but i think we had a lot of good discussions!
21:05:25 <timburke> if there are any other notes we should write up about them, please add them to the etherpad while they're fresh in your heads
21:05:27 <fulecorafa> o/
21:05:33 <mattoliver> it was great, just need to get stuff done while the fire's still lit on some of those discussions :)
21:05:34 <timburke> #link https://etherpad.opendev.org/p/swift-ptg-epoxy
21:06:08 <timburke> speaking of fires lit by those discussions...
21:06:13 <timburke> #topic pyeclib release
21:06:39 <timburke> there's been one! in fact, there've been 4 (if you count the pre-release)
21:07:29 <timburke> big main thing i wanted out of it was to start publishing binary wheels so `pip install pyeclib` won't require a build environment
21:07:46 <timburke> and we did it! eventually ;-)
21:08:32 <timburke> the first three wound up including some AVX2 instructions that weren't available on all of the various CI nodes
21:08:45 <timburke> so sorry for that gate breakage
21:09:15 <mattoliver> ahh, so that's what was happening!
21:09:22 <timburke> but it got fixed up by p 933338
21:09:22 <patch-bot> https://review.opendev.org/c/openstack/pyeclib/+/933338 - pyeclib - wheels: Disable optimizations for liberasurecode (MERGED) - 1 patch set
21:10:24 <timburke> i realized that this could also be a good opportunity to slim down some of our requirements in swift, if we could rely on pyeclib providing liberasurecode
21:10:41 <timburke> so i proposed p 933697
21:10:41 <patch-bot> https://review.opendev.org/c/openstack/swift/+/933697 - swift - Pull out a bunch of liberasurecode requirements - 2 patch sets
21:11:12 <timburke> ...but then realized that i need to wait on that until the new pyeclib release gets into the global upper-contraints file
21:12:25 <timburke> (i see a couple general "Updated from generate-constraints" patches that include it, but i may write a dedicated patch to just bump pyeclib)
21:12:59 <timburke> speaking of gate failures...
21:13:05 <timburke> #topic py38 gate failures
21:13:24 <timburke> this was another thing that came up toward the end of last week
21:16:06 <timburke> with global requirements dropping py38 support in p 925201 our py38 jobs stopped being able to find a valid version of some transitive dep to install
21:16:07 <patch-bot> https://review.opendev.org/c/openstack/requirements/+/925201 - requirements - Remove Python 3.8 tests and constraints (MERGED) - 5 patch sets
21:16:45 <mattoliver> oh yay
21:16:50 <indianwhocodes> damn
21:17:07 <indianwhocodes> from 2 to 3.9
21:17:10 <mattoliver> so since PTG you've bacially been fighting the gate, thanks for all that Tim
21:17:11 <timburke> i cleared it up for now in p 933363 by doing the same basic thing that we did when py37 got dropped
21:17:11 <patch-bot> https://review.opendev.org/c/openstack/swift/+/933363 - swift - CI: use py36 constraints for py38 (MERGED) - 1 patch set
21:17:44 <indianwhocodes> nicee
21:17:45 <timburke> but i'm realizing that the existing py36-constraints.txt file isn't really a good solution
21:17:53 <indianwhocodes> it isn't
21:18:04 <indianwhocodes> it was never a good soln lol
21:18:40 <clarkb> one option that might be worth trying is running without constraints since many things drop support for older python anyway you may get implicit constrints due to having an upper bound on the library side
21:18:41 <timburke> i'm trying to get it to a *slightly* better place with p 933680, but even that's probably still not ideal
21:18:41 <patch-bot> https://review.opendev.org/c/openstack/swift/+/933680 - swift - CI: Drag forward more constraints - 4 patch sets
21:19:09 <clarkb> of course you always run the risk of having things update and breaking you, but maybe that will be infrequent enough with older python versiosn due to the python requires usage in pypi packages
21:19:33 <indianwhocodes> p 927883
21:19:33 <patch-bot> https://review.opendev.org/c/openstack/swift/+/927883 - swift - manage py36 constraints for swift (MERGED) - 8 patch sets
21:20:12 <timburke> this manually-managed constraint file is causing grief in other ways, too -- mainly because we've been using it for our func-py3 tox environment
21:20:28 <indianwhocodes> true ^^^^
21:21:30 <timburke> this is causing trouble for devstack trying to move to ubuntu noble/py312, because our constraint is pinning greenlet to a pretty ancient version which can't compile for py312
21:21:57 <timburke> see p 931697
21:21:57 <patch-bot> https://review.opendev.org/c/openstack/devstack/+/931697 - devstack - Switch devstack nodeset to Ubuntu 24.04 (Noble) - 1 patch set
21:23:56 <timburke> i think the more-general py3-constraints.txt that i've proposed will get things squared -- or there's also the initial patchset of https://review.opendev.org/c/openstack/swift/+/933369/1 that tries to move the func-py3 jobs back to using global constraints
21:23:57 <patch-bot> patch 933369 - swift - CI: Move a bunch of func test jobs from py38 to py312 - 2 patch sets
21:24:46 <timburke> ...but both of those trip func test failures in https://github.com/openstack/swift/blob/master/test/functional/s3api/test_xxe_injection.py
21:25:13 <timburke> so i think i'm going to need to spend some time hunting down what changed there :-/
21:26:34 <timburke> it seems likely to be related to our using boto (not boto3) for the test, which in turn expects a *really old* version of requests
21:26:39 <timburke> see also https://bugs.launchpad.net/swift/+bug/1557260
21:26:39 <patch-bot> Bug #1557260 - Swift3 functests should transition to using boto3 (New)
21:27:56 <indianwhocodes> noooo
21:28:20 <indianwhocodes> lol
21:28:50 <timburke> anyway, i think that's most of what i've got to say about the gate right now -- there's surely more work that can be done, and there's been a bunch of deferred maintenance that seems to be coming due
21:29:55 <timburke> #topic https://bugs.launchpad.net/swift/+bug/2081103
21:29:56 <patch-bot> Bug #2081103 - s3api: Deleting the current version of an object can (sometimes?) 500 (In Progress)
21:30:18 <timburke> we still haven't fixed fulecorafa's bug :-(
21:30:35 <timburke> if anybody has time to review p 931325 i'd appreciate it
21:30:35 <patch-bot> https://review.opendev.org/c/openstack/swift/+/931325 - swift - versioning: 411 PUTs with neither content-length n... - 2 patch sets
21:30:58 <fulecorafa> timburke that's ok. I saw ou opened a PR in eventlet github right?
21:31:26 <timburke> yes! https://github.com/eventlet/eventlet/pull/985
21:31:54 <timburke> i should follow up on that; they requested some clarification in the commit message
21:32:22 <fulecorafa> How's it going there? I took a look at it, LGTM but I saw no one else reviewed it right?
21:32:29 <timburke> but the swift patch should be sufficient
21:32:49 <fulecorafa> Oh ok. Then I will look at it ASAP
21:34:09 <timburke> yeah, no one else has looked yet -- if you could try it out in a pre-prod environment, say, and leave a review for how it went, that'd be awesome
21:34:28 <mattoliver> me too. I'll put it on my list. Should get some swift work done (sick of banging my head against the downstream work wall) :)
21:34:38 <mattoliver> +1
21:34:41 <fulecorafa> I will try it, yes
21:34:46 <timburke> thanks
21:35:17 <fulecorafa> About testing, I will bring a topic at open discussion that may be interesting
21:35:31 <timburke> next up
21:35:47 <timburke> #topic multi-policy containers
21:36:00 <timburke> fulecorafa, how's it going? just wanted to check in since it's been a bit
21:36:27 <fulecorafa> Oh, didn't I metion it? We got it working in prod right now
21:37:03 <timburke> yes! i remember that part :-) wanted to see if we should be trying to block out some review time in the near future
21:37:43 <fulecorafa> We're activelly working on bringing it up to date now so we can patch it in swift, but there is quite a lot to do in that regard
21:38:19 <timburke> fair enough -- it's tricky rebasing a large body of work
21:38:30 <fulecorafa> Sorry if we're slow in bringing things here, it's been a pain point to have an outdated version of swift running
21:38:40 <timburke> especially in s3api, which sees no small amount of churn
21:39:18 <fulecorafa> In regards to implementation, we did it through a middleware, with little changes to s3api actuall
21:40:28 <timburke> yeah, totally makes sense to prioritize being able to run newer swift over getting the new feature merged -- in fact, it's probably a necessary pre-req
21:41:11 <fulecorafa> I think somethings wwe have done are not even patch-able, which is the most hold up
21:41:40 <fulecorafa> As in, we need to rewrite it so it is even possible to make it work with newer code
21:41:58 <timburke> ah, yeah. sorry :-/
21:42:31 <fulecorafa> But we're trying, it will just take some time to get it ok
21:42:54 <timburke> out of curiosity, where's the friction turning up? we usually try to not be terribly disruptive to third-party middlewares
21:44:33 <fulecorafa> I would say it comes down to patches in s3api, pro controller and related. This patches are not easily migratable due to changes in function name/code placement/functionality changes
21:44:46 <timburke> ah
21:44:49 <timburke> all right, then i think that's all i've got
21:44:54 <timburke> #topic open discussion
21:45:00 <timburke> what else should we discuss?
21:45:13 <fulecorafa> While the midddleware could be done through a simple `git apply`, these patches would be necessary to rewrite and reconsider
21:45:26 <fulecorafa> I've got a topic regarding testing environment
21:46:13 <timburke> go ahead!
21:46:36 <fulecorafa> We've been trying to use both the oficial SAIO docker image to test and the NVidia one, but is very problematic, mostly to the wa we do things, but also with some breaks with errors not related to our active development
21:47:33 <fulecorafa> So we've been working with a lxc environment, which has come to be pretty cool. Now we're automating everthing, trying to take off developper hands apply and patch and start servers within lxd
21:48:07 <fulecorafa> Would this be an addition to the project? Would you like to have a look at it?
21:49:40 <timburke> nice! i remember notmyname tinkering with lxc/lxd ages ago, but can't find it now...
21:49:57 <mattoliver> I do like a good lxd container, havn't used one lately. I'd love to take a look
21:50:11 <mattoliver> I tinkered years ago, with notmyname too. So might be cool
21:50:25 <fulecorafa> Nice
21:50:29 <timburke> maybe? i'd be happy to look, anyway. fwiw, most of us have tended to settle on https://github.com/NVIDIA/vagrant-swift-all-in-one/ for a dev env
21:50:51 <fulecorafa> I'll talk with my side to make this code available. Any tips on sharing with ou guys?
21:51:30 <timburke> mattoliver, ah, good! you have a copy! https://github.com/matthewoliver/runway
21:52:33 <timburke> though it looks like https://github.com/kevin-wyx/runway might have a slightly more recent snapshot
21:52:37 <mattoliver> there you go :P never remove a cloned repo :P
21:52:53 <mattoliver> yeah probably
21:53:59 <timburke> fulecorafa, it's up to you -- if you'd can and would like to make it public, that's probably easiest, but you could also add us separately to some private repo if you'd prefer
21:54:46 <fulecorafa> Nice, I'll discuss making the repo public then with the other authors. Thanks a lot
21:55:08 <timburke> sure thing! thanks for wanting to make the swift developer experience better!
21:55:29 <mattoliver> +100
21:55:45 <mattoliver> I've finally pushed up a new version of https://review.opendev.org/c/openstack/swift/+/916861
21:55:46 <patch-bot> patch 916861 - swift - db_auditor: add vacuum support - 11 patch sets
21:56:33 <mattoliver> That one dumps the bucket bloatiness stats to recon and sends them as gauges in statsd at the end of an audit cycle.
21:57:03 <mattoliver> so it's the gauge for the last run. which seems to make more sense to me.
21:57:44 <mattoliver> It also adds statsdclient.gauge so we can use gauges now.
21:59:18 <timburke> i think that makes sense enough -- it's tricky striking the right balance between emitting stats frequently enough that your graph/timeseries db doesn't forget about the value and slowly enough that you can roll up some useful info
21:59:27 <mattoliver> thats all I really have to report on my end
21:59:40 <timburke> i see that it tripped https://bugs.launchpad.net/openstacksdk/+bug/2085654 :-/
21:59:40 <patch-bot> Bug #2085654 - Two intermittent functional test failures (New)
21:59:57 <timburke> at least i finally wrote up a bug about it, though!
22:00:07 <mattoliver> yay
22:00:27 <timburke> all right, we're at time -- i oughta let mattoliver get on with his morning ;-)
22:00:38 <timburke> thank you all for coming, and thank you for working on swift!
22:00:41 <timburke> #endmeeting