| opendevreview | Tim Burke proposed openstack/swift master: Refactor some file-like iters as utils.InputProxy subclasses https://review.opendev.org/c/openstack/swift/+/940059 | 16:36 |
|---|---|---|
| opendevreview | Tim Burke proposed openstack/swift master: Add support of Sigv4-streaming https://review.opendev.org/c/openstack/swift/+/836755 | 18:24 |
| opendevreview | Alistair Coles proposed openstack/swift master: Refactor and add some encrypter unit tests https://review.opendev.org/c/openstack/swift/+/942843 | 18:28 |
| opendevreview | Alistair Coles proposed openstack/swift master: encrypter: always send an override etag footer https://review.opendev.org/c/openstack/swift/+/942844 | 18:28 |
| opendevreview | Tim Burke proposed openstack/swift master: Refactor some file-like iters as utils.InputProxy subclasses https://review.opendev.org/c/openstack/swift/+/940059 | 18:40 |
| opendevreview | Tim Burke proposed openstack/swift master: Add support of Sigv4-streaming https://review.opendev.org/c/openstack/swift/+/836755 | 18:40 |
| opendevreview | Tim Burke proposed openstack/swift master: CI: run S3 cross-compat tests as part of DSVM jobs https://review.opendev.org/c/openstack/swift/+/942133 | 19:52 |
| opendevreview | Tim Burke proposed openstack/swift master: Add config option for whether to run s3_acl-requiring tests https://review.opendev.org/c/openstack/swift/+/942850 | 19:52 |
| mattoliver | morning | 21:00 |
| timburke | o/ | 21:00 |
| timburke | #startmeeting swift | 21:00 |
| opendevmeet | Meeting started Wed Feb 26 21:00:43 2025 UTC and is due to finish in 60 minutes. The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot. | 21:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 21:00 |
| opendevmeet | The meeting name has been set to 'swift' | 21:00 |
| timburke | who's here for the swift meeting? | 21:00 |
| mattoliver | o/ | 21:00 |
| boosungkim[m] | hello | 21:01 |
| timburke | as usual, the agenda's at | 21:01 |
| timburke | #link https://wiki.openstack.org/wiki/Meetings/Swift | 21:01 |
| timburke | first up | 21:02 |
| timburke | #topic failing DSVM jobs | 21:02 |
| timburke | so a recent python-openstackclient release broke keystone credential creation, which our DSVM jobs use so we can test keystone+s3api | 21:02 |
| timburke | the good news is there's a fix which has merged | 21:03 |
| timburke | #link https://review.opendev.org/c/openstack/python-openstackclient/+/942721 | 21:03 |
| mattoliver | oh nice | 21:03 |
| timburke | and there's even been a release to include the fix | 21:03 |
| timburke | #link https://review.opendev.org/c/openstack/releases/+/942834 | 21:04 |
| timburke | the bad news is that the update to global requirements is going to fail | 21:04 |
| timburke | #link https://review.opendev.org/c/openstack/requirements/+/942845 | 21:04 |
| timburke | ...because all the recent jobs have been failing: https://zuul.opendev.org/t/openstack/builds?job_name=requirements-tox-py312-check-uc&project=openstack/requirements | 21:05 |
| timburke | that leaves us with a few options: | 21:06 |
| timburke | 1. just wait until this all gets resolved | 21:06 |
| timburke | 2. mark our dsvm jobs non-voting until it gets resolved | 21:07 |
| timburke | 3. stop testing keystone+s3api until it gets resolved | 21:07 |
| timburke | i'm kind of inclined toward 1 (or at least give things another day or two before doing anything else) | 21:08 |
| mattoliver | how long do we think the wait is? | 21:10 |
| timburke | dunno -- but https://review.opendev.org/c/openstack/swift/+/940059 has been trying to merge for a couple days now | 21:10 |
| timburke | eh, i'll at least give it till the end of the week, re-evaluate monday | 21:11 |
| timburke | speaking of requirements, i should go poke someone about https://review.opendev.org/c/openstack/requirements/+/941708 ... it'd be nice to be able to use subtests | 21:12 |
| mattoliver | kk, sounds like a plan then | 21:12 |
| jianjian | btw, what's DSVM? | 21:12 |
| timburke | good question! "devstack vm" -- it's a job that builds on https://github.com/openstack/devstack/ which is kind of like a SAIO but for all of openstack | 21:13 |
| mattoliver | its some intergrated rest of openstack tests via openstack devstacks (the main openstack dev env tool). | 21:14 |
| jianjian | oh, I see. that's why you also suggested second option "mark our dsvm jobs non-voting until it gets resolved" | 21:14 |
| timburke | between 2 and 3, i'm kinda torn: on the one hand, ignoring the whole test suite isn't great, but on the other, at least there'll be a constant reminder that the temporary fix is in place since those job results will have a (non-voting) flag | 21:16 |
| timburke | we'll cross that bridge when we get there | 21:17 |
| timburke | #topic PTG | 21:17 |
| jianjian | okay, thanks for trying to fix it | 21:17 |
| timburke | i almost forgot, but i set up a time slot poll and put up a skeleton etherpad for topics | 21:18 |
| timburke | #link https://framadate.org/LBfzMX2I4C1JL9cZ | 21:18 |
| timburke | #link https://etherpad.opendev.org/p/swift-ptg-flamingo | 21:19 |
| timburke | and again, that'll be Apr 7-11 | 21:19 |
| jianjian | \o/ | 21:19 |
| mattoliver | cool, nice one timburke I'll fill out what I can of both :) | 21:20 |
| timburke | i should double check whether they've mentioned official time slots yet or not -- i made the poll with the assumption that they'd have the same slots as the last several vPTGs | 21:21 |
| timburke | though as mattoliver mentioned last week, it might not matter too much given that we could just all join a meetpad at some time regardless | 21:22 |
| timburke | all right, next up | 21:23 |
| timburke | #topic aws-chunked | 21:23 |
| timburke | i think acoles and i are coming around to feeling like the new protocol's mostly ready | 21:24 |
| timburke | #link https://review.opendev.org/c/openstack/swift/+/836755 | 21:24 |
| timburke | there are some rough edges, but we can always refine it on master | 21:24 |
| timburke | i think the biggest hesitation to merging it right now is that we don't respect any additional checksums that might be sent -- so clients may not have the data-integrity guarantees they were expecting | 21:25 |
| timburke | those patches still seem a little ways out -- there are some complications involving getting extra data into container listings and sending some metadata in footers | 21:27 |
| mattoliver | yeah. If we land the first patch, do you think we can get the algorithm follow ups in before the next swift release? | 21:27 |
| mattoliver | if so, I'd worry less because they're coming. | 21:27 |
| timburke | i'd be nervous about that -- we need to get a swift release out in the pretty near term since we're getting to the end of the cycle | 21:28 |
| timburke | one thought we had though was to get *partial* support -- just enough that we can reject uploads if checksums don't match, but don't bother writing anything down | 21:29 |
| mattoliver | what would be the minimal we'd need to support initially? crc64? | 21:30 |
| timburke | but i need to see how much of it we need to implement to make sure MPU uploads both work and protect against bit-flips | 21:30 |
| mattoliver | I wouldn't mind getting some crc support in swift, even if it just starts in s3api, then we can use it more internally later. | 21:31 |
| mattoliver | yeah kk | 21:31 |
| timburke | iirc boto3 defaults to crc32, but some other clients are known to use crc32c. from docs, it seems like AWS is pushing for crc64-nvme | 21:31 |
| mattoliver | see I'm on the fence. I want the chunked stuff in so people can use the most recent boto and it works. | 21:32 |
| timburke | fortunately the actual algos aren't terribly difficult to implement -- the tricky parts are storing the results, and combining partial crcs to give a full-object crc for MPUs | 21:33 |
| timburke | same. it sucks that what was previously a pretty well-supported client is now broken by default | 21:33 |
| mattoliver | Does the patch need to have a doc warning that aws-chunked is partially supported with the extra argoritim caveats? | 21:34 |
| mattoliver | Then we can land it, but make it known the rest is still in development. | 21:35 |
| mattoliver | I mean we'd still md5 behind the scenes, we just don't check their given crcs | 21:35 |
| timburke | i think for the most part *users* don't really know or care -- the *client* is just making decisions and the user is trusting it. so idk that a doc fix is really the right way to look at it -- the user probably neither knows nor cares about that level of implementation detail | 21:36 |
| mattoliver | yeah, but it covers our butt :P | 21:37 |
| timburke | i think as much as anything i just wanted to give some context -- there might be some new patches to split it like "respect additional checksums on upload" / "store additional checksums" | 21:38 |
| timburke | we'll see how my investigation goes | 21:38 |
| timburke | last topic from me | 21:39 |
| timburke | #topic swift release | 21:39 |
| timburke | it's that time in the cycle again! | 21:39 |
| mattoliver | \o/ | 21:39 |
| timburke | we already have a new swiftclient release out (not that it has any material changes from the last release) | 21:39 |
| mattoliver | shall we update swift priority reviews with patches we might want to try and include (those that are close) in the next release? | 21:40 |
| mattoliver | And we might need to swift-bench release :P | 21:40 |
| timburke | so i'm going start getting release notes together for a swift release | 21:40 |
| timburke | yeah, priority reviews page is probably a good idea | 21:40 |
| mattoliver | maybe just replace it with ssbench, swift-bench ng :P | 21:40 |
| timburke | we've got a few patches that already have +2s | 21:41 |
| timburke | #link https://review.opendev.org/q/p:openstack/swift+label:Code-Review%252B2+is:open+-age:26w | 21:41 |
| timburke | #undo | 21:41 |
| opendevmeet | Removing item from minutes: #link https://review.opendev.org/q/p:openstack/swift+label:Code-Review%252B2+is:open+-age:26w | 21:41 |
| timburke | #link https://review.opendev.org/q/p:openstack/swift+label:Code-Review%2B2+is:open+-age:26w | 21:41 |
| timburke | ...but again, we need a working gate before we can land anything anyway ;-) | 21:42 |
| timburke | *including* release notes :P | 21:42 |
| mattoliver | lol | 21:43 |
| jianjian | haha | 21:43 |
| timburke | so that's a decent part of why i'm thinking about contingencies in case the requirements stuff takes a bit to resolve | 21:43 |
| jianjian | I remember Tim said he would like this patch to land into this release: https://review.opendev.org/c/openstack/swift/+/938631 | 21:44 |
| mattoliver | yeah, well I lean towards 1 then 3 then 1. So we still get intergration testing and just loose keystone + s3api for a bit. | 21:44 |
| mattoliver | s/1/2/ | 21:44 |
| jianjian | I had +1, Clay had +2, Tim, are you still planning to do anything, or we should just merge it? | 21:45 |
| timburke | jianjian, yeah, i still think that's a good idea | 21:45 |
| mattoliver | yup, add it to prioirty reviews | 21:45 |
| timburke | i've not given the patch much though in weeks. let's merge it if y'all think it's ready | 21:45 |
| timburke | much thought* | 21:46 |
| jianjian | okay, I will look at it again later this week | 21:46 |
| timburke | thansk | 21:47 |
| timburke | man, i can't type today | 21:47 |
| jianjian | 😉 | 21:47 |
| timburke | all right, that's all i've got | 21:47 |
| timburke | #topic open discussion | 21:47 |
| timburke | anything else we should bring up this week? | 21:48 |
| timburke | oh! a couple weeks ago, i saw someone wanting to clean up some of the pyeclib test matrix: https://review.opendev.org/c/openstack/pyeclib/+/941122 | 21:49 |
| timburke | which refreshed my memory about my own plan to do that: https://review.opendev.org/c/openstack/pyeclib/+/938098 | 21:49 |
| timburke | we should decide what versions of python we actually want to be supporting -- currently pyeclib is testing against 10 different minor versions! | 21:50 |
| timburke | i want to cut that in half | 21:51 |
| mattoliver | oh cool. pyeclib, I'll take a look | 21:51 |
| timburke | thanks mattoliver -- if you like that one, there's a whole chain of stuff we could look at cleaning up since we don't have to worry about old python | 21:52 |
| timburke | all right, i think i'll call it | 21:54 |
| timburke | thank you all for coming, and thank you for working on swift! | 21:54 |
| timburke | #endmeeting | 21:54 |
| opendevmeet | Meeting ended Wed Feb 26 21:54:14 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 21:54 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/swift/2025/swift.2025-02-26-21.00.html | 21:54 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/swift/2025/swift.2025-02-26-21.00.txt | 21:54 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/swift/2025/swift.2025-02-26-21.00.log.html | 21:54 |
| mattoliver | good riddance py2! | 21:54 |
| timburke | ok, i think i understand the requirements failure now -- new setuptools oughta fix things too for more details, see https://github.com/pypa/setuptools/issues/4853 | 22:17 |
| opendevreview | Tim Burke proposed openstack/swift master: Add config option for whether to run s3_acl-requiring tests https://review.opendev.org/c/openstack/swift/+/942850 | 22:27 |
| opendevreview | Tim Burke proposed openstack/swift master: CI: run S3 cross-compat tests as part of DSVM jobs https://review.opendev.org/c/openstack/swift/+/942133 | 22:27 |
| opendevreview | Tim Burke proposed openstack/pyeclib master: CI: bump up timeout for pyeclib-build-wheels-arm64 https://review.opendev.org/c/openstack/pyeclib/+/942868 | 22:33 |
| opendevreview | Tim Burke proposed openstack/swift master: tests: Fix versioning cross-compat tests https://review.opendev.org/c/openstack/swift/+/942382 | 23:24 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!