21:00:34 <timburke> #startmeeting swift
21:00:34 <opendevmeet> Meeting started Wed Jan 19 21:00:34 2022 UTC and is due to finish in 60 minutes.  The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:34 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:34 <opendevmeet> The meeting name has been set to 'swift'
21:00:41 <timburke> who's here for the swift meeting?
21:00:47 <mattoliver> o/
21:00:48 <kota> o/
21:00:50 <seongsoocho> o/  hi
21:00:52 <acoles> \o
21:01:51 <timburke> as usual, the agenda's at https://wiki.openstack.org/wiki/Meetings/Swift
21:02:08 <timburke> first up, in case you missed it on the ML
21:02:14 <timburke> #topic z cycle naming
21:02:23 <timburke> #link http://lists.openstack.org/pipermail/openstack-discuss/2022-January/026620.html
21:02:38 <timburke> we can propose names on the wiki at
21:02:41 <timburke> #link https://wiki.openstack.org/wiki/Release_Naming/Z_Proposals
21:03:38 <mattoliver> Zorro!
21:03:55 <mattoliver> 😜
21:04:20 <timburke> nominations close monday, then as i recall, the TC votes for the final name
21:04:49 <timburke> on to swift-y topics!
21:04:56 <timburke> #topic updater fairness
21:05:26 <timburke> the first patch in the series (to add per-container ratelimits) landed -- https://review.opendev.org/c/openstack/swift/+/820962
21:06:18 <timburke> the second one (to add a queue of deferred updates to chew on when the disk sweep is shorter than the configured interval) is still in review -- https://review.opendev.org/c/openstack/swift/+/821736
21:06:57 <timburke> we've started running with it in prod, though, and are definitely seeing some processing from the deferred queue; it seems like a good and useful idea
21:07:17 <mattoliver> Oh cool.
21:07:37 <timburke> anyone want to commit to reviewing it for next week? i know it's on my radar
21:08:14 <mattoliver> I was somewhat involved, but happy to review it again.
21:09:00 <acoles> I had a thought that it might benefit from more commentary on the design of the deferral queue...let me know if you feel it does
21:09:08 <acoles> (for the sake of future us)
21:09:21 <timburke> thanks
21:09:54 <timburke> #topic memcache periodic refresh
21:10:53 <timburke> acoles spotted a bug in how i did the skip percentage on a previous patchset of https://review.opendev.org/c/openstack/swift/+/736802
21:12:10 <timburke> which gave me an opportunity to look at the stats i was emitting with fresh eyes -- the tl;dr is that this no longer requires any changes to the memcache client, and it's on callers to decide whether to skip or not before calling memcache
21:13:16 <mattoliver> Oh, yeah I guess that makes sense. I thought thr doing in the client had the benefit of happening in one place.
21:13:17 <timburke> which also means that the caller can spit out more-detailed stats; having memcache hit/miss/skip counts wasn't near as useful without knowing what type of key we were looking for
21:13:51 <opendevreview> Ade Lee proposed openstack/swift master: Add FIPS CI jobs  https://review.opendev.org/c/openstack/swift/+/796057
21:14:14 <timburke> there may be more revisions, but i think that patch is getting close
21:14:32 <mattoliver> Cool, I'll take a look
21:15:46 <timburke> next in the chain was removing pipeline_property -- https://review.opendev.org/c/openstack/swift/+/821920 -- there was a fair bit of back-and-forth, but i think it's settling down; it has one +2 on it now
21:15:46 <acoles> @mattoliver for me, having the more-detailed stats won over having the skip pct applied in just one place
21:17:02 <mattoliver> Kk
21:17:04 <timburke> the get info calls have also been reworked to have more-detailed stats (as well as reduce test churn)
21:18:38 <timburke> it still hasn't seen many eyes yet, but i'd prefer we review & land the first two
21:19:00 <timburke> #topic tempurl signatures in logs
21:19:19 <timburke> we've still got the open patch at https://review.opendev.org/c/openstack/swift/+/822585
21:20:16 <timburke> acoles, still think you might find time to take a look?
21:20:31 <acoles> oh, sorry, yes I will try
21:20:52 <timburke> thanks!
21:21:04 <timburke> we'll close that bug one of these days ;-)
21:21:08 * acoles upgrades the review to bold on todo list ;)
21:22:39 <timburke> looks like clayg isn't around again, so i guess i'll skip the mpu-expiration overview again
21:22:51 <timburke> #topic request tracing
21:22:52 <zaitcev> What's an MPU?
21:23:14 <timburke> multipart-upload -- how S3 does large (>5GB) objects
21:23:20 <timburke> #undo
21:23:20 <opendevmeet> Removing item from minutes: #topic request tracing
21:24:32 <timburke> because of how s3api puts segments in this mostly-client-inaccessible namespace, we're feeling pretty comfortable with the idea that expiring an MPU manifest ought to automagically clean up the segments
21:25:53 <timburke> clayg (and acoles) have been thinking about how to do that (as well as including a backstop daemon that goes around looking for orphaned segments and deleting them)
21:26:07 <timburke> #topic request tracing
21:26:13 <timburke> mattoliver gave his talk!
21:26:16 <timburke> #link http://youtu.be/iP_qy8KKDno
21:26:34 <mattoliver> \o/ talk complete
21:26:37 <timburke> i haven't watched it yet, but plan to soon
21:26:44 <kota> excellent
21:27:06 <seongsoocho> wow good!  thanks mattoliver
21:27:45 <mattoliver> I've been off looking after kids and that conference, so will be back at work today playing catchup.
21:28:03 <timburke> 👍
21:28:08 <timburke> that's all i've got
21:28:12 <timburke> #topic open discussion
21:28:24 <timburke> anything else we ought to bring up this week?
21:28:54 <mattoliver> I got nothing.. I'm still in holiday mode :)
21:30:29 <acoles> I've been working on a fix for some strange stasd naming mutation that we spotted https://review.opendev.org/c/openstack/swift/+/824608
21:30:42 <acoles> *statsd
21:31:37 <acoles> it does result in some metric names emitted by the proxy changing, but only because they were unpredictable before
21:32:44 <acoles> in summary, the proxy would prefix a metric name with [account|container|object] but the prefix mutated if a subrequest was issued, so the prefix was unpredictable.
21:33:25 <mattoliver> Oh good find
21:33:32 <acoles> I plan to write a bug report, now the patch is done ;-)
21:34:03 <timburke> things like needing to fetch shard ranges to find out which shard an object update should go to would trip it, yeah?
21:34:03 <mattoliver> Lol
21:34:28 <acoles> we spotted this when studying the outputs of timburke's memcache skipping stats
21:34:57 <timburke> do you remember if it would be that the container request would get stats that made it look like an object request, or vice-versa?
21:35:07 <acoles> timburke: yes, or needing to do a container HEAD to populate container_info during an object request
21:35:44 <acoles> I think object requests mutated to container, or account
21:36:41 <acoles> sometime the prefix was lost altogether, which I haven't reproduced but could be when there is an error handling a (sub)request before the proxy controller is instantiated
21:37:15 <timburke> 🤮
21:38:11 <timburke> i pushed up a new patchset for staticweb+tempurls: https://review.opendev.org/c/openstack/swift/+/810754
21:38:27 <acoles> the prefix would change such as: proxy-server -> proxy-server.object -> proxy-server -> proxy-server.container for example
21:38:41 <timburke> review it this month and you'll get to browse through pictures from a trip i took back in college ;-)
21:39:59 <mattoliver> Lol timburke
21:40:52 <timburke> all right, i'm'a call it
21:41:05 <timburke> thank you all for coming, and thank you for working on swift!
21:41:08 <timburke> #endmeeting