21:00:34 #startmeeting swift 21:00:34 Meeting started Wed Jan 19 21:00:34 2022 UTC and is due to finish in 60 minutes. The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:34 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:34 The meeting name has been set to 'swift' 21:00:41 who's here for the swift meeting? 21:00:47 o/ 21:00:48 o/ 21:00:50 o/ hi 21:00:52 \o 21:01:51 as usual, the agenda's at https://wiki.openstack.org/wiki/Meetings/Swift 21:02:08 first up, in case you missed it on the ML 21:02:14 #topic z cycle naming 21:02:23 #link http://lists.openstack.org/pipermail/openstack-discuss/2022-January/026620.html 21:02:38 we can propose names on the wiki at 21:02:41 #link https://wiki.openstack.org/wiki/Release_Naming/Z_Proposals 21:03:38 Zorro! 21:03:55 😜 21:04:20 nominations close monday, then as i recall, the TC votes for the final name 21:04:49 on to swift-y topics! 21:04:56 #topic updater fairness 21:05:26 the first patch in the series (to add per-container ratelimits) landed -- https://review.opendev.org/c/openstack/swift/+/820962 21:06:18 the second one (to add a queue of deferred updates to chew on when the disk sweep is shorter than the configured interval) is still in review -- https://review.opendev.org/c/openstack/swift/+/821736 21:06:57 we've started running with it in prod, though, and are definitely seeing some processing from the deferred queue; it seems like a good and useful idea 21:07:17 Oh cool. 21:07:37 anyone want to commit to reviewing it for next week? i know it's on my radar 21:08:14 I was somewhat involved, but happy to review it again. 21:09:00 I had a thought that it might benefit from more commentary on the design of the deferral queue...let me know if you feel it does 21:09:08 (for the sake of future us) 21:09:21 thanks 21:09:54 #topic memcache periodic refresh 21:10:53 acoles spotted a bug in how i did the skip percentage on a previous patchset of https://review.opendev.org/c/openstack/swift/+/736802 21:12:10 which gave me an opportunity to look at the stats i was emitting with fresh eyes -- the tl;dr is that this no longer requires any changes to the memcache client, and it's on callers to decide whether to skip or not before calling memcache 21:13:16 Oh, yeah I guess that makes sense. I thought thr doing in the client had the benefit of happening in one place. 21:13:17 which also means that the caller can spit out more-detailed stats; having memcache hit/miss/skip counts wasn't near as useful without knowing what type of key we were looking for 21:13:51 Ade Lee proposed openstack/swift master: Add FIPS CI jobs https://review.opendev.org/c/openstack/swift/+/796057 21:14:14 there may be more revisions, but i think that patch is getting close 21:14:32 Cool, I'll take a look 21:15:46 next in the chain was removing pipeline_property -- https://review.opendev.org/c/openstack/swift/+/821920 -- there was a fair bit of back-and-forth, but i think it's settling down; it has one +2 on it now 21:15:46 @mattoliver for me, having the more-detailed stats won over having the skip pct applied in just one place 21:17:02 Kk 21:17:04 the get info calls have also been reworked to have more-detailed stats (as well as reduce test churn) 21:18:38 it still hasn't seen many eyes yet, but i'd prefer we review & land the first two 21:19:00 #topic tempurl signatures in logs 21:19:19 we've still got the open patch at https://review.opendev.org/c/openstack/swift/+/822585 21:20:16 acoles, still think you might find time to take a look? 21:20:31 oh, sorry, yes I will try 21:20:52 thanks! 21:21:04 we'll close that bug one of these days ;-) 21:21:08 * acoles upgrades the review to bold on todo list ;) 21:22:39 looks like clayg isn't around again, so i guess i'll skip the mpu-expiration overview again 21:22:51 #topic request tracing 21:22:52 What's an MPU? 21:23:14 multipart-upload -- how S3 does large (>5GB) objects 21:23:20 #undo 21:23:20 Removing item from minutes: #topic request tracing 21:24:32 because of how s3api puts segments in this mostly-client-inaccessible namespace, we're feeling pretty comfortable with the idea that expiring an MPU manifest ought to automagically clean up the segments 21:25:53 clayg (and acoles) have been thinking about how to do that (as well as including a backstop daemon that goes around looking for orphaned segments and deleting them) 21:26:07 #topic request tracing 21:26:13 mattoliver gave his talk! 21:26:16 #link http://youtu.be/iP_qy8KKDno 21:26:34 \o/ talk complete 21:26:37 i haven't watched it yet, but plan to soon 21:26:44 excellent 21:27:06 wow good! thanks mattoliver 21:27:45 I've been off looking after kids and that conference, so will be back at work today playing catchup. 21:28:03 👍 21:28:08 that's all i've got 21:28:12 #topic open discussion 21:28:24 anything else we ought to bring up this week? 21:28:54 I got nothing.. I'm still in holiday mode :) 21:30:29 I've been working on a fix for some strange stasd naming mutation that we spotted https://review.opendev.org/c/openstack/swift/+/824608 21:30:42 *statsd 21:31:37 it does result in some metric names emitted by the proxy changing, but only because they were unpredictable before 21:32:44 in summary, the proxy would prefix a metric name with [account|container|object] but the prefix mutated if a subrequest was issued, so the prefix was unpredictable. 21:33:25 Oh good find 21:33:32 I plan to write a bug report, now the patch is done ;-) 21:34:03 things like needing to fetch shard ranges to find out which shard an object update should go to would trip it, yeah? 21:34:03 Lol 21:34:28 we spotted this when studying the outputs of timburke's memcache skipping stats 21:34:57 do you remember if it would be that the container request would get stats that made it look like an object request, or vice-versa? 21:35:07 timburke: yes, or needing to do a container HEAD to populate container_info during an object request 21:35:44 I think object requests mutated to container, or account 21:36:41 sometime the prefix was lost altogether, which I haven't reproduced but could be when there is an error handling a (sub)request before the proxy controller is instantiated 21:37:15 🤮 21:38:11 i pushed up a new patchset for staticweb+tempurls: https://review.opendev.org/c/openstack/swift/+/810754 21:38:27 the prefix would change such as: proxy-server -> proxy-server.object -> proxy-server -> proxy-server.container for example 21:38:41 review it this month and you'll get to browse through pictures from a trip i took back in college ;-) 21:39:59 Lol timburke 21:40:52 all right, i'm'a call it 21:41:05 thank you all for coming, and thank you for working on swift! 21:41:08 #endmeeting