21:00:11 <timburke> #startmeeting swift
21:00:11 <opendevmeet> Meeting started Wed Feb  9 21:00:11 2022 UTC and is due to finish in 60 minutes.  The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:11 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:11 <opendevmeet> The meeting name has been set to 'swift'
21:00:18 <timburke> who's here for the swift meeting?
21:00:31 <kota> o/
21:02:52 <acoles> o/
21:03:18 <timburke> as usual, the agenda's at https://wiki.openstack.org/wiki/Meetings/Swift
21:03:26 <timburke> #topic PTG
21:03:39 <timburke> first, a reminder to register
21:03:47 <timburke> #link https://openinfra-ptg.eventbrite.com/
21:03:59 <timburke> the team sign-up instructions are also out
21:04:07 <timburke> #link http://lists.openstack.org/pipermail/openstack-discuss/2022-February/027051.html
21:04:30 <timburke> i'll put together a doodle poll by next week at the latest to help us select times
21:04:38 <mattoliver> Sorry im here too, just getting distracted by parenting duties.
21:04:44 <mattoliver> Kk
21:05:43 <timburke> the last couple times, i had a time shift part-way through the week -- has that worked out OK for everyone, or would it be better to try to keep it consistent all week? or no real preference?
21:06:44 <mattoliver> I guess I don't mind, however can't guarantee brain power at 2am  😜
21:06:55 <mattoliver> So it worked well for me.
21:07:20 <acoles> split times seems fairer
21:08:32 <timburke> 👍
21:08:54 <timburke> #topic summit
21:09:06 <timburke> it's fast approaching!
21:09:26 <timburke> i hadn't realized, but the call for presentations is closing soon!
21:09:36 <timburke> #link http://lists.openstack.org/pipermail/openstack-discuss/2022-February/027129.html
21:09:54 <timburke> #link https://cfp.openinfra.dev/app/berlin-2022
21:10:44 <timburke> if you've got ideas for swift talks, please get them submitted asap :-)
21:11:11 <timburke> #topic election season
21:11:33 <timburke> nominations are now open!
21:11:38 <timburke> #link http://lists.openstack.org/pipermail/openstack-discuss/2022-February/027117.html
21:12:12 <timburke> i'm happy to run for swift ptl again -- though i need to write up my statement :-)
21:12:56 <mattoliver> +100 for swift ptl! So long as you ok with it :)
21:13:05 <mattoliver> Tim for swift ptl
21:13:11 <timburke> all right, i think that covers the general announcements...
21:13:18 <mattoliver> Phone lost the Tim in thr first one
21:13:25 <timburke> :-)
21:13:27 <timburke> #topic 2.29.0 release
21:14:05 <timburke> if you haven't already, i'd encourage everyone to read over the proposed change log
21:14:10 <timburke> #link https://review.opendev.org/c/openstack/swift/+/826947
21:14:19 <timburke> there's a lot of great stuff in there!
21:14:28 <acoles> timburke: run run run for ptl!!
21:15:56 <timburke> there's still one more patch i'd like to land, and a couple more release notes, but i think i should be ready to tag later this week or early next week
21:17:04 <timburke> speaking of the patch i'd like to land...
21:17:10 <mattoliver> nice
21:17:13 <timburke> #topic signatures in logs
21:17:36 <timburke> the new registry module has landed
21:17:38 <timburke> #link https://review.opendev.org/c/openstack/swift/+/825694
21:17:46 <mattoliver> oh cool!
21:18:08 <timburke> and acoles has a +2 on the actual fix
21:18:14 <timburke> #link https://review.opendev.org/c/openstack/swift/+/822585
21:19:31 <kota> registry, cool
21:20:11 <mattoliver> Thanks for cleaning them up and pushing them across the line (or almost across for the last one).
21:20:32 <timburke> acoles, i think you still had a few comments in test_proxy_logging -- is it safe to assume those could be addressed as a follow-up?
21:21:35 <mattoliver> I like how in fixing the CVE we actually add a usful feature inside swift for devs.
21:21:51 <kota> +1
21:22:08 <acoles> wait, I think I fixed those comments today but maybe I didn't push to gerrit :O
21:23:26 <timburke> then i'll hold off on approving *right now* ;-)
21:24:04 <timburke> fwiw, i was eyeing the "_sensitive_headers and _sensitive_params are not cleared at end of this test" comment in particular, though i think there was another case-sensitivity comment that looked un-addressed
21:25:19 <timburke> hmm... i keep leaving "mpu expiration" on the agenda hoping that clayg will be around to give a quick run-down
21:25:47 <mattoliver> its we can just skip it until he's here ;)
21:27:23 <timburke> i had this memory that there were a bunch of patches, and it was tricky to see which arrow had the wood behind it... but maybe a bunch of them got abandoned?
21:27:40 <timburke> i *think* this is the primary patch:
21:27:43 <timburke> #link https://review.opendev.org/c/openstack/swift/+/800701
21:28:56 <timburke> at any rate, that's the one we're trying out in prod :P
21:29:21 <timburke> well, anyway
21:29:27 <timburke> #topic open discussion
21:29:35 <timburke> what else should we bring up this week?
21:29:42 <zaitcev> I'm bored with vPTG.
21:29:58 <timburke> me too. not the same :(
21:30:18 <mattoliver> who knows, could be the last virtual one
21:30:51 <timburke> 🤞
21:31:10 <mattoliver> I've been playing with encoding the device into the container db ID. As this ID is recreated after any rsyc or rsync_then_move.
21:31:13 <mattoliver> https://review.opendev.org/c/openstack/swift/+/828069
21:31:25 <mattoliver> Or rather adding it not really encoding it.
21:32:15 <mattoliver> In an attempt to make it easier to debug replication issues because you can then go look at incoming/outgoing syncs and know where each id came from (at least at a particular timestamp).
21:32:25 <timburke> oh yeah, i'd been meaning to take a look at that...
21:33:08 <mattoliver> The db id is just text so shouldn't hurt. And the device is pulled from the broker path.
21:33:50 <mattoliver> So long as you have unique device names across the cluster, this will allow you to do a lookup in your ring to know what node is syncing with a container.
21:35:03 <mattoliver> This kinda came up when I was attempting to figure out which handoff was breaking / resetting the root epoch. But got the incoming_sync table and was a deadend.
21:35:39 <timburke> do we make an explicit recommendation to have unique device names across the cluster? i know it's certainly the way *i'd* recommend, but i'm not sure if we've got that written down anywhere...
21:36:11 <mattoliver> no we don't. But I could add some text to the doc to recommend it.
21:36:45 <timburke> mattoliver, speaking of, have we seen that root epoch thing come up any more recently? if memory serves, there was some patch to at least detect the situation better...
21:36:45 <acoles> timburke: re https://review.opendev.org/c/openstack/swift/+/822585, it's all good, I got confused by the gerrit comments, I think I did address my own concerns in patchset 12
21:37:01 <timburke> 👍
21:37:15 <mattoliver> I'd love to add something more, like device ID, but I can grab the device from the path for free as it all happens inside the broker classes. And anything else might involve having access to the ring.
21:38:49 <timburke> *cough, cough* https://review.opendev.org/c/openstack/swift/+/670674
21:38:51 <timburke> ;-)
21:38:53 <mattoliver> On another note, I have a "new" version of serialiaztion V2 based on the discussions from the last vPTG: https://review.opendev.org/c/openstack/swift/+/827276
21:39:07 <timburke> i ought to rebase that and try it out again
21:39:21 <mattoliver> oh yeah.
21:39:30 <mattoliver> that would change things :)
21:40:22 <mattoliver> on the ring V2 ptg front, managed to get it to know behave in both Py2 and Py3
21:40:47 <timburke> \o/
21:40:58 <mattoliver> py2 has seeking backward issues. So turns out I just always seek forwards and now V2 will work for both versions.
21:41:34 <timburke> thanks for pushing on that. i keep meaning to look at it again
21:41:52 <mattoliver> nps
21:42:03 <mattoliver> just wanted to get something up before the next PTG :P
21:43:28 <acoles> I revived an old patch of timburke 's this week https://review.opendev.org/c/openstack/swift/+/569847
21:43:48 <acoles> it's a change to the way we lookup shard ranges in container backend
21:44:03 <acoles> delegates more filtering to the sql query rather than python
21:45:18 <acoles> turns out it makes a big difference to the lookup time i.e. faster
21:45:51 <acoles> especially when there are larger numbers of shard ranges in the table (like, 1000's)
21:46:12 <mattoliver> brilliant!
21:47:05 <acoles> I benchmarked it with a copy of one of our production db's and it's 20x faster!
21:47:34 <mattoliver> \o/
21:47:39 <mattoliver> doing it down in SQL land would be far more efficient. Nice. I wonder if there are other areas we could push things down to SQL land.
21:49:16 <timburke> theeeee change itself is pretty targetted... i wonder if it's the sort of thing where it's worth running an experiment in prod...
21:50:17 <timburke> all right
21:50:34 <timburke> thank you all for coming, and thank you for working on swift!
21:50:37 <timburke> #endmeeting