21:00:04 <timburke> #startmeeting swift 21:00:04 <opendevmeet> Meeting started Wed Mar 6 21:00:04 2024 UTC and is due to finish in 60 minutes. The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:04 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:04 <opendevmeet> The meeting name has been set to 'swift' 21:00:11 <timburke> who's here for the swift meeting? 21:00:23 <kota> hi 21:01:23 <mattoliver> o/ 21:01:43 <timburke> as usual, the agenda's at 21:01:44 <timburke> #link https://wiki.openstack.org/wiki/Meetings/Swift 21:02:04 <timburke> though i forgot to add the first thing i ought to talk about ;-) 21:02:10 <timburke> #topic swift release 21:02:35 <timburke> it's time to put together a release for caracal! 21:02:36 <mattoliver> lol 21:03:14 <timburke> they need cycle highlights this week, and i can never come up with them unless i'm doing a release 21:03:46 <timburke> i think we might technically have until next week to make the release, but sooner tends to be better 21:03:53 <mattoliver> I assume we can use the priority reviews page for patches we want to target to get into the release? 21:04:01 <acoles> o/ sorry I'm late 21:04:04 <timburke> absolutely! 21:04:09 <jianjian> o/ me too 21:04:20 <mattoliver> but, kk. Try and put my reviewer had on some more this week. 21:04:21 <timburke> #link https://wiki.openstack.org/wiki/Swift/PriorityReviews 21:04:38 <timburke> i need to update it; at least a few of those have merged :D 21:04:52 <mattoliver> but thats a good problem to have :) 21:06:13 <timburke> if anyone has other patches to add, please let me know! i should probably kick some of the ones currently listed off until after the release, as they're unlikely to make it (aws-chunked, i'm looking at you) 21:07:00 <jianjian> from patch list, looks like PriorityReviews means patches are almost done and need landing upstream soon 21:07:16 <mattoliver> maybe we just add a sub heading for non release priority reviews so they don't fall off our radar. 21:08:07 <mattoliver> jianjian: yeah, there are so many patches out there. Priority reviews is a way to tell reviewers which are a higher priority for reasons. so when you don't know what to review its a good place to go. 21:08:09 <timburke> i'll often stuff them into an HTML comment, just to avoid distracting from what matters *this week* ;-) 21:08:30 <mattoliver> yeah, thats ok too :) 21:08:47 <timburke> next up (and somewhat related) 21:09:01 <timburke> #topic expirer grace period 21:09:11 <timburke> #link https://review.opendev.org/c/openstack/swift/+/874806 21:09:11 <patch-bot> patch 874806 - swift - expirer: per account and container grace period - 12 patch sets 21:09:17 <timburke> #link https://review.opendev.org/c/openstack/swift/+/874710 21:09:17 <patch-bot> patch 874710 - swift - add x-open-expired to access expired objects - 30 patch sets 21:09:36 <timburke> the follow-ups have been squashed in so we've got just these two patches 21:09:53 <mattoliver> oh cool! I was about to ask if that had happened! 21:10:04 <timburke> there's a pep8 issue on the second, but otherwise i think they're ready for reviews 21:10:26 <mattoliver> kk! 21:10:54 <acoles> oh noce, Anish has stacked them in a chain too 21:10:57 <timburke> biggest question on my mind (with the release on the horizon) is whether we want to wait for both to be ready before merging the first one 21:10:59 <acoles> nice* 21:12:41 <mattoliver> I guess their independent.. but would it be annoying to have a release between the 2? possibly 21:12:51 <acoles> I think it makes sense, grace period is not much use without being able to access the expired object 21:13:03 <mattoliver> +1 21:13:12 <timburke> it may also be somewhat moot -- this week might not be quite enough time 21:13:31 <timburke> but i think i *will* add it to the priority reviews page 21:13:33 <jianjian> makes sense, +1 21:13:46 <timburke> next up 21:13:53 <timburke> #topic aws-chunked 21:13:55 <indianwhocodes> o/ 21:14:01 <mattoliver> well lets get some +2/+1s on them, but only land the first when the other is ready.. ie both or nothing? 21:14:17 <timburke> 👍 21:14:25 <zaitcev> "Access expired objects"? Really, guys. Just how far deep the rabbit hole leads? 21:14:41 <mattoliver> zaitcev: lol, don't ask :P 21:15:42 <timburke> i've got the broad strokes of the aws-chunked chain laid out 21:15:49 <timburke> #link https://review.opendev.org/c/openstack/swift/+/909049 21:15:50 <patch-bot> patch 909049 - swift - s3api: Improve checksum-mismatch detection - 4 patch sets 21:15:56 <timburke> #link https://review.opendev.org/c/openstack/swift/+/909800 21:15:57 <patch-bot> patch 909800 - swift - utils: Add crc32c function - 4 patch sets 21:16:03 <timburke> #link https://review.opendev.org/c/openstack/swift/+/909801 21:16:03 <patch-bot> patch 909801 - swift - s3api: Add support for additional checksums - 5 patch sets 21:16:09 <timburke> #link https://review.opendev.org/c/openstack/swift/+/909802 21:16:09 <patch-bot> patch 909802 - swift - WIP: s3api: Additional checksums for MPUs - 5 patch sets 21:16:15 <timburke> #link https://review.opendev.org/c/openstack/swift/+/836755 21:16:16 <patch-bot> patch 836755 - swift - Add support of Sigv4-streaming - 14 patch sets 21:16:36 <mattoliver> oh wow, becoming quite a chain! 21:16:49 <mattoliver> oh and you have a crc32c one now. Nice 21:17:27 <indianwhocodes> that sure is a lot! 21:17:27 <zaitcev> timburke: re.909049, you see that BaseException.args is absent for me. I added the console log. What version were you at? 21:17:44 <zaitcev> timburke: I mean the version of python. Mine was 3.12. 21:17:58 <timburke> those middle three might all be able to get squashed into one, but i think they're already reasonably large 21:18:24 <mattoliver> oh maybe another 3.12 issue maybe? 21:18:34 <indianwhocodes> ^^ 21:18:35 <zaitcev> I suppose I can +2 it as it is and wait until it blows up on py312, and feel smug? 21:18:48 <mattoliver> lol 21:19:06 <timburke> zaitcev, .args, or .arg? the snippet you had at https://review.opendev.org/c/openstack/swift/+/909049/4/swift/common/middleware/s3api/s3request.py#873 had .arg, which would be missing 21:19:06 <patch-bot> patch 909049 - swift - s3api: Improve checksum-mismatch detection - 4 patch sets 21:19:26 <zaitcev> Just use that str() dammit, what are we even discussing. I'm not inventing a new algorithm for the sharder here. 21:19:52 <zaitcev> Oh 21:19:54 <zaitcev> Hmm. 21:20:31 <timburke> but yeah, str() could work fine. i mainly was thinking i'd be explicit about how/where it came from 21:20:35 <jianjian> on patch "909800: utils: Add crc32c function", is this where the fast isa-l library will be used potentially? 21:20:45 <zaitcev> No, no, I was clearly wrong. 21:22:06 <timburke> jianjian, yup -- isa-l is preferred; if not available, fall back to kernel sockets on py3, or a simple pure-python implementation on py2 21:23:12 <timburke> my current stumbling block is some seemingly unrelated probe tests failing starting at p 909801 -- i haven't been able to reproduce them locally, and they don't seem to make much sense to me 21:23:12 <patch-bot> https://review.opendev.org/c/openstack/swift/+/909801 - swift - s3api: Add support for additional checksums - 5 patch sets 21:23:20 <jianjian> 👍 21:24:11 <timburke> they're all about signals and reload handling, but the patch is only touching things way over in s3api 21:25:03 <timburke> i'll keep on it, but if anyone has spare cycles to look at it, maybe a fresh set of eyes will see it more quickly 21:25:23 <timburke> #topic drive-full checker 21:25:36 <mattoliver> yeah that seems weird, I'll see if it fails for me 21:26:00 <timburke> i wonder if maybe i need to try under centos 8... 21:26:54 <mattoliver> oh thats an idea. I do have the centos PR for vsaio floating around. 21:27:11 <timburke> i've been reviewing zigo's patch! it's looking fairly good. i probably should have looked at how we (NVIDIA) deal with shutting down rsync earlier, tho :P 21:27:13 <mattoliver> let me try and recreate 21:27:48 <mattoliver> I was hoping I'd trick^Wconvince our SRE to look into it 21:28:19 <timburke> oh, boo! i can't even ping csmart directly here ;-) 21:28:44 <mattoliver> csmart: I can ;) 21:29:12 <csmart> 👋 21:29:13 <mattoliver> wow it worked! 21:29:19 <mattoliver> 😆 21:29:26 <timburke> ✨magic✨ 21:29:44 <timburke> if anyone else wants to take a look, i'm sure zigo would appreciate it 21:29:47 <jianjian> csmart, welcome! :-) 21:29:59 <mattoliver> csmart: I was just says I'm trying to trick SRE into looking at the drive-full patch 21:30:17 <timburke> #link https://review.opendev.org/c/openstack/swift/+/907523 21:30:18 <patch-bot> patch 907523 - swift - drive-full-checker - 32 patch sets 21:30:24 <csmart> Sounds good to me! 21:30:38 <timburke> thanks! 21:30:55 <mattoliver> nice! 21:30:58 <acoles> csmart: welcome to the party 21:31:07 <timburke> zaitcev, out of curiosity, how do you guys deal with shutting down rsync when drives start to get full? 21:31:33 <timburke> might be good to have more than just the two perspectives on how to deal with it 21:31:44 <mattoliver> +1 21:32:11 <zaitcev> I don't think we deal with it at all. The field people know that Swift cannot survive a full filesystem. 21:33:03 <jianjian> btw, how do your field people define a full FS? what percentage of used? 21:33:16 <timburke> interesting. might be all the more reason to be interested -- there'd be a new cron job you could run to try to defend against it 21:33:42 <timburke> the good news is, i trust that csmart wouldn't hold back in criticizing NVIDIA's approach if he disagrees ;-) 21:33:58 <zaitcev> In addition, our 2nd day story is fairly weak until OSP 19 or so. It's an upcoming release based on Caracal IIRC, and it uses k8s. Previously, there was no built-in way to monitor or resolve filesystems creeping to full. Just Zabbix and that's it. 21:34:35 <zaitcev> jianjian: 100% is full. When new files are all created with 0 size. 21:35:56 <zaitcev> In legacy Linux we had a root-only reserve, but I suspect rsync defeats it somehow. I see cases where filesystems are totally full. 21:36:18 <mattoliver> sounds like having this checking could be helping redhat too then, thats a win 21:38:11 <timburke> all right, next up 21:38:20 <timburke> #topic part-number support 21:38:23 <timburke> #link https://review.opendev.org/c/openstack/swift/+/894570 21:38:23 <patch-bot> patch 894570 - swift - slo: part-number=N query parameter support - 90 patch sets 21:38:33 <timburke> #link https://review.opendev.org/c/openstack/swift/+/894580 21:38:34 <patch-bot> patch 894580 - swift - s3api: Support GET/HEAD request with ?partNumber - 99 patch sets 21:39:11 <mattoliver> woah 99, just one more indianwhocodes :P 21:39:30 <timburke> indianwhocodes, how's it going? anything we should watch out for as we review? 21:39:44 <jianjian> zaitcev: wow, you are dealing with 100% full, good to know 21:39:52 <mattoliver> yeah, ready for a review. Might be nice to get into the release if its ready 21:39:57 <jianjian> has to be 99 lol 21:40:10 <timburke> mattoliver, +1 21:41:05 <timburke> and that set, i feel pretty good about releasing just the first if that's how far we get 21:43:40 <timburke> all right, one last topic i had 21:44:03 <timburke> #topic drop support for libec<1.4 21:44:06 <timburke> #link https://review.opendev.org/c/openstack/pyeclib/+/839643 21:44:07 <patch-bot> patch 839643 - pyeclib - Drop support for liberasurecode<1.4.0 - 10 patch sets 21:45:09 <timburke> it's been more than seven years since 1.4.0 came out, i think we can feel pretty comfortable doing it 21:45:53 <mattoliver> yeah, sounds good to me. 21:45:56 <timburke> just wanted to give everyone one last chance to object 21:46:21 <timburke> i might still wait another month or so before merging, just to avoid it looking like it was supposed to go out in caracal 21:46:53 <zaitcev> I have no objection. We used to ship with 1.0.9, but that was a very long time ago. 21:47:08 <zaitcev> Before the zlib CRC problme, IIRC 21:47:42 <timburke> all right, sounds good. i'll self-approve if needed ;-) 21:47:47 <timburke> #topic open discussion 21:47:54 <timburke> anything else we ought to bring up this week? 21:48:21 <kota> Just FYI, I'll be at SanJose at GTC week. 21:48:22 <mattoliver> I have nothing 21:49:01 <timburke> kota, hurrah! want to try to meet up? 21:49:07 <acoles> timburke: any progress towards a feature branch for mpu? I have code I want to push somewhere 21:49:33 <acoles> nobody ever comes to the UK :'( 21:49:57 <timburke> acoles, thanks for the nudge! i hadn't done anything with it yet; if you've got code already, i'll try to do that in the next day or two 21:49:58 <kota> timburke: maybe, will check free time slot in the week evening. it might be fully busy week. 21:50:19 <acoles> timburke: thanks, let me know if I can do anything 21:51:45 <indianwhocodes> @timburke sorry lost connection there 21:51:54 <indianwhocodes> yes part-num patchg is ready to be merged upstream 21:53:55 <timburke> excellent -- i'll try to take a look this week 21:54:34 <indianwhocodes> kota will be nice to meet you in-person 21:55:16 <kota> indianwhocodes: oh, you'll be at there. 21:55:28 <indianwhocodes> yes. 21:56:05 <mattoliver> jianjian: is in the area too 21:56:05 <kota> good to know, let's plan to meet in person 21:56:33 <mattoliver> Wish acoles and I could go to GTC now :( 21:56:50 <timburke> i can check in with notmyname, too 21:57:12 <kota> awesome 21:58:03 <timburke> all right, we're about at time 21:58:13 <timburke> thank you all for coming, and thank you for working on swift! 21:58:18 <timburke> #endmeeting