21:01:22 #startmeeting swift 21:01:23 Meeting started Wed Dec 2 21:01:22 2020 UTC and is due to finish in 60 minutes. The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:01:24 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:01:27 The meeting name has been set to 'swift' 21:01:31 who's here for the swift meeting? 21:01:38 o/ 21:01:41 o/ hi! 21:01:50 o/ 21:01:52 o/ 21:02:39 o/ 21:02:48 o/ 21:03:26 o/ 21:03:30 agenda's at https://wiki.openstack.org/wiki/Meetings/Swift 21:03:54 sorry, i'm still catching up from the week off, so i haven't updated it 21:04:08 #topic audit watchers 21:04:21 mattoliverau, i saw you did some reviews! thanks! 21:05:04 yeah, well... 21:05:17 I'm looking at Matt's comments. 21:05:44 Honestly I'm tempted to ask for a follow-up patch for the biggest of them, where he wants to reorganize the tests. 21:06:06 Sorry, but I think it's great and am almost ready to +2 21:07:17 I wish we could just sic acoles on it. That man is a genius at refactoring our tests that are sometimes convoluted. 21:07:27 out of curiosity, what sort of test watchers did you make, mattoliverau? 21:07:39 Yeah, if we want that in a follow up, seeing as it's just test reorg. 21:08:21 i've been meaning to try some SLO-validator-type watchers... 21:08:43 Nothing amazing, just wrote one that logged things so I could get an idea of how it all worked, passed in some params via the calendar config. 21:08:53 Etc 21:09:16 Based it off the commit message code skel, which is why I found it wrong 21:09:32 cool. i'll still try to get a fresh review in soon, too. sorry it's taking me so long, zaitcev 21:10:15 Should do something more interesting. But didn't want to take too long. 21:10:35 I'm sure you will do the right thing. I kind of doubt myself because I want it "over", so my judgement may be off. But of course if I promise the follow-up, I'll do it. 21:12:48 timburke: Anyway, I think we can move on. 21:13:15 i agree that test re-org can and should be a follow-up, and i don't even know that having it proposed should be a blocker on landing what we've got. sounds like we all mostly know what we need to do next, yeah? 21:13:35 (sorry for the delay, plumber finally came) 21:13:49 the reality of life 21:14:11 My biggest fear is that you'll find a cool application of watchers that requires them to be stateful 21:14:11 #topic s3api, +segments, and ACLs 21:14:34 i thought we merged something that address that 21:14:57 clayg, i saw you +2'ed https://review.opendev.org/c/openstack/swift/+/763106/, any reason not to +A? 21:15:08 (beyond general gate breakage, i suppose...) 21:15:32 You're sure you're not thinking about PUT+POST where we proposed changes just to keep the state? Didn't merge though, it's sitting hibernating in that revew. 21:16:09 nope, looks like a win to me 21:17:16 zaitcev, i'm not opposed to proposing things that aren't necessarily going to land (hi, https://review.opendev.org/c/openstack/swift/+/648263 !). i do feel bad that we haven't done more with PUT+POST though, sorry :-( 21:18:14 i just mean to say that we can take on the test re-org later, and having the follow-up written should not be a blocker 21:18:28 #topic gate breakage 21:18:51 *hopefully* this will all be resolved shortly, but i figure i ought to keep everyone informed 21:19:08 I see Clay landed 763106 just now 21:19:39 yeah that one looked straightforward to me - we're trying to get it running in prod currently 21:19:47 but packaging software sucks 21:20:00 but packaging^W software sucks 21:21:13 recently, some of the integrated jobs broke because of some changes to the pip resolver -- details aren't so important (i think), but they should be fixed by https://review.opendev.org/c/openstack/devstack/+/764811 21:21:42 yay! 21:23:08 but we've also got *another* constantly-failing job with lower-constraints, where it'd try to install a py3-only version of keystoneclient on py2 21:23:34 that one seems to be fixed by https://review.opendev.org/c/openstack/swift/+/764856, though it needs a recheck 21:24:00 *hopefully* then we can finally start merging real patches again 21:25:00 like i said, ideally you all won't need to think about it too much 21:25:10 one last impromptu topic 21:25:25 #topic part power increase 21:26:21 clayg, acoles and i have been realizing we need to actually use this great feature cschwede wrote! 21:26:50 and i thought i'd see if anyone else has used it much, and if so what tips they might have about it 21:26:51 So, what's the issue? It worked in my tests. 21:27:01 no, not "much", sorry 21:27:07 it is pretty great, but I'm only used in the dev. 21:27:26 we did it in production, multiple times. it worked. we have a doc with useful things to do before/after (to check everything went fine, etc…). Can paste that somewhere 21:27:42 In our case it gets excercised because Director makes rings with just 9 bits 21:27:58 I always thought we'd use it and think of ways to make it more automatic rather then stopping replication engine.. also expend it to work with account and containers. 21:28:19 Because they sometimes need an increase too. 21:28:22 cool, thanks rledisez! and thanks for https://review.opendev.org/c/openstack/swift/+/695344 :-) 21:29:04 yeah, it's the "stopping replication engine" that has me the most worried. hard to do when you're *also* constantly expanding 21:29:43 From experience, with I/O limitation to not impact performance too much, it take about 24h/48h to do a relink 21:30:07 rledisez: good info! how did you ratelimit the relinker? 21:30:37 nice! yeah, we were thinking about adding a ratelimiting option... 21:30:50 i'm also debating about trying to enable moving 2 or 3 steps at a time -- we weren't paying enough attention and have some catching up to do :-( 21:30:58 we start one relinker per device, each is put in a cgroup with io limitation. we have a script for that. I can share it too, there is no secret sauce in it :) 21:32:01 oh yeah, and i guess i did https://review.opendev.org/c/openstack/swift/+/733296/ a while back... might want to merge that, too... 21:32:34 all right, that's all i've got 21:32:39 #topic open discussion 21:33:01 anything else we ought to discuss this week? 21:34:40 heads up on another py3 bug: https://launchpad.net/bugs/1906289 21:34:42 Launchpad bug 1906289 in OpenStack Object Storage (swift) "Uploading a large object (SLO) in foreign language characters using S3 browser results in 400 BadRequest - Error in completing multipart upload" [High,Confirmed] 21:34:57 have a fix already at https://review.opendev.org/c/openstack/swift/+/765204/ 21:35:14 i'll plan on backporting as far back as train 21:37:12 ahh more wsgi str fun :P 21:37:19 I'm still stuck with "secure" and "consistent" RBAC, basically tinkering with authorize(). 21:37:25 hi sorry I'm late 21:37:31 That code is scary, too much baggage. 21:38:52 yeah, auth is thorny and we've (too) often neglected it 21:39:55 reminds me that i ought to revisit https://review.opendev.org/#/c/620189/ and those RBAC tests we've got... 21:40:45 Yeah, the curse of RBAC and how it made Ho-san quit 21:40:50 Concerning! 21:42:26 all right, i think that's about it then 21:42:37 thank you all for coming, and thank you for working on swift! 21:42:41 #endmeeting