Wednesday, 2022-07-27

opendevreviewAlistair Coles proposed openstack/swift master: sharder: emit misplaced stats to statsd  https://review.opendev.org/c/openstack/swift/+/85090410:04
opendevreviewAlistair Coles proposed openstack/swift master: sharder: emit stats for shard DBs created or existing  https://review.opendev.org/c/openstack/swift/+/85090510:04
opendevreviewMatthew Oliver proposed openstack/swift master: Sharder: Fall back to local device in get_shard_broker  https://review.opendev.org/c/openstack/swift/+/85059711:28
opendevreviewAlistair Coles proposed openstack/swift master: sharder/replicator: emit stats for DBs created or existing  https://review.opendev.org/c/openstack/swift/+/85090513:03
opendevreviewAlistair Coles proposed openstack/swift master: sharder: process deleted DBs  https://review.opendev.org/c/openstack/swift/+/84954814:51
opendevreviewAlistair Coles proposed openstack/swift master: sharder/replicator: emit stats for DBs created or existing  https://review.opendev.org/c/openstack/swift/+/85090516:00
opendevreviewAlistair Coles proposed openstack/swift master: Sharder: Fall back to local device in get_shard_broker  https://review.opendev.org/c/openstack/swift/+/85059716:04
acolesrebase ^^16:05
kotagood morning20:56
timburke_kota, o/20:58
*** timburke_ is now known as timburke20:58
kotatimburke_: o/20:58
timburke#startmeeting swift21:00
opendevmeetMeeting started Wed Jul 27 21:00:06 2022 UTC and is due to finish in 60 minutes.  The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot.21:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.21:00
opendevmeetThe meeting name has been set to 'swift'21:00
timburkewho's here for the swift meeting?21:00
zaitcevo21:00
zaitcevo721:00
kotahi21:00
timburkeas usual, the agenda's at https://wiki.openstack.org/wiki/Meetings/Swift21:01
timburkefirst up21:01
timburke#topic sha1 deprecations21:01
timburkethe server patches have merged!21:01
timburke#link https://review.opendev.org/c/openstack/swift/+/84586221:01
timburketo walk back the deprecation a bit in tempurl21:02
timburke#link https://review.opendev.org/c/openstack/swift/+/83371321:02
mattolivero/ (sorry im late)21:02
timburketo do a similar sort of deprecation for formpost21:02
acoleso/21:02
timburkei started on a similar patch for /info calls21:03
timburke#link https://review.opendev.org/c/openstack/swift/+/85078721:03
timburkebut it needs some more work -- maybe i'll get to it for next week21:03
timburkethere's still one open question for me around the sha1 deprecation21:04
timburkeand that's what should happen with the client backport at https://review.opendev.org/c/openstack/python-swiftclient/+/84739821:05
timburkenow that we've walked back the deprecation process such that sha1 is still allowed by default, i'm not sure it's still necessary. i'll follow up on it21:06
timburkeanybody have any questions or comments about the patches?21:07
zaitcevnot me21:07
mattoliverI'll revisit the client one too21:07
timburkethanks -- i'm not completely sure what was going on with the gate -- there's a decent chance that our non-master jobs are currently busted :-/21:09
timburkeall right. let's keep the merge-train rolling21:09
timburke#topic stuck sharding DBs21:09
timburkenothing new has actually merged yet here, but the outstanding patches are progressing well21:10
acoleswe unstuck some dbs!21:10
timburke#link https://review.opendev.org/c/openstack/swift/+/84954821:10
timburkefor processing deleted DBs21:11
timburke#link https://review.opendev.org/c/openstack/swift/+/85059721:11
timburkefor trying harder to find a device to use as a handoff21:11
acolesthat one is new since last week's meeting^^^21:11
acolesthe db's we're trying to get sharded are on nodes that are retiring and turns out they had no devices with non-zero weight in the ring :( so the sharder couldn't find a device on which to cleave shard dbs21:12
acolesthe solution is to just *pick a device* for the temporary shard dbs, even if zero weight21:13
mattoliverYeah so progress could be made21:14
timburkeyou can run into similar issues with the reconciler -- but since it would happen during the _post_replicate_hook(), we'd log the error then go ahead and delete the handoff (assuming everything sync'ed)21:14
mattoliverThe latest patchset has the fallback moved to find_local_handoff_for_part so they both should get the fallback21:15
acoles☝️21:16
timburke👍 i'll take another look this week21:16
timburke#topic SLO resource leak21:16
mattoliverThanks21:16
timburkeOVH spotted a bug recently: https://bugs.launchpad.net/swift/+bug/198095421:17
timburkeand even provided a fix21:17
timburke#link https://review.opendev.org/c/openstack/swift/+/85035721:17
timburkei had a thought for how we might make it a little less resource-intensive21:18
timburke#link https://review.opendev.org/c/openstack/swift/+/85078221:18
timburkesince the original patch would drain the response iter (which could amount to several gigs) before closing21:19
timburkethanks for helping review them both, zaitcev!21:19
zaitcevnp21:19
zaitcevBut I think I bikeshedded something there21:19
zaitcevOh, I remember now, n/m21:19
timburkei feel i should point out that there is an upgrade concern for my follow-up, though: if you've still got pre-2.24.0 object nodes, the slo-delete call could 50021:20
zaitcevSorry, I'm not seeing the scenario.21:22
zaitcevThe first patch just adds the drain. Seems like no impact.21:22
zaitcevThe follow-up has 2 consistent actions that do not happen one without the other... And it's fully implemented on the proxy.21:23
timburkeit's a concern in the follow-up. with it, if you've got an object-server that doesn't have https://github.com/openstack/swift/commit/e8b654f3 then it won't know about X-Backend-Ignore-Range-If-Metadata-Present, so it'll respect the Range that was sent and the json.loads() will fail21:24
zaitcevoh... that old21:25
timburkei could probably add some special casing to have it fall back to old behavior if it sees an SLO respond with just one byte... or we could just allow the 500, and let the client retry21:26
timburkeyeah, it puts it at Feb 2020 -- part of why i wanted to flag it up and get people's opinions21:26
timburkewell, we can sort it out on the review. maybe i'll at least try writing the fallback, see how bad it gets21:29
timburke#topic ring v221:30
timburkemattoliver has a +2 on21:30
timburke#link https://review.opendev.org/c/openstack/swift/+/83426121:30
mattoliveryeah I do21:30
timburkeanyone else want to weigh in before we go ahead and merge?21:30
mattolivernows the chance to have a say on the new ring on disk format. 21:31
timburkewell, now, and the last two ptgs, and... ;-)21:32
timburkemaybe i'll see if i can get clayg to take a look later by threatening him to have to support a new file format he hasn't thoroughly looked at :P21:33
mattoliverlol21:34
timburkeall right. one last thing i forgot to add to the agenda21:34
acolesI'll try to cast an eye over it...but I can't promise21:34
timburkethanks acoles21:34
timburke#topic PTG in ohio21:34
timburkeunfortunately, NVIDIA recently sent out an email saying "We are suspending business travel in order to prioritize our investments, effective immediately. Travel for internal meetings, conferences, and events should be cancelled. Use virtual meetings instead."21:34
timburkeso... i don't think acoles, clayg, mattoliver, or i are likely able to go :-(21:35
kotaoh21:36
timburkebut i'd be happy to help with planning if other people are going to be able to get together there21:36
timburketo make sure you'd have a room, etc.21:36
kotacould we plan another virtual setting of ptg for a specific project?21:36
zaitcevI see the final touch-up for dark data watcher is slated for PTG, so is this okay to poke acoles now? Since he's not coming anyway.21:38
zaitcevWell the ring v2 is clearly more important but I hope this is simpler21:38
zaitcevhttps://review.opendev.org/c/openstack/swift/+/78765621:38
timburkei see no reason we couldn't do a virtual meetup outside of the context of a PTG :-)21:38
acoles+121:38
kota+121:38
mattolivergreat idea21:39
zaitcevYou know21:39
zaitcevWhen Covid just hit, in a few months our management started a crackdown on coworking speakeasies.21:40
zaitcevApparently teams would get together in locked and abandoned offices21:40
zaitcevOr in hotels that still operated21:40
zaitcevWe should rent a cabin at Lake Tahoe.21:41
acolesreminds me of the fallback to the airbnb in Dublin during the snow :)21:41
timburkei'd be real tempted :-) biggest trouble is getting someone to pay for flights for acoles, kota, and mattoliver ;-)21:42
timburkei guess the question then would be, when would you all like to do a virtual meeting? i'm guessing we'd want something along the lines of our previous vPTGs for structure...21:44
timburkemaybe i'll put together a doodle poll to pick a week in the next couple months21:47
timburkeall right, that's all i've got21:48
mattoliverthat's a good idea. 21:48
timburke#topic open discussion21:48
timburkeanything else we should bring up this week?21:48
acolesdepeding on if anyone is going to the in person ptg, we could go virtual the same week21:48
acolesthat'd save me changing my calendar ;-)21:49
timburkenow that the sha1 patches are mostly in order, i think i'll try to get a release together21:50
timburkelet me know in the next week or so if there's any other patches you think ought to land before a release21:50
mattoliverkk21:51
timburkeok, i'm calling it21:52
timburkethank you all for coming, and thank you all for working on swift!21:52
timburke#endmeeting21:53
opendevmeetMeeting ended Wed Jul 27 21:53:03 2022 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)21:53
opendevmeetMinutes:        https://meetings.opendev.org/meetings/swift/2022/swift.2022-07-27-21.00.html21:53
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/swift/2022/swift.2022-07-27-21.00.txt21:53
opendevmeetLog:            https://meetings.opendev.org/meetings/swift/2022/swift.2022-07-27-21.00.log.html21:53

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!