21:00:17 <timburke> #startmeeting swift
21:00:18 <openstack> Meeting started Wed Aug 28 21:00:17 2019 UTC and is due to finish in 60 minutes.  The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:19 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:22 <openstack> The meeting name has been set to 'swift'
21:00:35 <timburke> who's here for the swift meeting?
21:00:44 <mattoliverau> o/
21:00:58 <rledisez> hi o/
21:01:19 <tdasilva> o/
21:01:37 <kota_> hi
21:01:51 * notmyname lurks
21:02:25 * timburke will have to avoid bad-mouthing the previous PTL now
21:03:04 <timburke> all right, let's get started. agenda's at
21:03:09 <timburke> #link https://wiki.openstack.org/wiki/Meetings/Swift
21:03:17 <clayg> Heh
21:03:27 <timburke> first, a couple house-keeping items
21:03:30 <timburke> #topic elections
21:03:37 <timburke> it's election season!
21:03:41 <timburke> #link http://lists.openstack.org/pipermail/openstack-discuss/2019-August/008832.html
21:03:47 <timburke> fwiw, it looks like 4 of the current TC members will not be seeking re-election
21:03:52 <timburke> (that's out of 6 seats up for election this cycle)
21:03:58 <timburke> ...so if you've ever wanted to be on the comittee, your odds might be improved this cycle ;-)
21:04:20 <timburke> as for swift, i'd be happy and honored to continue serving as PTL
21:04:27 <timburke> i feel like there's still room for me to grow in this role :-)
21:04:33 <timburke> though if anyone would like to serve instead (or otherwise has misgivings), feel free to speak up. now or later; here, in -swift, or in private
21:04:34 <kota_> that's great news!
21:04:42 <timburke> i serve at y'all's pleasure ;-)
21:05:29 <timburke> kota_, oh i'm glad :-) i remember you and mattoliverau talking about maybe wanting to run -- and i really would hate to get in the way of that
21:06:27 <kota_> my point is if no one else wants to run. so I'm happy with you continue to run the PTL.
21:06:29 <kota_> ;-)
21:06:35 <timburke> any questions/comments/concerns? on either part of the elections
21:07:10 <mattoliverau> timburke: thanks, your doing a great job. notmyname has large shoes to fill, but your doing good. Keep it up!
21:07:17 <tdasilva> timburke: you are doing a great job, thanks for your efforts in leading the project!
21:07:29 <timburke> i'll work on writing up my candidacy email and plan on sending it out tonight or tomorrow
21:07:39 <notmyname> timburke: you're doing a great job :-)
21:08:08 <timburke> notmyname, oh how would *you* know, lurker!? :P
21:08:21 <timburke> #topic shrinking the TC
21:08:28 <timburke> #link http://lists.openstack.org/pipermail/openstack-discuss/2019-August/008791.html
21:08:34 <timburke> mostly just a heads-up, keeping people in the loop if you haven't been keeping up on the mailing list
21:08:42 <timburke> apparently the TC is going to shrink over the next year and a half or so from 13 members to 11 or 9
21:08:47 <timburke> (details still being ironed out)
21:08:55 <timburke> i don't think it'll really have much direct impact on us, just an fyi
21:09:37 <timburke> on to updates!
21:09:43 <notmyname> that's interesting
21:10:19 <timburke> good news is, if we have to argue about golang for losf, we'll have fewer people we have to convince, right? ;-)
21:10:28 <timburke> #topic py3
21:10:35 <timburke> all func tests have merged! probe tests are mostly ported and have one +2 already!
21:10:37 <timburke> \o/
21:10:44 <timburke> #link https://review.opendev.org/#/c/671333/
21:10:45 <patchbot> patch 671333 - swift - py3: (mostly) port probe tests - 3 patch sets
21:11:01 <mattoliverau> nice
21:11:06 <alecuyer> wow :)
21:11:06 <timburke> i still need to work on getting a gate job going for it
21:11:14 <timburke> it's been a long journey, thank you all for helping
21:11:19 <timburke> i was looking back at what we had under test when, and wanted to share a little
21:11:27 <timburke> our first openstack release with at least one test module getting exercised was mitaka, just testing exceptions
21:11:39 <timburke> by ocata, we'd gotten up to 4 modules... but then we stalled out and were still there for queens
21:11:48 <timburke> rocky, we were up to 30
21:11:53 <timburke> then 90 in stein
21:11:59 <timburke> now we're at all 128! that seems amazingly fast for the size and scope of the work
21:12:08 <timburke> congrats everybody, and again, thank you all
21:12:38 <timburke> (particularly zaitcev_ and mattoliverau :-)
21:13:17 <mattoliverau> Especially timburke and zaitcev_, I rode on their coat tails :)
21:13:52 <timburke> heh. now we just need to test it enough to really feel comfortable with it
21:14:14 <timburke> #topic versioning
21:14:21 <timburke> clayg, how's it going?
21:15:49 <timburke> ok, so that may have been rhetorical so i didn't feel like i was talking too much
21:16:06 <timburke> we've got concerns about symlink versioning actually doing all the things we want it to do
21:16:23 <timburke> #link https://review.opendev.org/#/c/633857/
21:16:24 <patchbot> patch 633857 - swift - symlink-backed versioned_writes - 19 patch sets
21:17:21 <timburke> remember, in particular we want to get to the point that we can pretty faithfully mimic S3's notion of versioning
21:17:43 <timburke> so i started playing around with adding *another* mode, like history mode but with a naming convention for the versions that would be more useful there
21:17:57 <timburke> #link https://review.opendev.org/#/c/678962/
21:17:57 <patchbot> patch 678962 - swift - WIP: Add another versioning mode with a new naming... - 1 patch set
21:18:47 <timburke> it needs lots of tests and i ought to start actually trying to wire in the s3api changes to see how well it works
21:19:34 <timburke> ...but i think it might work? biggest downside is that there isn't a good way to transition from using x-versions-location *or* x-history-location to the new mode :-(
21:20:29 <mattoliverau> moar symlinking!
21:20:35 <timburke> if anybody has thoughts on versioning and/or s3api, it might be worth taking a look
21:20:40 <mattoliverau> :P
21:21:02 <kota_> hmm
21:21:31 <clayg> timburke: sorry, i'm back
21:21:35 <timburke> mattoliverau, yeah, i was about to say "(or really, *any* way)" but then i remembered that i'd thought about symlinking the new-style to the existing old name :-)
21:22:45 <clayg> timburke: if we're going this route can we make the third mode always use symlinks?
21:22:59 <clayg> i'm excited about the new ?versions=v params
21:23:15 <clayg> I'd love to see if we could get that work with a copy/restore
21:23:23 <timburke> clayg, probably? it'll still need to be able to splice the listings though -- in case you turned it off then back on
21:23:33 <notmyname> clayg: what's the status of your investigation? learned enough to compare to timburke's 3rd mode?
21:23:48 <timburke> i *think* it'll mostly Just Work with COPY?
21:24:17 <timburke> oh yeah, and to fill out the context some more...
21:24:19 <timburke> #link https://review.opendev.org/#/c/673682/
21:24:20 <patchbot> patch 673682 - swift - s3api: Implement versioning status API - 2 patch sets
21:24:59 <timburke> clayg, anything else i'd forgotten to mention?
21:25:12 <clayg> notmyname: well, I think the symlink-versioned writes code is going to port over pretty well - there's some differences because s3request objects aren't exactly the same as plain swob requests but mostly I can fork lift just the bits I need
21:26:29 <clayg> the diff is going to be much different - instead of versioned_writes accrete a bunch of new conditional branches based on which mode you're using there's just going to be some new code paths that do symlink writes and other version'd operations in s3api mode
21:26:45 <clayg> it'll be using it's own sysmeta so it's compleatly orthogonal to versioned_writes
21:27:20 <clayg> it'll be interesting to see what looks like the best path forward in the short term - or long term depending on the bits we do/don't like
21:28:17 <timburke> completely orthogonal -- so it won't be respected for bimodal access, yeah?
21:28:20 <clayg> instead of reviewing a change to versoined writes and a new s3api feature for translation - it'll just be a change to s3api
21:29:04 <clayg> timburke: exactly - it's a s3api feature only available through the s3api - at least that's the experiment i'm running - i'm not advocating for anything other than we should take the time to understand what an implementation like that would look like
21:29:32 <timburke> 👍
21:29:52 <timburke> good to go into these things with our eyes as open as we can have them :-)
21:30:06 <kota_> :-)
21:30:10 <clayg> timburke: but related to that I would like to pull back on the forced use_symlinks migration for existing versioned writes containers
21:31:18 <timburke> i'm torn -- that race condition is the commit message is still very much a thing :-/
21:31:44 <clayg> and it has been for a long time
21:31:48 <timburke> *in the commit message
21:31:52 <timburke> true enough
21:33:18 <timburke> anyone else have questions, comments, or other feedback?
21:35:00 <timburke> #topic lots of small files
21:35:16 <timburke> alecuyer, i saw the RPC-ove-HTTP patch go by
21:35:18 <timburke> #link https://review.opendev.org/#/c/679022/
21:35:20 <patchbot> patch 679022 - swift (feature/losf) - WIP - Support RPC over plain HTTP (vs gRPC) - 1 patch set
21:35:24 <alecuyer> yes,
21:35:40 <alecuyer> it removes the grpc dependency,
21:36:08 <kota_> cool
21:36:10 <alecuyer> or almost : python still wants to find grpc to compile the protobuf definitions, I have to figure that one out
21:36:38 <alecuyer> and I need to fix tests related to that change.
21:37:14 <timburke> i was just noticing the "ImportError: cannot import name rpc_grpc" in logs :-)
21:37:57 <alecuyer> so I pushed that a little quick today so you could have a look, but its not finished :) It should not take too long though to clean it up and fix the tests
21:38:33 <timburke> nice! anything else we should be looking at for losf?
21:38:42 <alecuyer> (cherry picked it from our branch which is still a little different from master)
21:38:53 <alecuyer> well it's great to have your tests feedback
21:39:25 <alecuyer> that's it for me for this week
21:40:44 <timburke> #topic sharding
21:41:22 <timburke> mattoliverau, how's it going?
21:41:44 <timburke> i know there was a bit of back-and-forth on https://review.opendev.org/#/c/675820/
21:41:45 <patchbot> patch 675820 - swift - sharder: Keep cleaving on empty shard ranges - 2 patch sets
21:42:51 <mattoliverau> I've marked it as WIP as I have a go at reworking the cleave code
21:44:15 <mattoliverau> clayg and timburke made some good points about code flow and we should avoid using exception handling to do so. Esp in py3. It was a way to not touch too much of the cleaving code.
21:44:44 <mattoliverau> But will have a play and rework cleaving to better fit skipping empty shards ranges in.
21:45:15 <timburke> thanks -- and i didn't mean to be discouraging -- i know i prefer not to fiddle too much with working code, too ;-)
21:45:31 <mattoliverau> :)
21:46:07 <mattoliverau> yeah and cleaving is already a little.. interesting. But to do it properly is the right way forward :)
21:47:15 <timburke> #topic open discussion
21:47:23 <timburke> anybody have anything else to bring up?
21:49:00 <zaitcev> I was disconnecting too much and missed any replies if you testd py2-py3 migrations.
21:49:27 <timburke> zaitcev, no, not yet :-(
21:50:00 <timburke> fwiw, i re-organized the priority reviews page a bit to better capture some of the work in flight. sorry for hogging it for py3 stuff so much lately
21:50:02 <timburke> #link https://wiki.openstack.org/wiki/Swift/PriorityReviews
21:51:37 <timburke> as always, if there's anything else you'd like to see on there, feel free to add it
21:53:03 <timburke> all right, i'm calling it
21:53:13 <timburke> thank you all for coming, and thank you for working on swift!
21:53:18 <timburke> #endmeeting