21:02:45 <timburke> #startmeeting swift
21:02:46 <openstack> Meeting started Wed Sep  4 21:02:45 2019 UTC and is due to finish in 60 minutes.  The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:02:47 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:02:49 <openstack> The meeting name has been set to 'swift'
21:02:52 <timburke> who's here for the swift meeting?
21:02:56 <kota_> o/
21:02:59 <zaitcev> o/
21:03:01 <mattoliverau> o/
21:03:04 <alecuyer> o/
21:03:07 <rledisez> o/
21:03:28 <timburke> full crowd :-)
21:03:31 <tdasilva> o/
21:03:41 <timburke> thanks for coming, and sorry to start a few minutes late
21:03:52 <timburke> agenda's at https://wiki.openstack.org/wiki/Meetings/Swift
21:04:03 <timburke> #topic election results
21:04:16 <timburke> #link http://lists.openstack.org/pipermail/openstack-discuss/2019-September/009072.html
21:04:44 <timburke> (just making sure people are aware of any leadership changes)
21:05:17 <timburke> the most interesting part of it (i think) is that *no* seat was contested. neither PTLs nor on the TC
21:06:10 <timburke> #topic https://etherpad.openstack.org/p/swift-ptg-shanghai
21:06:31 <timburke> #undo
21:06:32 <openstack> Removing item from minutes: #topic https://etherpad.openstack.org/p/swift-ptg-shanghai
21:06:36 <timburke> #topic Shanghai
21:07:00 <timburke> #link https://etherpad.openstack.org/p/swift-ptg-shanghai
21:07:34 <timburke> we're two months out from the summit/ptg -- i figured i ought to start prodding people to put topics on the etherpad :-)
21:08:06 <kota_> interesting, there are topics for project onboarding
21:08:49 * kota_ is removing `(planning)` text
21:09:23 <timburke> kota_, i was trying to come up with some ideas and see if i could convince new people to put their name down so i could better estimate attendance ... but so far it hasn't quite panned out
21:09:50 <kota_> i see. sounds good.
21:09:50 <timburke> we'll see how onboarding goes, and how many people show up
21:10:06 <timburke> worst case, more time for hacking, i suppose
21:10:09 <kota_> yah. it's hard to estimate.
21:10:20 <alecuyer> maybe people that will come are not on the "right" MLs?
21:11:09 <timburke> alecuyer, yeah, i had a similar thought. or if they are, that they'd be hesitant to go editing things
21:11:28 <timburke> oh yeah, clayg, am i right to think that you'll be in the "coming" category?
21:12:05 <clayg> Yup!
21:12:09 <timburke> 👍
21:12:23 <kota_> +1
21:12:24 <timburke> all right, updates!
21:12:27 <timburke> #topic py3
21:13:05 <timburke> thanks for taking a look at probe tests zaitcev and mattoliverau! we've nearly got all our tests ported :-)
21:13:24 <timburke> so now i'm remembering my idea of have a pre-mortem for the py3 roll-out
21:14:10 <timburke> i'm still not entirely sure of the right format for that discussion though
21:15:49 <timburke> maybe we could all commit to thinking about some of the ways this could all go sideways and discuss it at the next meeting?
21:16:03 <zaitcev> We tried to imagine failure scenarios, so I suppose it's going to be a laundry list of scenarios and canned workarounds.
21:17:52 <mattoliverau> may an etherpad to help collect sideway ideas.
21:18:09 <mattoliverau> I'm not a morning person, so my brain doesn't fire well this time of the morning :P
21:18:11 <timburke> yeah, i'm not entirely sure that it'll be as useful as i found it for sharding. the failures all look like "py3 did something different from py2" and the fixes are all "make py3 do what py2 did" :-/
21:18:48 <timburke> mattoliverau, fair enough. and i'm certainly open to doing it at another time
21:18:59 <mattoliverau> oh the next meeting is good.
21:19:29 <mattoliverau> but an etherpad to collect ideas in the mean time then we can go through.. or bring up ideas at the time.
21:19:51 <timburke> sounds good. i can get that set up
21:20:42 <timburke> #topic versioning
21:21:12 <timburke> clayg, how's it going? anything the rest of us should be thinking about or helping with?
21:26:13 <timburke> so... we've realized that we need to change the naming convention for the archived versions
21:27:20 <timburke> and we're debating about whether to do that *just* for s3api, do it as another "mode" of versioned_writes, or even to pull it out to another separate middleware
21:28:30 <timburke> i'm not sure how everyone else feels about those options, though
21:28:35 <zaitcev> I thought you guys wanted the mode route, otherwise simultaneous access through 2 APIs gets difficult.
21:28:53 <timburke> zaitcev, that's certainly my contention
21:29:15 <zaitcev> That's more technical baggage but oh well.
21:29:26 <clayg> Yeah. Clients should really only set the new mode and forget.
21:29:42 <clayg> Via s3api it’s just “enable”
21:29:44 <mattoliverau> Is this renaming just for the s3api versioning, or the whole idea of symlink versioning?
21:29:50 <timburke> but it also requires writing a new API for swift, which has historically taken a while for us to settle on
21:29:55 <zaitcev> Wait, operator can't make it default for a new cluster?
21:29:57 <mattoliverau> symlink versioning is a big win, so don't want it s3api only
21:31:11 <clayg> Symlink versioning turned out to be incompatible and incomplete on its own.
21:31:56 <timburke> symlinks and the naming change are orthogonal, i think. i agree that symlinks on their own would be a significant win for operators and users
21:32:51 <clayg> I agree it would be nice to have better versioning available in swift - but no one is really asking for that. And I’m frustrated adding more and more modes to the existing versioned-writes
21:33:18 <clayg> It’s a lot of work. Every new mode makes exponential test growth.
21:33:32 <kota_> right
21:33:41 <clayg> And it’s not getting us where we want to go. Which is s3api versioning.
21:34:29 <timburke> fwiw, the issue prompting the naming change is that s3api wants an index like (name, version DESC), while our existing naming scheme gets us (len(name), name, version)
21:36:25 <tdasilva> I've started to play with pulling out "new mode" into a separate middleware. but as tim pointed out it's really a new api and changes the behavior a bit from versioned-writes. it would mirror much more s3. One of the challenges is that I'd like to have the same option of "enable/disable" version on a container, which I think would require "hiding" the versions container
21:37:25 <timburke> tdasilva, like, put it in a dot-account? or some other notion of "hiding"?
21:37:26 <tdasilva> I don't want users mocking directly into older versions, they would have to go through the version-id API, but that's something very different from what we have done in the past with swift
21:37:27 <clayg> timburke: also the whole tempest and impossible to dump non-symlink versions. I mean, it might be worth it if that was “all” but basically the cruft feels never ending to me. Stack vs history was already a lot to carry.
21:38:01 <tdasilva> timburke: well..dot-account would solve, but I don't think we can use that in this case
21:38:21 <timburke> yeah, i was thinking similarly -- makes the accounting messy
21:38:54 <tdasilva> so my idea was that the middleware would attempt to create a container with some long uuid and only operators would have write/read access into it??? is that crazy talk?
21:39:50 <clayg> WFM (but hard to enforce)
21:40:41 <clayg> Like we’d decorate it with sys meta, but if client PUTs after rebalance. 🤔
21:41:26 <timburke> operator seems too high a requirement to me... if i'm getting charged for usage at OVH, i better be able to clean out my old versions of things
21:41:32 <clayg> FWIW S3 clients already can’t write to +versioning - so that’s solved
21:41:40 <timburke> (without needing to talk to alecuyer or rledisez)
21:41:53 <tdasilva> timburke: you can clean out old versions, but you need to go through the version-id api
21:42:07 <timburke> ah...
21:42:13 <clayg> timburke: I assume there’s api access to delete - like S3
21:42:15 <tdasilva> DELETE /a/co?version=laksdj
21:42:27 <clayg> ^ 👍
21:44:02 <mattoliverau> ok. So sounds like we should make a new middleware for some versions-next-gen. Shall we just aim for a new middleware that people can either use or use the legecy. If so what do we name it. And if its not backward compatible.. which is fine, we just need to think about good documentation. and make it an Op decision if they want to add it to their pipelines.
21:45:34 <mattoliverau> I'd prefer to get access to it from Swift api, ie have it available in swift and not just s3api. because swift api should always be the first class citizen
21:45:53 <mattoliverau> at least those are my opinions
21:47:05 <clayg> Good input
21:48:59 <timburke> any other inputs for this discussion? clayg, anything else we want from this discussion?
21:49:19 <clayg> We can come back this. We’ve got a lot to do.
21:49:35 <timburke> ain't it the truth
21:49:42 <timburke> #topic lots of small files
21:49:43 <clayg> 😞
21:50:13 <timburke> alecuyer, i saw more patch sets on https://review.opendev.org/#/c/679022/ \o/
21:50:14 <patchbot> patch 679022 - swift (feature/losf) - WIP - Support RPC over plain HTTP (vs gRPC) - 4 patch sets
21:50:19 <alecuyer> Yes,
21:50:43 <kota_> great
21:50:56 <alecuyer> I finished changing the golang tests, fixed one thing with the statsd go library, and updated the go.mod imports
21:51:05 <alecuyer> (and got rid of everything grpc finally)
21:51:47 <alecuyer> I don't think it should change so much now, so if anyone wants to take a look thats great
21:52:34 <clayg> So long grpc 😞
21:52:54 <alecuyer> well.. it was that or eventlet.. it may come back ;)
21:53:12 <timburke> kota_, i feel like you've probably got the most experience with losf here (outside of OVH); think you'd be able to take a look?
21:53:28 <timburke> i could also see if i can convince swifterdarrell to take a look ;-)
21:53:48 <kota_> I like to try it; not sure how i can take long time to see.
21:53:55 <timburke> thanks
21:54:26 <timburke> #topic sharding
21:54:49 <timburke> mattoliverau, i saw a new patchset on https://review.opendev.org/#/c/675820/
21:54:49 <patchbot> patch 675820 - swift - sharder: Keep cleaving on empty shard ranges - 4 patch sets
21:54:54 <timburke> how are you feeling about it?
21:55:26 <mattoliverau> well I've removed the exception control handling
21:55:46 <clayg> woot!
21:55:47 <mattoliverau> turned out to be easier then I expected and so not too much has actaully changed
21:56:33 <timburke> 👍
21:56:40 <timburke> i'll try to take a look this week
21:57:00 <mattoliverau> So take a look. Still happy to rework the whole cleaving if we want. That is if you think it's still hard to follow.
21:57:13 <mattoliverau> although the cleaving inception isn't "simple"
21:57:15 <mattoliverau> :P
21:57:15 <clayg> it looks pretty straight forward to me..
21:57:20 <mattoliverau> cool
21:57:35 <mattoliverau> yeah, I think I over thought it the first time.. story of my life :)
21:57:46 <clayg> 🤣
21:57:49 <clayg> 🤗
21:58:16 <timburke> that's defintiely one of the things i really value about code review from you all :-)
21:58:27 <mattoliverau> +1
21:58:35 <timburke> all right, we're about out of time
21:58:48 <timburke> thank you all for coming, and thank you for working on swift!
21:58:58 <timburke> (and sorry again about being a few minutes late)
21:59:04 <timburke> #endmeeting