21:00:12 <timburke> #startmeeting swift
21:00:12 <openstack> Meeting started Wed Aug  7 21:00:12 2019 UTC and is due to finish in 60 minutes.  The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:13 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:15 <openstack> The meeting name has been set to 'swift'
21:00:17 <timburke> who's here for the swift meeting?
21:00:19 <clayg> o/
21:00:29 <timburke> clayg, so excited! you beat the meeting start!
21:00:45 <kota_> o/
21:00:46 <kota_> lol
21:00:46 <clayg> 😁
21:01:04 <rledisez> o/
21:01:28 <tdasilva> o/
21:01:29 <mattoliverau> o/
21:01:38 <timburke> agenda's mostly a continuation of a lot of outstanding work
21:01:44 <timburke> #link https://wiki.openstack.org/wiki/Meetings/Swift
21:01:53 <timburke> #topic shanghai
21:02:01 <timburke> just a quick reminder that prices for the shanghai summit increase next week -- if you're going, you might want to buy now
21:02:09 <timburke> (i'm not sure off-hand what the price will increase *to*, i just know it will increase)
21:02:13 <timburke> #link https://www.openstack.org/summit/shanghai-2019/
21:02:59 <timburke> thanks everybody for decalring whether you'll be there or not on the etherpad -- it should definitely help the foundation plan for attendance
21:03:08 <kota_> and also timburke should respond to the TC how *Swift* PTG will be going?
21:03:28 <timburke> yup -- on my list for this week :-)
21:03:36 <kota_> great
21:03:52 <timburke> (attendance will probably be a little light, but then, we were expecting that)
21:04:00 <timburke> on to updates!
21:04:06 <timburke> #topic py3
21:04:11 <timburke> i've been working on finishing up the py3 func tests recently
21:04:18 <timburke> there were a couple patches small enough that i didn't feel bad +A'ing myself
21:04:22 <timburke> #link https://review.opendev.org/#/c/674878/
21:04:25 <timburke> #link https://review.opendev.org/#/c/674703/
21:04:34 <timburke> aside from that, the main advance was in starting on the s3 tests
21:04:38 <timburke> #link https://review.opendev.org/#/c/674716/
21:04:56 <timburke> and i saw that mattoliverau's been doing some review, so thanks for that!
21:05:30 <mattoliverau> Nps
21:06:05 <timburke> i'm torn between wanting probe tests (because they'll exercise more of the system) or func tests (so we can start reducing the total number of gate jobs running)
21:06:16 <timburke> but we're getting there on both fronts!
21:06:51 <timburke> any questions on py3?
21:07:25 <timburke> #topic lots of small files
21:07:56 <timburke> i'm guessing there hasn't been too much going on upstream here, with alecuyer still on vacation
21:08:05 <kota_> timburke: true
21:08:15 <rledisez> timburke: yes
21:08:18 <kota_> nothing except I bought the PTG ticket.
21:08:27 <timburke> but we've been playing around with it at SwiftStack! we can get it to run, at least :-)
21:08:55 <kota_> +1
21:09:29 <timburke> is there anything else we should talk about for it? what are the next steps that we want to take?
21:09:31 <rledisez> wow, that's something! somebody else make it running \o/
21:09:52 <timburke> rledisez, always good when we can get past "works on my machine" :-)
21:10:07 <clayg> or "works in my public cloud" or w/e
21:10:18 <rledisez> :)
21:10:32 <clayg> but definately progress - it's gunna be a hoot
21:11:27 <timburke> hopefully i'll be able to give it more attention Real Soon Now
21:12:02 <timburke> rledisez, as we find issues, where's a good place for us to collect them?
21:12:21 <clayg> gerrit?  with fixes?
21:12:53 <rledisez> clayg: +1
21:12:54 <clayg> obviously if we don't know *how* to fix it a failing test and a ping in IRC would be a good start... probably NOT lp bugs
21:13:04 <timburke> clayg, that's my usual way of dealing with issues, but sometimes people want something more trackable ;-P
21:13:16 <clayg> but we could certainly do a etherpad or trello tracker if activity ramps up to the point that it's needed?
21:13:23 <timburke> i was trying to remember if we had a trello board set up or anything
21:13:34 <kota_> trello might be good place
21:13:43 <rledisez> and definitively a ping, because we are some commits ahead for some bugfix (I think it's on alecuyer todolist to upstream them asap)
21:13:47 <kota_> https://trello.com/b/xhNxrcLX/losf
21:13:52 <kota_> #link: https://trello.com/b/xhNxrcLX/losf
21:13:56 <timburke> thanks kota_
21:13:59 <clayg> ^ oh, BOOM
21:14:44 <timburke> love it. i think there's one thing i oughta write up (or as clayg suggested, just go ahead and fix)
21:15:01 <timburke> #topic sharding
21:15:14 <timburke> i know we've got mattoliverau's patches
21:15:18 <mattoliverau> I haven't done much extra here
21:15:50 <timburke> i put up one or two others -- not exactly to do with autosharding, but certainly to do with having a completed feature
21:15:51 <mattoliverau> I've been meaning to do some more testing of the existing patches, but got caught up with work :(
21:16:01 <mattoliverau> I saw! thanks
21:16:29 <timburke> #link https://review.opendev.org/#/c/675014/
21:16:35 <mattoliverau> One tracking updated and some py3 fixes right?
21:16:46 <timburke> in particular, to stop hammering the root container quite so much
21:17:08 <mattoliverau> yeah thank'll be a good thing :)
21:17:15 <mattoliverau> *that'll
21:17:21 <timburke> oh, right -- the other thing wasn't a patch but a bug
21:17:23 <timburke> #link https://bugs.launchpad.net/swift/+bug/1839355
21:17:24 <openstack> Launchpad bug 1839355 in OpenStack Object Storage (swift) "container-sharder should keep cleaving when there are no rows" [Undecided,New]
21:17:54 <mattoliverau> oh, interesting
21:18:29 <timburke> though i'm not entirely sure on the right resolution there. working through mostly-empty DBs quickly would be good, but i wonder if we could save the hassle of creating (and replicating) all those shard dbs
21:18:44 <timburke> somethign to think about, anyway
21:19:04 <mattoliverau> +1
21:19:18 <timburke> #topic symlinks and versioning
21:19:31 <clayg> the realization for me was that it can take a long time to shard an empty container just because of the cardinality of shard ranges (the handoff knows about ranges because of replication, but it still takes many cycles to say "yup, noting in that range, nothing in that one either; good enough for this cycle - see you next time!" ZZZzzzzz
21:20:50 <timburke> yeah, that was about the point at which i wrote up the bug, shortly after doing the math to figure out how long it'll take to clear it
21:21:05 <clayg> hardlinks to SLOs won't always have the slo_etag and symlink_bytes bits correct in the container listings -> https://review.opendev.org/#/c/633094/24/swift/common/middleware/symlink.py
21:21:38 <timburke> not sure there's much we can do about that, though :-/
21:22:05 <clayg> would love to have some feedback on how we put that in the docs without sounding stupid... "fixing it" seems not worth it considering the cruft and time it would take to essentially just reupload the manifest for them
21:22:33 <clayg> sometimes trying to make things better just makes things more complicated 🤷‍♂️
21:22:34 <timburke> on the plus side, neither will the SLO it's pointing at...
21:22:47 <clayg> timburke: that's not a plus by any measure
21:23:23 <clayg> it's just bad and now the badness will accretes
21:23:30 <timburke> true enough. but we're *mostly* not making things actively worse?
21:24:25 <clayg> I mean I think that paragraph that explains how hardlinks worse *does* make it actively worse  - like the sphere of context where the problem forced users to have to think about it is bigger now
21:24:40 <clayg> ... but on the whole I think the change is still a net gain - obviously I want hardlinks!
21:24:40 <timburke> i'm still hopeful that the total number of SLOs written *with* the metadata will be orders of magnitude larger than the total number written *without*, given enough time
21:24:50 <clayg> just rubber hit the road and this is best we've come up with yet
21:24:57 <clayg> ... suggestions welcome 😁
21:26:06 <clayg> i'm rebasing symlink_versions now and mostly hoping i'm done with hardlinks (sans the amazing doco suggestions I'm going to get following this meeting)
21:26:28 <clayg> I am also finding it annoying that you have to dig out the manifest etag to make a hardlink
21:26:56 <timburke> clayg, is there anything the rest of us should be weighing in on? are you looking for consensus on what the listing should look like, say?
21:27:14 <clayg> I'd like to either add support python-swiftclient's `stat` command to do query-params (?multipart-manifest=get) OR make *LO's always return the an X-Manifest-Etag
21:27:23 <clayg> ... or both?  preferences?  opinions?
21:27:59 <clayg> I would like as many people as possible to read the docs on the hardlink patch so they understand the feature
21:27:59 <timburke> x-manifest-etag's not a bad idea... we've already got it loaded in our head anyway...
21:28:07 <tdasilva> seems like the api change is better than the client change??
21:28:42 <clayg> if they want to play with `swift upload links -H 'content-length: 0' --object-name link.version -H 'x-symlink-target: test/big.version' -H 'x-symlink-target-etag: b89c395acda3886b823803b7dccb4765' - </dev/null` that's cool too, practically a review!
21:29:04 <clayg> but just reading the docs and asking questions about the parts that seem weird would be SUPER helpful - maybe we missed something obvious
21:29:33 <clayg> but even better - it adds documentation to the review - so later after we merge it and we're like "why didn't we do XYZ" we wrote that down somewhere
21:29:33 <timburke> client change might be nice, too -- i could see some utility to being able to download the SLO manifest from the cli more easily
21:30:55 <clayg> timburke: well there's also symlink=get - so yeah broad support in the cli for query params would be pretty sick
21:31:05 <tdasilva> timburke: agree, was just thinking in terms of one versus the other, but both sound even better :)
21:31:06 <clayg> but it sounds like everyone likes x-manifest-etag too - so that's probably even cheaper
21:31:47 <timburke> clayg, do you want to add that as part of the hardlinks patch, or should i propose that separately?
21:32:20 <timburke> or you could do it. maybe you'd prefer i spend my time reviewing ;-)
21:32:23 <clayg> no strong preference on my end - probably sufficiently orthogonal to warrent a seperate patch if for no other reason to keep patch count down
21:32:35 <timburke> 👍
21:32:52 <clayg> well, I'm not going to do anything until I finish the rebase on symlink_versions and I'm out of time today...
21:33:13 <timburke> i'll see about knocking that out real quick
21:33:16 <clayg> so if while looking at hardlinks you were to "accidently" purpose a dependent patch that added x-manifest-etag I could +2 it in the morning ;)
21:33:40 <timburke> anything else for symlinks and versioning?
21:34:36 <clayg> not at this time - other non swiftstack cores - please take 10m to read the docs and throw some comments on there
21:34:42 <clayg> happy to take questions
21:34:48 <clayg> appreciate the feedback
21:35:06 <kota_> will do
21:35:17 <kota_> might take a time Friday on my time.
21:35:42 <timburke> thanks kota_
21:35:44 <timburke> #topic open discussion
21:35:53 <timburke> anything else we ought to discuss?
21:37:27 <timburke> fwiw i've been playing with different ways to get a feel for what's going on in swift, and i feel like i've come up with a decent dashboard, at least to prep for this meeting
21:37:30 <timburke> #link https://review.opendev.org/#/dashboard/?title=The+Week+That+Was&foreach=-age:1week&Landed+(Server)=is:merged+project:openstack/swift&Landed+(Client)=is:merged+project:openstack/python-swiftclient&Active+(Server)=is:open+project:openstack/swift&Active+(Client)=is:open+project:openstack/python-swiftclient&Abandoned+(Server)=is:abandoned+project:openstack/swift&Abandoned+(Client)=is:abandoned+project:openstack/python-swiftclient
21:38:24 <kota_> looks good
21:38:58 <timburke> i might need to propose fewer patches though -- i'm not sure i like seeing my name that much :-/
21:39:24 <mattoliverau> timburke: don't stop being you, your awesome!
21:40:55 <timburke> all right, seems like there isn't much more to talk about right now -- let's call it a little early and let kota_ and mattoliverau get breakfast ;-)
21:41:04 <mattoliverau> And just to jump back a bit. I'll start reading the hardlink docs today. I have some long meetings this morning... I'll need something to read :P
21:41:18 <mattoliverau> \o/ breakfast time!
21:41:21 <timburke> thanks mattoliverau!
21:41:24 <timburke> thank you all for coming, and thank you for working on swift!
21:41:32 <timburke> #endmeeting