21:00:45 <notmyname> #startmeeting swift
21:00:46 <openstack> Meeting started Wed Dec 13 21:00:45 2017 UTC and is due to finish in 60 minutes.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:47 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:49 <openstack> The meeting name has been set to 'swift'
21:00:51 <timburke> yay! wooooo!
21:00:53 <notmyname> who's here for the swift team meeting?
21:00:57 <torgomatic> .
21:01:31 <kota_> hi
21:01:35 <rledisez> o/
21:01:46 <acoles> here
21:01:49 <notmyname> tdasilva: cschwede: ?
21:01:50 <mattoliverau> o/
21:01:55 <notmyname> clayg: ?
21:02:01 <clayg> yeah i'm here
21:02:58 <notmyname> agenda for this week is
21:02:59 <notmyname> #link https://wiki.openstack.org/wiki/Meetings/Swift
21:03:11 <notmyname> only a couple of things on it, and follow-ups from last week
21:03:19 <notmyname> #topic symlink status
21:03:25 <notmyname> patch 232162
21:03:25 <patchbot> https://review.openstack.org/#/c/232162/ - swift - Symlink implementation.
21:03:31 <notmyname> is it merged yet?
21:03:44 <kota_> not yet but close to land, imo
21:03:48 <notmyname> (I mean, obviously not, but what do we need?)
21:03:53 <notmyname> acoles: looks like you added a question to the meeting agenda
21:04:24 <cleong> hi
21:04:42 <acoles> notmyname: yes, about one last change that we should maybe squash in before symlink merges
21:05:05 <notmyname> cleong: welcome
21:05:31 <acoles> w.r.t. how container sync translates symlink sysmeta to x-symlink-target
21:06:20 <clayg> symlinks is going to be so awesome
21:06:28 <acoles> I have proposed that the symlink middleware should be deployed in container sync internal client pipeline (a) to avoid leaking symlink implementation detail to container sync and (b) to make it more explicit that symlinks are handled
21:06:33 <acoles> https://review.openstack.org/#/c/527126/
21:06:34 <patchbot> patch 527126 - swift - Use symlink in container-sync internal client pipe...
21:06:38 <clayg> ^ this is a pretty good idea
21:07:07 <acoles> TBH it was that way at some point - jrichli had a similar patch squashed in, then it changed
21:07:12 <clayg> using the public api interface (which we have to maintain) is a minor reduction of coupling between components
21:07:52 <acoles> the reason i am pressing the point is that deciding on the approach NOW avoids an upgrade issue later if we delay the change
21:08:15 <clayg> @acoles or we carry the cruft
21:08:38 <acoles> clayg: yes. so lets just call it one way or the other
21:08:42 <clayg> acoles: the pipeline change could just keep importing from symlink mw until some undetermined time in the future after the sun goes out
21:09:03 <kota_> one possible concern i had is it could cause operational failure, tho.
21:09:12 <clayg> kota_: +++
21:09:28 <notmyname> kota_: how so?
21:09:36 <kota_> exactly, it's tradeoff thing to leak the code into container-sync or configuration at internal-client.
21:09:38 <clayg> you HAVE to add symlink to your pipeline (AND your container sync internal client pipeline) *after* you upgrade
21:09:58 <clayg> I think it's roughly the same thing with encryption and container sync yes?
21:10:06 <kota_> the case I minded is operator sets the symlink to *proxy pipeline* but forget to set it in the internal-client pipeline.
21:10:08 <clayg> optional feature - and you have to add it to proxies and also to container sync
21:10:15 <acoles> clayg: yes, same with encryption
21:10:21 <kota_> clayg: yes, same thing with encryption
21:10:36 <clayg> kota_: +1 if you don't put it in internal-client pipeline with acoles change you just double copy objects instead of sync'ing symlinks
21:11:07 <clayg> kota_: not trivial to fix... but... I think we have some precedence... swift is sadly not known for being easy to configure
21:11:07 <notmyname> ask for a new feature to be enabled by putting it in two places doesn't seem too bad
21:11:14 <kota_> so it's why I just thought it can be acceptable just one line leak to the container-sync
21:11:34 <clayg> notmyname: I agree it's not bad - but maybe having container sync import mw code isn't so bad *either*
21:11:47 <clayg> acoles: just needs the decision hammer - squash or not - either way symlinks is going to be awesome
21:12:07 <timburke> what else uses internal-client.conf? will they need to know about symlinks, too?
21:12:09 <m_kazuhiro> sorry I'm late.
21:12:14 <clayg> kota_: I think your judgement is reasonable
21:12:23 <clayg> timburke: expiring objects?  no?
21:12:39 <clayg> timburke: reconciler... probably?  shit...
21:12:52 <kota_> welcome m_kazuhiro!
21:13:00 <notmyname> admittedly, I haven't done a deep examination of the symlink patch. but thinking of it from the perspective of docs (or just explaining it), it seems that explicitly adding it to two places is simplest. otherwise we've got additional things to carry in our head forever about interdependencies of code
21:13:17 <m_kazuhiro> kota_: o/
21:13:22 <clayg> notmyname: seems pretty justifiable stance to tae
21:13:43 <clayg> notmyname: "you want symlinks great - put it in all your piplines and the code will do the right thing!  forget it in some pipeline and you screwed up"
21:13:44 <kota_> notmyname: fair point
21:14:01 <clayg> i remember it was huge pain to unerstand all of the coupling of SLO with container sync
21:14:29 <notmyname> yeah, it would be different if it were an existing feature (like DLO)
21:14:33 <clayg> ... it has the extra complexity of out of order segment & manifest syncing - but the coupling there is bananas (again it's "just one line" but the ramifications were huge )
21:14:42 <kota_> perhaps, I can make a follow up to add warning and sample config for internal-client pipeline for container-sync
21:15:07 <clayg> notmyname: I think long term the "answer" might be making symlinks NOT optional
21:15:08 <kota_> warning if you missed to configure the pipeline.
21:15:15 <clayg> notmyname: timburke is trying to do the same thing for SLO I think
21:15:23 <notmyname> yep, I like that too :-)
21:15:24 <clayg> notmyname: because "swift api" should be "swift api"
21:15:26 <mattoliverau> kota_: good idea, like we do for encryption in the sample config
21:15:29 <notmyname> yep
21:15:43 <notmyname> kota_: yeah, that sounds like a good idea
21:15:45 <clayg> it's nice to add things as "experimental" or "optional" but at some point it's like ... yeah you need this thing in your swift
21:15:53 <timburke> clayg: makes me think of https://review.openstack.org/#/c/504472/ ...
21:15:54 <patchbot> patch 504472 - swift - Shorten typical proxy pipeline.
21:16:09 <clayg> timburke: that's pretty cool!
21:16:12 <notmyname> so, make it an explicit thing to be added now, then later make it required everywhere
21:16:16 <clayg> *starred*
21:16:18 <notmyname> timburke: yeah, I *so* want that
21:16:30 <acoles> kota_: I think having commented sections in internal-client.conf-sample with note about when they might need to be uncommented would be great
21:17:00 <kota_> acoles: sure. that's what I thought.
21:17:01 <clayg> m_kazuhiro: do you have any *reservations* about merging in acoles file work on the container-sync pipeline?  notmyname wants to make the call to squash and we *__ALL__* want symlinks to merge!
21:17:15 <clayg> acoles: you already have all that in your patch tho yeah?
21:17:40 <acoles> clayg: IIRC I have plenty of doc but not the comments in the actual conf sample
21:18:12 <acoles> but it has occurred to me since that doing that for encryption also would be good
21:18:28 <m_kazuhiro> clayg: What reservation means?
21:18:38 <notmyname> hesitations or concerns
21:18:57 <acoles> and perhaps also in proxy-server.conf sample, add notes that 'if you enable this here you may want to enable it in internal client config too'
21:20:00 <notmyname> m_kazuhiro: kota_: I feel like that conversation went pretty quickly. let me know when you've caught up and are comfortable
21:20:32 <kota_> notmyname: no objection from me. it's fair enough and make sense what way we should go.
21:20:38 <notmyname> rledisez: do you have any concerns, operator-wise, with adding the symlink entry into both proxy configs and also internal-client configs?
21:20:42 <m_kazuhiro> clayg: notmyname: I see. I don't have reservations.
21:21:18 <kota_> notmyname: please go a head the meeting
21:21:45 <notmyname> waiting on rledisez now :-) (he said he was here for the meeting)
21:21:48 <rledisez> notmyname: nope. if documented and set by default in a consistent way, no concerns at all
21:21:52 <notmyname> great!
21:22:07 <notmyname> ok, so it sounds like everyone is good with an explicit symlinks mention
21:22:18 <notmyname> who's going to squash the patch?
21:22:27 <notmyname> acoles: ? m_kazuhiro: ? kota_: ?
21:22:50 <acoles> I can
21:23:00 <kota_> notmyname: just talked with m_kazuhiro in internal chat about that
21:23:20 <kota_> acoles: thanks!
21:23:29 <m_kazuhiro> acoles: Thank you very much!
21:23:48 <notmyname> everything sounds good. is there anything else to bring up on the symlinks patch in this meeting?
21:23:58 <acoles> m_kazuhiro: no problem, thank you for driving symlinks to completion
21:24:03 <notmyname> I'll bring it up next week and expect it to be on master by then :-)
21:24:05 <acoles> and kota_ , tdasilva too
21:24:19 <kota_> :-)
21:24:27 <notmyname> all right, if nothing else, let's move on
21:24:33 <notmyname> #topic SLO data segments patch
21:24:39 <notmyname> patch 365371
21:24:39 <patchbot> https://review.openstack.org/#/c/365371/ - swift - Add support for data segments to SLO and Segmented...
21:25:08 <notmyname> it also has not landed yet
21:25:20 <timburke> and it has a pre-req: https://review.openstack.org/#/c/512804/
21:25:20 <notmyname> IIRC timburke was going to look at it too
21:25:21 <patchbot> patch 512804 - swift - Add base64decode function to common/utils
21:25:24 <notmyname> ah right
21:25:40 <timburke> (pretty painless pre-req, though)
21:26:20 <notmyname> I'll look at the pre-req one this week
21:26:32 <notmyname> is there anyone else who can too? (looks like an easy review)
21:26:38 <notmyname> did I just curse it?
21:27:41 <notmyname> wooo... such commitment ;-)
21:27:55 <clayg> i'd be ok doing the *pre-req* :P
21:28:02 <notmyname> thanks :-)
21:28:16 <notmyname> timburke: you still ok with reviewing the data segments one?
21:28:28 <notmyname> I know you were pretty involved, but IMO it's ok
21:28:54 <kota_> i could take a time to look at them.
21:29:08 <notmyname> ok, thanks kota_
21:29:08 <kota_> (expecting symlink will land soon)
21:29:21 <notmyname> #topic open discussion
21:29:22 <timburke> yeah -- i'm still a little hesitant to +A in light of how many of the recent patchsets i pushed, but i'm looking at it now
21:29:32 <notmyname> anything else to bring up?
21:29:42 <notmyname> acoles: you're getting back into container sharding, right?
21:30:15 * mattoliverau waits for acoles answer
21:30:17 <acoles> notmyname: yes, ramping up again and in particular working on delete
21:30:24 <mattoliverau> \o/
21:30:49 <acoles> there's a new trello card with some thoughts on managing to delete a sharded container without having to fix sysmeta deletion etc
21:31:20 <mattoliverau> acoles: I'll take a look and maybe comment today
21:31:22 <notmyname> #link https://trello.com/b/z6oKKI4Q/container-sharding
21:31:31 <acoles> so we'll warm up feature/deep again (I know mattoliverau did some work while we went quiet - thanks mattoliverau )
21:31:52 <mattoliverau> not as much as I'd have liked.. but works got me busy :(
21:31:56 <notmyname> any other topics from anyone?
21:31:59 <timburke> oh yeah, https://review.openstack.org/#/c/478611/ would probably be nice...
21:32:00 <patchbot> patch 478611 - python-swiftclient - Allow for object uploads > 5GB from stdin.
21:32:04 <acoles> https://trello.com/c/SDCjvOyt/135-dont-delete-shards-when-marking-root-as-deleted
21:32:09 <mattoliverau> but have a new manager so might be able to get more swift time in
21:32:17 <notmyname> yay!
21:33:01 * kota_ wanted to get s3api progress if tdasilva is here but it doesn't seem he is here, today.
21:33:03 <timburke> i might try to get to that this week, though maybe peoplewould be more interested in me actually finishing the bimodal-access docs i'd mentioned last week...
21:33:25 <notmyname> kota_: I'll put it on next week's agenda
21:33:31 <timburke> kota_: i was thinking about that recently, too :-)
21:33:37 <kota_> notmyname: :-)
21:33:43 <kota_> timburke: ^^^
21:34:31 <notmyname> looks like that's it, then
21:34:41 <notmyname> thanks everyone for coming. thanks for working on swift!
21:34:45 <notmyname> #endmeeting