21:00:31 <notmyname> #startmeeting swift
21:00:32 <openstack> Meeting started Wed Apr  6 21:00:31 2016 UTC and is due to finish in 60 minutes.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:33 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:35 <openstack> The meeting name has been set to 'swift'
21:00:40 <notmyname> who's here for the swift meeting?
21:00:49 <dmorita> o/
21:00:52 <joeljwright> hey
21:00:53 <cutforth> hola
21:00:54 <gmmaha> o/
21:00:54 <jrichli> o/
21:00:54 <kota_> hello
21:00:55 <sgundur> hi
21:01:19 <tdasilva> hi
21:01:47 <siva_krishnan> o/
21:01:54 <acoles> here
21:01:59 <tsymanczyk> wc
21:02:00 <ntata> o/
21:02:12 <kota_> hi
21:02:13 <notmyname> welcome everyone
21:02:19 <notmyname> agenda is at
21:02:22 <notmyname> #link https://wiki.openstack.org/wiki/Meetings/Swift
21:02:35 * notmyname wishes the # commands would work mid-line
21:03:07 <notmyname> several interesting things have been proposed on the agenda this week
21:03:14 <notmyname> #topic rolling upgrade tests
21:03:24 <notmyname> first up, checking in on the rolling upgrade testing
21:03:31 <notmyname> #link https://etherpad.openstack.org/p/swift-rolling-upgrade-multinode-testing
21:03:33 <notmyname> cschwede: are you here?
21:03:48 <notmyname> the todo section at the bottom is what's needed, I think
21:04:00 <notmyname> acoles: timburke: do you have any other details or info?
21:04:34 <acoles> notmyname: sorry no, i have been engrossed in feature/crypto stuff this week
21:04:41 <notmyname> that's ok :-)
21:04:48 <timburke> over on swift3 stuff, myself
21:05:27 <notmyname> I haven't seen cschwede around much this week, bus I suspect that's because of the summer time zone. seems like I'm on relatively later in the mornings now
21:05:43 <ntata> cd
21:05:43 <notmyname> I'll keep bugging people about this and helping where I can
21:06:10 <notmyname> although we got a reprieve on the tag thing, we still need the testing to be implemented relatively quickly
21:06:12 <acoles> I'll try to check in with cschwede tomorrow Europe time
21:06:21 <notmyname> acoles: thanks
21:06:32 <notmyname> next up, a quick topic
21:06:33 <clayg> notmyname: yeah i thought there was a timelimit component - if we're under the gun I might be able to steal some cycles if needed
21:07:01 <notmyname> clayg: we were spared this time, with a "we'll check back in a few weeks" vague comemnt
21:07:08 <notmyname> that's all I know
21:07:13 <clayg> uh huh
21:07:22 <notmyname> #topic reverse listings bug follow-up
21:07:37 <notmyname> in the process of setting up some of this testing, we discovered https://bugs.launchpad.net/swift/+bug/1562083
21:07:39 <openstack> Launchpad bug 1562083 in OpenStack Object Storage (swift) "mid-upgrade clusters can cause versioned write errors" [Medium,Fix released] - Assigned to Tim Burke (1-tim-z)
21:07:46 <notmyname> and timburke fixed it. yay! it's on master now
21:08:14 <notmyname> IMO it's a good candidate for a backport
21:08:25 <acoles> agree
21:08:42 <notmyname> #topic idea: swift.callback.* namespace in wsgi environment
21:08:58 <notmyname> this is something tdasilva and acoles were talking about when i got online this morning. kinda interesting, i think
21:09:05 <notmyname> acoles: tdasilva: can you give a summary?
21:09:26 <acoles> this came up while reviewing https://review.openstack.org/#/c/237393/25
21:09:36 <acoles> patch 237393
21:09:49 <timburke> sad patchbot :(
21:09:59 <notmyname> ah
21:10:15 <notmyname> patch 237393
21:10:16 <patchbot> notmyname: https://review.openstack.org/#/c/237393/ - swift (feature/crypto) - Support for http footers - Replication and EC
21:10:16 <acoles> where we introduce a callback for middleware to get the chance to add footers to a multipart object PUT to backend
21:10:24 <acoles> hey patchbot!
21:10:59 <acoles> and tdasilva raised idea of starting to use some uniform naming of things swift puts into the req environ
21:11:15 <notmyname> specifically the callbacks, right?
21:11:26 <acoles> so in this case swift.callback.update_footers or similar
21:11:29 <acoles> notmyname: yes
21:11:44 <notmyname> what else is there? swift.authorize. others?
21:12:01 <tdasilva> yeah, the name itself is not the point, just that we could have something uniform...swift.callback, swift.hook, etc...
21:12:03 <acoles> the only existing callback i could think of is authorize which we probably need to stick with for 3rd party auth m/ware
21:12:30 <acoles> and copy_hook which may go away soon anyway
21:12:40 <notmyname> may?
21:12:57 * tdasilva hopes that it *will*
21:13:13 <acoles> notmyname: its not done til its done ;)
21:13:20 <notmyname> :-)
21:13:22 <tdasilva> true :)
21:13:56 <tdasilva> acoles: are you suggesting people should review copy middleware?
21:14:05 <notmyname> so the idea is swift.callback.authorize (and probably some migration plan), swift.authorize.update_footers, swift.authorize.do_what_i_mean, and antyhing else goes there
21:14:06 * tdasilva is done with shameless plug
21:14:15 <acoles> tdasilva: subtlely yes
21:14:50 <acoles> notmyname: yes but less of the authorize ;)
21:14:55 <tdasilva> lol
21:14:56 <tdasilva> yes
21:15:48 <acoles> question is whether it has value/merit? IDK cos right now there's so few examples
21:15:52 <notmyname> so, not to be argumentative, but we've got 2 examples of callbacks right now, and one won't use this new thing. so why do we want to have a new convention for it?
21:16:10 <clayg> notmyname: I see very little value in trying to change the name of the authorize callback
21:16:38 <clayg> notmyname: it's one of those backwards compat crufts that's a bit out of our control - easier to just live with the old and have a better plan for the moving forward IMHO
21:16:56 <notmyname> clayg: do you think this idea is reasonable going forward?
21:17:10 <acoles> agree, i think changing authorize is just making unnecessary work
21:17:44 <notmyname> ok. I'm not trying to argue that at all.
21:17:49 <jrichli> btw, technically there is a fetch_keys callback introduced in crypto too
21:18:17 <acoles> jrichli: right! i'd missed that, so we have two new examples
21:18:21 <clayg> notmyname: if we're going to change the interface for authorize we should do it for something functional - like lots of auth middleware have to do these silly closures of env data because of make_sub_request - last time I thought about it; i seem to recall thinking the callback should have some better args - something more contextextual about who & what - something like policy
21:19:24 <notmyname> acoles: jrichli: that's twice as many as before! ;-)
21:19:39 <notmyname> I think having a callback namespace (swift.callback.*) is fine. seems like something future me would appreciate
21:20:05 <notmyname> and since the only 2 examples we have right now are part of the crypto branch, then there's not yet any worries about migrations
21:20:11 <clayg> notmyname: yeah having all new call backs under common name is a start - documentation is next - maintainace is forever :'(
21:20:14 <notmyname> so it's more of a "this seems reasonable so let's do it"
21:20:44 <notmyname> clayg: you told me yesterday "tests are the hard part". I think "docs are the hardest part" should be added there :-)
21:20:54 <clayg> notmyname: nice
21:21:16 <clayg> code is easy, tests are hard, words are near impossible
21:21:20 <notmyname> lol
21:21:38 <notmyname> anyone else have somethign to share about a swift.callback.* convention in the wsgi environment?
21:21:44 <tdasilva> sounds like none of you have done GUI work
21:21:55 <clayg> tdasilva: rofl!
21:21:56 <joeljwright> :D
21:22:03 <acoles> tdasilva: thank goodness! lol
21:22:05 <notmyname> I thought we were talking about reality ;-)
21:22:23 <notmyname> "...pixels are just right out"
21:22:58 <notmyname> so does the callback naming cause any major waves for the crypto branch at this point?
21:23:01 <tdasilva> notmyname: it sounds like crypto is a good place to start
21:23:05 <clayg> FWIW i think callbacks are an annoying interface to document and maintain - you put this here and the code will call it ... right ... *here*
21:23:36 <clayg> I think sometimes it's obviously the simplest problem to address the challange at hand - but it doesn't always scale
21:23:38 <jrichli> no major waves really: just a patch to replace the strings being used now
21:23:42 <clayg> so this is me having no better ideas :'(
21:23:44 <clayg> DO IT!
21:23:48 <acoles> notmyname: ok I/we will update our callback names accordingly
21:23:53 <notmyname> clayg: meh. but it's probably better than other sort of IPC like queues and shared memory?
21:24:27 <notmyname> ok, let's move on then
21:24:32 <notmyname> #topic copy middleware patch
21:24:36 <notmyname> https://review.openstack.org/#/c/156923/
21:24:36 <patchbot> notmyname: patch 156923 - swift - Refactor server side copy as middleware
21:24:46 <notmyname> I should have added this to the agenda, but thanks for bringing it up tdasilva
21:24:49 <tdasilva> sneaky
21:25:27 <notmyname> this is by far the most heavily starred patch in the swift community, for what that's worth. and beyond simplifying some existing code, it's critical for the crypto work
21:25:39 <notmyname> in other words, it's pretty much the number one thing that needs to get done
21:25:40 <jrichli> patch 260179 first though
21:25:40 <patchbot> jrichli: https://review.openstack.org/#/c/260179/ - swift - decouple versioned writes from COPY
21:26:06 <clayg> I haven't reviewed it - I might have time around Friday - if we don't already have reviewers on deck?
21:26:12 <notmyname> what's the status for reviews?
21:26:42 <tdasilva> there's been some reviews from kota_, torgomatic, Hisashi
21:26:42 <notmyname> kota_: thanks for reviewing this one
21:26:45 <acoles> kota_: has done some reviews on 260179, thanks kota
21:27:12 <kota_> yup, mostly I've been in final straight to add +2 I think.
21:27:24 <kota_> I'm in testing in my lab env.
21:27:34 <tdasilva> thanks kota_
21:27:35 <clayg> are any swift-core recused?  tdasilva & acoles ?  Can we let them have a vote?
21:27:35 <notmyname> tdasilva: you're effectively the owner on both of these
21:27:46 <tdasilva> yeah, i can't vote
21:27:47 <notmyname> what's your sense of where it is? what do you need?
21:27:50 <kota_> maybe I can make comments (or just +2) in this week.
21:28:01 <tdasilva> i think it's good to go!
21:28:12 <timburke> i know i need to leave a comment; SLO copies can fail under some circumstances. but maybe we don't care
21:28:31 <clayg> can patch 156923 land without patch 260179 - or are they effectively a chain?
21:28:31 <patchbot> clayg: https://review.openstack.org/#/c/156923/ - swift - Refactor server side copy as middleware
21:28:32 <patchbot> clayg: https://review.openstack.org/#/c/260179/ - swift - decouple versioned writes from COPY
21:28:35 <tdasilva> timburke: i'm curious
21:28:46 <tdasilva> clayg: they are a chain
21:29:29 <notmyname> tdasilva: you'll have to wait for is review comment ;-)
21:29:33 <notmyname> *his
21:29:59 <clayg> we might have to -2 the head of the chain head then?  at least we've done that in the past when breaking up changesets that need to land together
21:30:24 <notmyname> is it bad if the first lands without the second?
21:30:27 <notmyname> if so, I agree
21:30:36 <acoles> i think the first can land on its own
21:30:43 <acoles> it still supports the COPY verb
21:30:50 <tdasilva> decouple can land on its own
21:31:01 <acoles> in fact once the second lands the COPY verb needs to be taken out of the decouple one IIRC
21:31:16 <notmyname> ok
21:32:15 <notmyname> therefore it doesn't need a -2 plug
21:32:19 <clayg> notmyname: full ack
21:32:20 <acoles> tdasilva: does the middleware patch remove COPY verb handling from versioned writes?
21:32:52 <notmyname> so if you're name is not tdasilva, then please review patch 260179 followed by patch 156923
21:32:52 <patchbot> notmyname: https://review.openstack.org/#/c/260179/ - swift - decouple versioned writes from COPY
21:32:53 <patchbot> notmyname: https://review.openstack.org/#/c/156923/ - swift - Refactor server side copy as middleware
21:32:56 <tdasilva> acoles: no, COPY verb gets removed in patch 260179
21:32:56 <patchbot> tdasilva: https://review.openstack.org/#/c/260179/ - swift - decouple versioned writes from COPY
21:33:55 <clayg> oh i had the chain backwards?  gd new gerrit ui
21:34:23 <notmyname> clayg: I was afraid I got it backwards
21:34:41 <tdasilva> clayg: sorry if I confused people, but basically copy middleware depends on decouple versioned writes from COPY
21:34:43 <acoles> tdasilva: yes versioned writes won't issue COPY requests but still expects to receive COPY requests I think
21:35:24 <tdasilva> acoles: only because it will land first...
21:35:27 <acoles> tdasilva: line 501 https://review.openstack.org/#/c/260179/16/swift/common/middleware/versioned_writes.py
21:35:27 <patchbot> acoles: patch 260179 - swift - decouple versioned writes from COPY
21:35:28 <clayg> acoles: tdasilva: that acctually makes pleanty of sense
21:35:36 <acoles> tdasilva: yep, its cool,
21:36:59 <tdasilva> acoles: once copy middleware lands, that code becomes useless, so we can remove it
21:37:19 <acoles> notmyname: clayg: IIRC I got to point where I didn't feel I should +2 either of those patches cos I got involved
21:37:23 * cschwede arrives late to the party, sorry
21:37:26 <acoles> tdasilva: yes thats what I was thinking
21:37:37 <kota_> that means patch 260179 still expects incomming COPY but doesn't use COPY in itself.
21:37:37 <patchbot> kota_: https://review.openstack.org/#/c/260179/ - swift - decouple versioned writes from COPY
21:37:47 <kota_> in my undestunding.
21:37:47 <clayg> tdasilva: oh oh oh *third* patch!  maybe a blocked series would be better
21:37:53 <notmyname> tdasilva: can you get this sort of info (the chain, what's removed where, and what happens after) into https://wiki.openstack.org/wiki/Swift/PriorityReviews?
21:38:01 <notmyname> tdasilva: I'll be happy to help, if you need it
21:38:18 <notmyname> tdasilva: I think having it written out there will help us all know first this, then that, etc
21:38:47 <tdasilva> notmyname: honestly, i'd prefer if people just focused on this two patches...this 3rd one that acoles and I are talking about is not super important
21:38:49 <clayg> ok, so al & thiago are out - maybe if tim can get his +2 on it in the next couple of days I can sweep the +A?
21:38:56 <tdasilva> so i don't want to confuse people
21:38:57 <clayg> oh - or Kota!
21:39:40 <acoles> tdasilva: or we add that to 156923 - i.e. remove COPY handling from versioned writes at same time COPY mware lands
21:39:43 <clayg> tdasilva: my thinking was more about if you have to load it all in your head to understand what's going on - and we're doing weird stuff to keep backwards compat during a transistion that only last from landing one patch to the next - a chain is a reasonable approach
21:39:57 <notmyname> yeah, I think we've got the bandwidth. with acoles and tdasilva writing, kota_ and timburke and torgomatic who have looked. maybe getting clayg myself or cschwede to take a look this week
21:40:16 <notmyname> tdasilva: yeah, I agree about staying focused
21:40:19 <clayg> cschwede: will be busy with upgrade tests :P
21:40:35 <notmyname> :-)
21:40:37 <clayg> is sam still sick?
21:40:43 <acoles> tdasilva and i may sum our +1's ;)
21:40:46 <notmyname> I'm guessing so. haven't seen him today
21:40:52 <tdasilva> lol
21:40:53 <cschwede> clayg: exactly
21:41:15 <notmyname> anythign else we need to cover on this topic in the meeting?
21:41:55 <clayg> tdasilva: you think the rolling upgrade impact is covered?
21:42:21 <tdasilva> clayg: mmm...now you got me
21:42:29 <clayg> tdasilva: commit message mentioned snarfing post_as_copy from proxy-app section - which sounded pretty reasonable
21:42:43 <tdasilva> clayg: right
21:43:47 <tdasilva> clayg: very quickly thinking about it, I don't think rolling upgrade would be an issue
21:44:36 <clayg> i bet it's going to be great - can't wait to look at it!
21:45:56 <notmyname> ok, let's move on
21:45:59 <notmyname> #topic summit prep status
21:46:03 <notmyname> mostly just a reminder
21:46:06 <notmyname> #link https://etherpad.openstack.org/p/swift-newton-summit-planning
21:46:08 <clayg> tdasilva: acoles: if kota_ or timburke are feeling particuarlly strong about it i'm ok with you one or either of you guys +A'ing - since we're at the start of the cycle - just IMHO
21:46:09 <notmyname> fill that out. add your ideas
21:46:25 <acoles> clayg: ack
21:46:35 <notmyname> and if you are giving a conference talk, please add it in the "conflicts" section
21:47:16 <notmyname> there's some good stuff there, but we've got room for more topics, i think
21:47:38 <notmyname> starting next week, I'll be organizing them into a schedule and pushign that to the scheduling system
21:47:40 <clayg> notmyname: hrou told me he's short on round tuits for sysmlinks work - and matt's still hung out to dry on container sharding ...
21:48:08 <clayg> ... I'm not sure when/how to deal with "XYZ will save us" only works out if we have infinite patience and people don't have lives/jobs
21:48:34 <notmyname> clayg: there's nothing (yet) on that etherpad about symlinks. if we have another champion, it should definitely be added
21:48:59 <notmyname> "But don't try to force dates or time frames; even if everyone is enthusiastic about a direction, it is unlikely that people can guarantee full-time effort for long enough to finish by a chosen date."
21:49:07 <notmyname> from the "lessons Learned" on http://www.aosabook.org/en/gdb.html
21:49:13 <clayg> notmyname: yeah that's my point - i don't think it has a champion - timburke wants it in some nebulous future of amazing super duper data protection wonderment - and I think tiering was counting on it?
21:49:54 <kota_> clayg: yeah
21:50:27 <clayg> notmyname: well i guess ... hrmm ... you know maybe I need to default to your judgement
21:50:48 <clayg> I'm thinking "hey no one is working on this we need to talk about that!" - maybe you're thinking "yeah, if someone is working on it - let's talk about it!"
21:51:08 <tdasilva> at one point I wanted to raise my hand to work on that, but now I'm busy with crypto...maybe after crypto??
21:51:30 <notmyname> it should be added to the etherpad. I don't want to overlook it when figuring out the schedule, but "someone who's working on it is present" is a heavy weighting factor in scheduling
21:51:57 <hrou> Hey All, yea I'd love for symlink to make some progress though : ) - I think there are enough folks who care (maybe cared is the right tense as I haven't checked in lately)
21:52:34 <notmyname> hrou: yeah, I think we all want it to make progress. just not sure about actual ability to pound out code
21:53:23 <hrou> notmyname, yea I think maybe what I could do to help is to put together a bit of status;  There was an etherpad floating around a while back but its a bit of date
21:53:39 <hrou> it really comes back to the handling of POST for fast-post (now default yey).
21:53:39 <notmyname> hrou: ok, thanks
21:53:54 <notmyname> I added a placeholder to the bottom of https://etherpad.openstack.org/p/swift-newton-summit-planning
21:53:58 <hrou> sure its my pleasure, I'd love to see it land one day.
21:53:58 <clayg> hrou: I don't think we quite went as far as changing the default FWIW
21:53:59 <hrou> Thanks!
21:54:06 <clayg> hrou: thank you!
21:54:21 <notmyname> hrou: any details/links you can throw in there would be good
21:54:30 <notmyname> ok, last topic
21:54:33 <hrou> clayg, meh one can hope ;)  I was being optimistic.  But yea we def want it working with both
21:54:33 <notmyname> #topic swiftclient docs
21:54:39 <notmyname> joeljwright: what's up here?
21:54:47 <notmyname> I hear you've been making great progress!
21:54:53 <joeljwright> well, we've made some really good progress
21:54:54 <joeljwright> https://review.openstack.org/#/c/288566/6
21:54:55 <patchbot> joeljwright: patch 288566 - python-swiftclient - WIP: This patch adds a new doc structure for swift...
21:55:10 <joeljwright> service-api.rst is now 'complete'
21:55:17 <joeljwright> and has a full complement of examples
21:55:27 <clayg> notmyname: I was *amazed* at what's there now - phenominal work - so many KUDOS to the people working on that
21:55:28 <joeljwright> cli.rst just needs examples
21:55:44 <joeljwright> connection-api.rst needs auth and examples
21:55:48 <joeljwright> I'm still working on them
21:55:54 <joeljwright> but would love people to start reviewing
21:56:00 <notmyname> yeah, that looks great
21:56:18 <joeljwright> suggestions/comments are most welcome
21:56:22 <notmyname> quick question, do you want to keep this all as one big chunk like this? or can/should it be split to land in smaller pieces?
21:56:32 <acoles> joeljwright: can the sections that are complete be broken out into a review that is ready to merge?
21:56:34 <clayg> joeljwright: do you really need the feedback or just the merges?
21:56:42 <joeljwright> I think we can get this one ready soon enough to land in one piece
21:56:50 <joeljwright> suggestions will probably have to be follow on
21:56:59 <clayg> joeljwright: we're going from 0 to %^&Uing awesome!  How is that not a win?  I'll go +A some stuff right now.
21:57:12 <notmyname> joeljwright: reall? that's great. when are you expecting it to be done, based on current progress?
21:57:18 <joeljwright> only the listed reviewers have really read it
21:57:22 <clayg> joeljwright: i know initially you guys were debating structure - but I *love* how things ended up layed out
21:57:25 <joeljwright> so it'd be good to get eyes on it
21:57:35 <clayg> you people are smart - asettle is owed beers for sure
21:57:36 <joeljwright> clayg: thanks :)
21:57:40 <joeljwright> definitely
21:57:47 * asettle appears
21:57:48 <asettle> Ya what?
21:57:48 <notmyname> I'll definitely give this patch set a look
21:57:52 <joeljwright> once she finally explained what 'active voice' meant
21:57:53 <joeljwright> :)
21:58:06 <notmyname> asettle: the client docs patchset is looking great. thanks for your help
21:58:06 <joeljwright> notmyname: thanks
21:58:07 <acoles> asettle: you have highlights set for "beer"? ;p
21:58:12 <notmyname> lol
21:58:14 <asettle> acoles: you know it.
21:58:19 <asettle> I appear \o/
21:58:19 <clayg> rofl
21:58:24 <asettle> I'm a lurker. Good meeting guys.
21:59:00 <notmyname> joeljwright: anything else on these docs?
21:59:05 <joeljwright> I would like help with 'interesting' examples for the connection api if possible
21:59:16 <notmyname> ok
21:59:20 <joeljwright> if not interesting, then at least useful
21:59:33 <notmyname> which file is that in?
21:59:33 <tdasilva> acoles: foster
21:59:34 <joeljwright> but that's it
21:59:52 <timburke> joeljwright: they can get even more interesting after https://review.openstack.org/#/c/298968/ lands :)
21:59:52 <joeljwright> client-api.rst
21:59:53 <patchbot> timburke: patch 298968 - python-swiftclient - Adding keystoneauth sessions support
22:00:11 <notmyname> joeljwright: ok
22:00:14 <joeljwright> yeah, that will be interesting
22:00:17 <notmyname> bah, look at the time
22:00:22 <notmyname> #topic other
22:00:38 <notmyname> I'll be in san antonio early next week with the intel/rackspace people
22:00:55 <notmyname> flying out next wednesday, so if travel plans conflict with the meeting, I'll update. for now, the meeting is on
22:01:02 <notmyname> s/out/home/
22:01:23 <notmyname> thanks for coming, everyone. thanks for working on swift
22:01:26 <notmyname> #endmeeting