21:00:51 <notmyname> #startmeeting swift
21:00:52 <openstack> Meeting started Wed Sep 28 21:00:51 2016 UTC and is due to finish in 60 minutes.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:53 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:57 <openstack> The meeting name has been set to 'swift'
21:01:03 <notmyname> who's here for the swift team meeting?
21:01:06 <mattoliverau> o/
21:01:11 <kota_> hello
21:01:12 <acoles> here
21:01:13 <kei_yama> o/
21:01:14 <cutforth> buenos dias
21:01:15 <bkeller`> o/
21:01:16 <sgundur> hi
21:01:19 <dmorita> hi
21:01:20 <nadeem> hi
21:01:22 <hosanai> o/
21:01:23 <mathiasb> o/
21:01:46 <tdasilva> hi
21:01:54 <torgomatic> 
21:02:00 <jrichli> h
21:02:04 <hurricanerix> o/
21:02:13 <notmyname> welcome, everyone
21:02:19 <notmyname> agenda this week is at
21:02:21 <notmyname> #link https://wiki.openstack.org/wiki/Meetings/Swift
21:02:42 <notmyname> let's dive in
21:02:51 <notmyname> #topic swift release for newton
21:03:12 <notmyname> we've got 2.10.0 tagged and released
21:03:46 <notmyname> there were a couple of other things that would have been really nice to have seen land before newton, so I asked the release team to not yet make the stable branch
21:03:52 <notmyname> however, we're out of time for that
21:04:14 <notmyname> so right after the meeting, I'll ask the release team to go ahead and create the stable branch from the 2.10.0 tag
21:04:34 <notmyname> rather than cutting a new 2.11 or 2.10.1 release with a little more stuff in it
21:04:42 <notmyname> make sense? any questions on that?
21:05:05 <mattoliverau> cool
21:05:28 <acoles> makes sense
21:05:35 <jrichli> yep
21:05:41 <notmyname> great
21:05:43 <notmyname> #topic barcelona summit
21:05:50 <notmyname> the summit is coming up quickly
21:06:09 <notmyname> I've created an etherpad where we can collect topic ideas
21:06:11 <notmyname> #link https://etherpad.openstack.org/p/ocata_swift_summit_topics
21:06:26 <notmyname> please add your topic/idea/subject as soon as possible
21:06:50 <notmyname> like recent summits, I want to schedule these in blocks of topics
21:07:17 <notmyname> I know there's some great stuff in progress and some other ideas not fully developed yet that we'll need to cover
21:07:52 <notmyname> any questions on the summit logistics or barcelona or anything else?
21:08:05 <acoles> notmyname: would it be useful for people to list any summit presentations they have to help plan to avoid conflicts?
21:08:09 <notmyname> yes
21:08:19 <acoles> on same etherpad?
21:08:54 <notmyname> yep. I just added a section to put that
21:09:08 <acoles> i see it :)
21:09:08 <notmyname> very good idea. thanks
21:09:44 <notmyname> clayg: nadeem: here yet?
21:09:57 <nadeem> yep
21:10:02 <clayg> yes
21:10:17 <notmyname> great
21:10:38 <notmyname> any other questions/comments about the summit before we move on to a hummingbird/golang topic?
21:11:10 <notmyname> ok
21:11:12 <notmyname> #topic update on hummingbird integration
21:11:20 <notmyname> #link https://etherpad.openstack.org/p/humming-replication-server
21:11:26 <notmyname> clayg: you're up with nadeem
21:11:40 <clayg> well - I mainly just wanted to broadcast a couple of updates
21:12:01 <clayg> nadeem: has been putting the rubber to the road so to speak and few things didn't work out quite like we "planned" back in austin
21:12:37 <clayg> 1) it's probably not going to be useful practicle to try to make a feature/repconn branch and select just bits with from feature/hummingbird while that branch continues to live
21:13:13 <clayg> ... instead dfg/redbo/nadeem et. al. are going to try to pair down feature/hummingbird to just the bits we want - then we'll rebase feature/hummingbird as a series of merged to master and bring over everything
21:13:38 <nadeem> agree
21:13:41 <clayg> unifiying dev on master - and additional experiment stuff like container servers and proxy work will happen wherever (more feature branches github w/e)
21:14:00 <clayg> 2) the object-server will be included as part of the merge
21:14:41 <clayg> so this means basically we'll have two object-servers - which we wanted to avoid - but trying to seperate the code from the object engine knowning that we *want* to add it back in is not practile
21:14:49 <clayg> practical?
21:15:03 <clayg> anyway - so we wanted to avoid having two object servers on master - but we not going to
21:15:37 <clayg> instead we're going to have *the* object server (the python one) and the hummingbird object server - which will be behind some kind of "do not use" door until we finish parity work and kill the python object server
21:16:17 <clayg> so... that was the "broadcast" - people can still complain - nothing has landed yet - we don't even know if *any* of it is *acctually* going to go on master
21:16:31 <notmyname> well, to be fair, we were probably going to have some small bit of overlap where we'd have both object servers. this just makes that overlap longer than initially expected
21:16:42 <clayg> but for the tim being the plan of "pulling stuff out of feature/hummingbird" only includes the parts we *don't* have a clear plan to merge
21:16:54 <notmyname> +1 good plan IMO
21:17:31 <clayg> stuff that we definately want to make feature complete and replace the python bits with hummingbird implementations are part of "the integration work" (i.e. object-replicator, object-server, and various minimal set of cli tools needed to support new replicator features)
21:17:36 <clayg> that's the upate
21:17:39 <notmyname> does that make sense to everyone?
21:17:50 <acoles> yes. thanks for the update
21:18:02 <clayg> again - feel free to raise concerns and ask questions complain - tell us how you would do it it different - blah blah balh
21:18:12 <clayg> either now or async - i just didn't want anyone not knowing what's going down
21:18:13 * acoles notes that he is not everyone
21:18:18 <timburke> while we have that overlap, do we expect that operators will only ever run the python object server?
21:18:39 <notmyname> timburke: that's what I expect right now
21:18:50 <mattoliverau> I know the line where to split object server from replicator has been really difficult, so I guess it makes sense. Are we going to leave the object server out of swift-init or something?
21:19:04 <ahale> presumably not _all_ operators ? :)
21:19:14 <clayg> timburke: yes, we have to code in tree - but as far as everyone who depends on swift should care there is only the one - and there will only ever *be* one - but two pieices of code will exist
21:19:18 <notmyname> ahale: you're special ;-)
21:19:29 <acoles> timburke: IIRC from a previous conversation the go object server would not be runnable via swift-init
21:19:30 <notmyname> mattoliverau: yeah. that's the plan. that's the gate behind which it hides
21:19:32 <ahale> thx :)
21:19:40 <notmyname> ahale: what acoles just said
21:19:49 <notmyname> ahale: which AFAIK isn't how you run it anyway
21:20:05 <clayg> ahale: yeah I mean that's acctually the main point - most of the decision to have feature/hummingbird go away is to support cloud files not having to track patches against multiple implementation branches
21:20:47 <clayg> and hopefully going up until and after integration a lot of the commits on the object-server will be focused on preparing to remove the python object server
21:20:50 <torgomatic> I thought the replicator was going to have its own replication-protocol socket that it listened on, so then replication wouldn't go through the object server at all; am I just making stuff up?
21:20:59 <ahale> cool - sorry, i was making a kind of bad joke but its good to have the reasoning out there too
21:21:15 <clayg> torgomatic: so that part isn't so much part of the "update" - but that design is being debated
21:21:38 <torgomatic> clayg: oh, okay. I thought it was a done deal.
21:21:39 <clayg> torgomatic: someone had that idea at one time - nadeem even wrote that implementation and if you check out feature/hummingbird right now you get that
21:21:42 <clayg> it's not great
21:21:56 <clayg> it also has not a lot to do with why the object-server is coming into master
21:22:22 <notmyname> all that (in great detail) is in the etherpad, right?
21:22:33 <clayg> "getting rid of feature/hummingbird" instead means taking everything cloud files needs for production
21:22:33 <nadeem> yep
21:22:47 <acoles> I guess it might be good to document that we would not be intending to support the golang object server upstream until it is "officially" available?
21:22:48 <notmyname> but there still isn't a final decision on where to do the object replicator listener
21:22:55 <clayg> like it doesn't really make sense to have to pacthc the object engine on maser so you can fix a bug in the object-server that's in a different branch
21:23:37 <clayg> acoles: we might have the code in there but disabled with a build flag - that's TBD - the point is that fixes to the object-server will happen in the normal review process to master
21:24:18 <acoles> clayg: yep
21:24:39 <tdasilva> what is the big work left to do in hummingbird object server to have parity with python object server? EC support?
21:24:47 <clayg> acoles: but yah - docs will be part of the merge - it should be clear from multiple documents what is supported expected and how to configure it / upgrade - everyhting
21:25:01 <clayg> if it's not crystal by the time we're trying to merge it that's a big red flag we're not doig it right
21:25:27 <clayg> tdasilva: yes all the stupid mime documnt handling is the big thing - needed for ec and encryption
21:25:44 <notmyname> did POST get in yet?
21:25:47 <clayg> there's some ec specific data layer object enngine stuff that's needed too obvs
21:26:12 <acoles> optimistic GETs I assume
21:26:14 <clayg> ... and POST - specifically fast-POST - i have some questions still about how the object engine will handle multipart timestamps
21:26:17 <nadeem> notmyname not yet
21:26:24 <clayg> acoles: that fucking api :)
21:26:39 <acoles> clayg: it just got another bug fixed ! :P
21:26:53 <acoles> clayg: you will learn to love it. you WILL. :)
21:26:56 <notmyname> tdasilva: in summary: more than a little bit ;-)
21:27:04 <clayg> hehe - probably
21:27:19 <tdasilva> hehe
21:27:24 <tdasilva> notmyname, clayg thanks
21:27:37 <notmyname> anything else to go over on hummingbird this week?
21:27:43 <clayg> ok so that's the update bit - there a design bit as well - but that's mostly on the etherpad and redbo & dfg will probably tell us what to do if we just let them think about it a few more days
21:27:50 <torgomatic> for reference, if anyone's got a better thing to do than that MIME garbage, please speak up
21:28:02 <torgomatic> I'll even help write the code
21:28:03 <tdasilva> notmyname: any update on the 'when can we merge' side of things?
21:28:23 <notmyname> tdasilva: not yet. some of that depends on the TC election and conversations in barcelona
21:28:26 <clayg> torgomatic: sorry - i wasn'tr trying to pick on anything :\
21:28:28 <torgomatic> especially if it makes the go object server easier to write
21:28:37 <torgomatic> clayg: it's deserving of being picked on :)
21:29:31 <notmyname> ok, if nothing else, then let's move on
21:29:43 <torgomatic> notmyname: my sister works on government-funded research, and I used to think it was terrible that there'd be all this uncertainty around where things were going until after Congressional elections
21:29:59 <torgomatic> ...well, here I am
21:30:12 <nadeem> :)
21:30:21 <timburke> torgomatic: there's a reason my wife is looking into industry positions...
21:30:30 <notmyname> #topic defcore updates
21:30:51 <notmyname> the defcore (now "interop") team is updating their definitions of the differnet openstack projects
21:31:06 <notmyname> as a quick overview, this is something they do a couple times a year
21:31:32 <notmyname> defcore has 2 things: capabilities (ie user-exposed features, tested via tempest) and code
21:31:45 <notmyname> so the code part is easy. we point to our repo and say "that"
21:31:56 <notmyname> but the capabilities is what they're updating now
21:32:20 <notmyname> so the idea is that anyone running something called "swift" has access to the same features and running the same code from upstream
21:32:29 <notmyname> really helps build the overall openstack ecosystem
21:32:48 <notmyname> so, that being said, we have an opportunity to add some capabilities into the definition
21:33:07 <notmyname> in order to add something, it's gotta have tests in tempest and it has to be something that was in liberty
21:33:14 <notmyname> the liberty release (or sooner)
21:33:32 <notmyname> the current swift definition is very sparse
21:33:53 <notmyname> here's what I proposed adding to the definition (stuff I've seen that has tests in tempest already)
21:34:27 <notmyname> bulk support, /info, expiring objects, staticweb, *LOs, conditional requests (if-match)
21:34:47 <notmyname> timburke has convinced me we shouldn't add "versioning" at this time
21:34:55 <timburke> next year
21:35:04 <clayg> timburke: sand baggin'
21:35:23 <notmyname> but the others are things that have been available for a long time and shoudl reasonably be expected to be in every swift deployment
21:35:47 <tdasilva> notmyname: something is not clear, what can be added to the definition, only stuff that is in tempest already, or can we add new tests to tempest now and add that to the defcore definition?
21:35:51 <notmyname> oh, if we add stuff now, it goes into "advisory" and then after (I think) 1 year it becomes required
21:35:56 <clayg> i'm more sure about *LO's, and if-match and expiring objects - than bulk support, and staticweb
21:36:24 <notmyname> tdasilva: yes. except we also have to fit into the bi-annual defcore/BoD approval timeframe
21:36:31 <notmyname> clayg: exactly my thoughts
21:36:38 <clayg> /info is not so well defined - is it really a "feature"
21:37:02 <clayg> discoverability is a feature... but we sort have a janky implementation - "here's some json; good luck"
21:37:11 <notmyname> there's a tempest test for it. basically that it returns something instead of an error. it doesn't parse the json
21:37:16 <acoles> notmyname: is it ok to add capabilities that are optional in a deployment, like SLO?
21:37:29 <notmyname> acoles: well, that's what I was getting aroudn to typign up
21:37:43 <clayg> acoles: I think adding them to DefCore means everyone everywhere *has* to deploy them?
21:37:53 <notmyname> there's stuff like SLO bulk and staticweb that are "optional". in that you can remove them from the pipeline
21:37:54 <clayg> or... "interop"
21:38:01 <notmyname> clayg: to conform to (tm) yes, that's it
21:38:17 <notmyname> however, these things are in the recommended pipeline already
21:38:19 <mugsie_> If something is in interop , it has to be deployed, yes.
21:38:28 <acoles> oic
21:38:29 <timburke> clayg: "here's some json; good luck" -- that sounds like some other entire *services*...
21:38:38 <notmyname> mugsie_: well, if it wants to conform ;-)
21:38:42 <clayg> yeah like if you wannt use swift because you're awesome you can put whatever middleware you want in there - you're aweomse - swift is awesome - everyhting is awesome
21:38:44 <mugsie_> Well, yeah
21:38:53 <mugsie_> :)
21:39:02 <clayg> you wanna call it openstack to someone branding marketing - whoa... hold the phone - not everyone is awseome where buddy
21:39:16 <notmyname> clayg: yes, exactly
21:39:39 <pdardeau> clayg: lol
21:39:42 <notmyname> and if we add these things, we'll probably want to work on the pipeline (which is a mess anyway IMO) to make sure some of these are there always
21:40:04 <clayg> notmyname: that's interesting collateratal damange :\
21:40:05 <notmyname> so adding these is ultimately about trademark compliance for things that are allowed to be called "openstack object storage"
21:40:14 <acoles> notmyname: that was what I was wondering, if we need to auto insert SLO like DLO
21:40:23 <notmyname> clayg: you say damage, I say "fixing" ;-)
21:41:06 <clayg> and we all believe that a thing called "openstack object storage" is important to client application developers - and client application developers are important to people using swift - which is tanginially related to my kids super
21:41:12 <clayg> so... interop is great
21:41:20 <notmyname> I mean, I think the pipeline mess needs to be fixed anyway, regarless of defcore. it's too long and confusing, even for us
21:41:24 <notmyname> yay interop!
21:41:49 <clayg> notmyname: well I hadn't consider that ti would mean you "can't turn it off" - more like "the path of least resistence is conforming to the interop definition"
21:41:52 <tdasilva> funny thing is that versioning is auto-inserted already..
21:41:57 <clayg> but maybe i misread what you said
21:42:04 <timburke> tdasilva: but not necessarily enabled
21:42:19 <tdasilva> true true, especially for history mode
21:42:38 <notmyname> clayg: nah, I was saying it's something we could do that's related and (IMO) makes things better. I'm not convinced it would be required b/c of something in defcore/interop
21:43:01 <clayg> timburke: oh man - i need to add history mode checkbox to the "package-next-swift" story
21:43:03 <notmyname> ok, so here's the question: of the list of stuff I mentioned, anything on the list that should NOT be added into the list?
21:43:25 <clayg> notmyname: well what about not adding bulk and staticweb?
21:43:40 <clayg> notmyname: i'm not sure either of those features really reached critical mass
21:43:45 <timburke> tdasilva: and history mode is the main reason i feel like that one ought to be revisited later -- no sense confusing people by saying "here's this thing that must be there!" and "but wait, here's this new way of using it that may or may not be available!" at the same time
21:43:46 <notmyname> also note that I'll be proposing the list of tested capabilities. it's the defcore team that will score them and then chose if they are required or not
21:44:15 <notmyname> tdasilva: yeah, what timburke said. that's what convinced me
21:44:29 <clayg> notmyname: can't we just do *LO, expiring objects and conditional requests for now?
21:44:32 <timburke> clayg: you get it for free, as long as we explicitly put it in the pipeline!
21:44:50 <notmyname> clayg: sure. those are very non controversial
21:44:50 <tdasilva> does *LO also include sloranges?
21:44:58 <clayg> tdasilva: it should!
21:45:08 <tdasilva> cool
21:45:11 <notmyname> tdasilva: not yet, because of tests in tempest. it depends on what's tested
21:45:22 <acoles> didn't notmyname say a feature had to have been in liberty?
21:45:23 <tdasilva> oh right
21:45:23 <notmyname> but it's easy to add tests to an existing capability
21:45:59 <mattoliverau> would server side copy be a thing, now that its a middleware.. or are there no copy tests in tempest
21:46:03 <clayg> notmyname: like i don't mind having tests for staticweb - i'm just - I think it's a lot like stack versions
21:46:19 <notmyname> clayg: if you think swift dev progress is slow, it's nothing compared to defcore. right now, this is for stuff that would first be implemented in mid 2017 and then made required in 2018. so that's why I'd push for bulk and staticweb now
21:46:35 <clayg> idk
21:46:53 <notmyname> mattoliverau: implementation doesn't matter. the capability is tested and already there, IIRC
21:47:32 <tdasilva> is 'swift policies' a feature? like the fact you can specify a header with a policy???
21:47:48 <notmyname> tdasilva: yeah, but not a user-facing one, so won't be in defcore
21:47:56 <tdasilva> oic
21:48:01 <notmyname> oh. copy is tested. but not in the standard
21:48:07 <notmyname> current definition is https://github.com/openstack/defcore/blob/master/2016.08.json#L89
21:48:10 <clayg> idk, they're in /info and you can create a container with different policies?
21:48:11 <timburke> i should probably go write some tempest tests for history mode & slorange....
21:48:45 <notmyname> bah. object-copy is right there at the top
21:48:51 <tdasilva> what's that deprecated test?
21:48:55 <clayg> we got rid of object access!?
21:49:04 <tdasilva> objectstore-object-access??
21:49:05 <notmyname> can't do that any more ;-)
21:49:11 <notmyname> write-only
21:49:14 <clayg> notmyname: nice
21:49:19 <clayg> that will simplify a few things
21:49:29 <notmyname> no, it was something that was related to AUTH IIRC
21:49:30 <acoles> post is not there
21:49:31 <tdasilva> that's torgomatic's s4 right?
21:49:41 <clayg> notmyname: i ... do not have a good feeling for what we're talking about anymore
21:49:43 <notmyname> I can check
21:49:51 <clayg> you said you're going to add some stuff to this thing I know nothing about
21:50:01 <clayg> i feel like - i should probably know more about it
21:50:18 <clayg> ... but even *not* knowing anything about it - i feel like I have opinions about how it should work anyway!!!
21:50:25 <clayg> ... so - i'm not helping
21:50:29 <notmyname> heh
21:50:37 <tdasilva> should expiration be there?
21:50:50 <tdasilva> it's user facing, in a way...
21:50:51 <notmyname> tdasilva: yeah, it's one I want to propose
21:50:57 <tdasilva> ok, sorry
21:51:13 <notmyname> tdasilva: there's tempest tests for it, but I haven't looked at the test implementation
21:51:30 <tdasilva> notmyname: i was just looking at that today
21:51:32 <tdasilva> it's fine
21:51:35 <notmyname> cool
21:51:40 <notmyname> so clayg doesn't like making bulk and staticweb required. any other comments on that?
21:52:08 <clayg> notmyname: and now you've got timburke running off to write tempest tests :'(
21:52:10 <timburke> i wonder what the difference is between objectstore-object-put and objectstore-object-upload...
21:52:16 <tdasilva> tempurl?
21:52:32 <clayg> and we don't even know what's covered currently :'(
21:52:35 <notmyname> tdasilva: there already
21:52:50 <timburke> clayg: idk about "running"...
21:53:07 <notmyname> well, assuming the names of the capabilities are close to accurate, objectstore-temp-url-get seems to cover that
21:53:16 <clayg> this is notmyname https://s-media-cache-ak0.pinimg.com/originals/10/1b/21/101b2173a16abc64aa3dd08b52ea55aa.jpg
21:53:19 <timburke> i *still* haven't done much about adding Session support to swiftclient, so...
21:53:32 <notmyname> LOL
21:54:03 <notmyname> and now did client sessions support come from?! get back intot he corral. we're going *that* way /me points
21:54:40 <acoles> notmyname: object POST has tempest tests but is not in the defcore json thing AFAICT
21:55:01 <notmyname> acoles: ack. would be a good thing to cover too
21:55:36 <clayg> acoles: is there a minimum request time requirement on POST to a 2GB object?
21:55:48 <notmyname> earlier today I told torgomatic "there's this thing I want to bring up in the meeting. I think it's probably a minor point, but I fear it might turn into this Big Thing"
21:55:56 <notmyname> guess what? turned into the Big Thing ;-)
21:56:07 <torgomatic> bikesheds expand to fill the space available ;)
21:56:45 <notmyname> ok, let's wrap up this topic: I'll write up something about the defcore stuff and ping people in the -swift channel
21:56:51 <notmyname> moving on
21:56:53 <timburke> torgomatic: but the internal space remains constant -- it's just all of the layers of paint ;-)
21:57:00 <acoles> clayg: idk
21:57:13 <notmyname> #topic stuff I wanted to spend time on but we're out of time for today
21:57:35 <notmyname> ok, I had an idea of something to do in this meeting, based on some conversations with some of you
21:57:46 <notmyname> I wanted to highlight 2 patches and 2 bugs to look at for this week
21:58:08 <notmyname> not to the exclusion of other stuff, but to shine a light on the top of the top priorities
21:58:23 <notmyname> for this week, i chose https://bugs.launchpad.net/swift/+bug/1328735 and https://bugs.launchpad.net/swift/+bug/1624088
21:58:25 <openstack> Launchpad bug 1328735 in OpenStack Object Storage (swift) "Object-updater gives up updating container with no success if all containers are placed at handoff" [High,In progress] - Assigned to Takashi Kajinami (kajinamit)
21:58:26 <openstack> Launchpad bug 1624088 in OpenStack Object Storage (swift) "EC missing durable can prevent reconstruction" [High,Confirmed]
21:58:31 <notmyname> both really need to be fixed ASAP
21:59:08 <notmyname> for (non bugfix-related) patches, I chose
21:59:09 <notmyname> https://review.openstack.org/#/c/210099/ -- process level concurrency for container sync
21:59:14 <notmyname> https://review.openstack.org/#/c/219165/ -- global EC work
21:59:30 <notmyname> so I want to be better about this meeting topic next week
21:59:44 <notmyname> and work with everyone on chosing the bugs and patches to highlight
22:00:07 <notmyname> but this week, take a look at these
22:00:13 <notmyname> ...and we're at time
22:00:18 <mattoliverau> I stated looking at patch 210099 last week, so I might continue looking and playing with it this week
22:00:29 <notmyname> thank you all for coming. thanks for working on swift!
22:00:31 <mattoliverau> when I have some free cycles
22:00:35 <acoles> I put up a fix for bug 1624088 but its not showing on launchpad, https://review.openstack.org/376630
22:00:36 <openstack> bug 1624088 in OpenStack Object Storage (swift) "EC missing durable can prevent reconstruction" [High,Confirmed] https://launchpad.net/bugs/1624088
22:00:36 <notmyname> #endmeeting