18:59:49 <notmyname> #startmeeting swift
18:59:50 <openstack> Meeting started Wed Feb 25 18:59:49 2015 UTC and is due to finish in 60 minutes.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:59:52 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:59:54 <openstack> The meeting name has been set to 'swift'
18:59:56 <notmyname> who's here for the swift meeting?
19:00:01 <peluse> howdy
19:00:05 <jrichli> here
19:00:14 <kota_> hi
19:00:16 <gvernik> hello
19:00:18 <torgomatic> 👋
19:00:40 <notmyname> torgomatic: you always intro yourself with an emoji
19:00:48 <mattoliverau> o/
19:00:58 <notmyname> mattoliverau: good morning
19:00:58 <tdasilva> hi
19:01:18 <notmyname> kota_: glad you're here. we were just looking at one of your patches
19:01:24 <cschwede> hello
19:01:32 <torgomatic> notmyname: 😃
19:01:34 <notmyname> cschwede: dinner was good?
19:01:37 <notmyname> #link https://wiki.openstack.org/wiki/Meetings/Swift
19:01:44 <cschwede> notmyname: yes, thx :)
19:02:06 <notmyname> ok, just a few things to cover this week. mostly a status check
19:02:27 <notmyname> first up, as promised last week, I'll be focusing on EC status
19:02:36 <notmyname> #topic EC status
19:02:46 <notmyname> GET patches have landed
19:02:49 <notmyname> yay
19:02:55 <notmyname> torgomatic: peluse: what's missing from them?
19:02:59 <peluse> write *and* read, wow
19:03:12 <peluse> all the partial PUT failure stuff on the GET side is missing
19:03:17 <peluse> but there's a trello card for that
19:03:26 <notmyname> gotcha
19:03:31 <peluse> tsg is, any minute, going to push the updated multiphase patch that has all of PUT complete
19:03:40 <notmyname> great
19:03:51 <peluse> except the correct ETAG/content-length in the DB
19:03:55 <notmyname> and peluse has the reconstructor patch
19:04:12 <peluse> that will be a new patch (item on trello) that thiago has agreed to knock out after alistair's override path clands
19:04:24 <notmyname> nice
19:04:27 <peluse> yes, on the reconstructor
19:04:40 <peluse> more progress on unit tests, etc. hwoever I think I'm going to take a day and break it up a bit
19:04:47 <notmyname> multiple patches?
19:04:50 <peluse> at least try to spearate the non-reconstructor diskfile chagnes out
19:04:53 <peluse> yes
19:04:59 <notmyname> ok
19:05:04 <peluse> essentially support for multiple fragment archive indexes
19:05:09 <notmyname> it does seem kinda big
19:05:11 * acoles apologises for late arrival
19:05:18 <notmyname> acoles: no worries
19:05:24 <peluse> because that will be needed by the remaining GET work which won't want to depend on my growing reconstructor patch
19:05:57 <peluse> so that will be the rest of my week... trying to carve that out and push an update to reconstructor as well
19:05:59 <notmyname> peluse: and so you expect that to be split up tomorrow some time?
19:06:05 <peluse> or Fri
19:06:05 <notmyname> ah. endof-week
19:06:10 <notmyname> ok
19:06:18 <peluse> maybe tomorrow :)
19:06:20 <notmyname> so looking at trello
19:06:22 <notmyname> #link https://trello.com/b/LlvIFIQs/swift-erasure-codes
19:06:35 <peluse> I will be setting up some time to online code review on Fri as well
19:06:52 <notmyname> we've got the GET handling you mentioned. the PUT to land. and the multi part reconstructor piece
19:06:59 <notmyname> and tests and docs
19:07:12 <peluse> yup
19:07:21 <notmyname> but looking at the functional pieces required to have a beta in kilo, that's what we've got
19:07:36 <notmyname> so, for everyone, is there something missing?
19:07:54 <peluse> I *think* the red items on trello make a good beta
19:08:28 <notmyname> we've talked a little about the docs parts. I think some docs are necessary (at least "how to set it up and use it"), but all those IMO aren't required
19:08:39 <notmyname> eg perf considerations or best prectices
19:08:40 <peluse> what's our target for landing the beta requirements on feature/ec?
19:08:43 <notmyname> I don't think we have any idea
19:08:53 <notmyname> (about the docs)
19:08:56 <peluse> :)
19:08:58 <peluse> I figured
19:09:01 <clayg> peluse: right before we propose to master
19:09:05 <notmyname> :-)
19:09:06 <peluse> ha!
19:09:07 <cschwede> maybe i missed that, but what if the reconciler has to move from 3-replicas to EC? is that/will that be supported?
19:09:24 <notmyname> cschwede: good questions
19:09:35 <notmyname> clayg: any ideas? just works? needs code?
19:09:43 <clayg> cschwede: we'll have to test it now that we have most of EC working - but internal client shouldn't really care about the source and destination policies
19:09:53 <peluse> yup, agreed
19:10:01 <clayg> GET -> PUT if it works for the client it should work for the reconciler
19:10:15 <peluse> back to the data question....
19:10:20 <peluse> notmyname?
19:10:23 <peluse> date, not data
19:10:30 <notmyname> ok, dates..
19:11:16 <notmyname> by next week we need to make a call. do we have enough velocity and small enough scope to finish what we'd be ok calling a beta by kilo
19:11:31 <clayg> peluse: i don't understand - we want everything on feature ec as soon as possible, but having docs on feature/ec two weeks before the two week merge period to master doesn't seem useful
19:11:32 <notmyname> we have about 4 weeks from now to start the merge to master
19:11:36 <notmyname> and then 2 weeks for the merge
19:11:39 <notmyname> then the RC
19:11:49 <peluse> looking at calendar, one sec
19:12:18 <peluse> so around Mar 25 start the merge to master you're thinking?
19:12:29 <clayg> not very far away
19:12:48 <peluse> nope, and I'll be drinking wine in Napa Mar 26/27 BTW
19:12:59 * notmyname pulls out notes
19:13:03 <peluse> 20 yr anniversary and all...
19:13:07 <notmyname> nice
19:13:23 <notmyname> mar 27
19:13:32 <peluse> but if that's the case the timing for our Mar 18 mini-F2F is just about perfect for identifying/wrapping loose ends, planning merge stratgy
19:13:37 <notmyname> estimating the RC on april 10
19:13:46 <mattoliverau> nothing say anniversary like some EC work :P
19:14:17 <peluse> yeah, that's go over well!  Hold on honey, gottta check zuul....
19:14:26 <mattoliverau> lol
19:15:10 <notmyname> so I think this last week (since last meeting), things have gone well. thank you to everyone for that
19:15:11 <peluse> OK, so anyway.  yeah I think we can make it.  Big thing for me is breaking the recon up a bit and spending some quality review time especially in some of the test code
19:15:55 <peluse> can we get another core set of eyes on acoles header patch?
19:16:00 <notmyname> for those looking to review something, https://review.openstack.org/#/c/158821/ by acoles is necessary for EC and crypto
19:16:03 <cschwede> peluse: yep me
19:16:11 <peluse> #link https://review.openstack.org/#/c/158821/
19:16:12 <peluse> thanks
19:16:23 <notmyname> peluse: and tsg will be pushing new patch sets asap?
19:16:28 <notmyname> that's the blocker there, I think
19:16:42 <peluse> he was aiming to have it done before this meeting.  since he's not here I assume he is frantically working on it :)
19:16:47 <notmyname> :-)
19:16:48 <notmyname> ok
19:16:48 <torgomatic> I looked at that patch some, but then I fell into the rabbit hole of failing probetests
19:16:54 <torgomatic> hope to get back to it today
19:16:56 <tdasilva> same here
19:17:04 <notmyname> ok, thanks for the status update
19:17:18 <acoles> peluse: let me know if there is anything more that i can usefully do building on that patch
19:17:19 <peluse> look for tsg's patch today, I'mm mention it in IRC when it comes out
19:17:29 <notmyname> kota_: FYI you might be interested in some of those probetest failures. see the playback in #openstack-swift for details
19:17:37 <notmyname> peluse: thanks
19:17:40 <notmyname> ok, moving on
19:17:44 <notmyname> #topic fsync on dirs
19:17:53 <notmyname> this is something we spent time on at the hackathon
19:17:56 <peluse> acoles, I think its good.  tdasilva already did some quick testing w/the DLO failure on EC GET and it worked (he can comment more) and will be finishing the EC specific stuff in that area after it lands
19:17:59 <notmyname> #link https://review.openstack.org/#/c/126923/
19:18:09 <notmyname> tdasilva: I think you wanted a status update on it
19:18:17 <notmyname> looks like ppai has pushed a new patch set
19:18:31 <tdasilva> acoles: I can provide a paste so you can see the patch I have for EC and how i'm building on top of your patch
19:18:43 <notmyname> and acoles has some updated benchmark numbers
19:18:44 <peluse> tdasilva, me too please
19:18:58 <tdasilva> notmyname: yeah...we left the hackathon trying to make a decision and waiting to hear from rackspace guys
19:18:59 <acoles> tdasilva: ok cool
19:19:04 * peluse apoligizes for not switching gears fat enough :)
19:19:05 <kota_> notmyname: ya, interesting (about probetests)
19:19:41 <acoles> re fsync on dirs i repeated my benchmark just fsyncing the leaf dir and it made no difference
19:19:45 <notmyname> tdasilva: ah, yes. we didn't hear anything from rackers yet. I'll add them to the review. but otherwise, Ithink it's ready to go
19:20:07 <notmyname> tdasilva: the latest patch set has the comments we made in person? eg the always on, no config?
19:20:12 <tdasilva> notmyname: ok, just wanted to make sure
19:20:44 <notmyname> tdasilva: IMO it should be reviewed, and I'll add dfg and glange and hurricanerix_ to it
19:20:49 <tdasilva> notmyname: need to double check, but I believe so
19:20:53 <notmyname> ok
19:21:17 <tdasilva> notmyname: yep, no option
19:21:18 <notmyname> I'll also add it to priority reviews
19:21:30 <notmyname> anythign else on that patch?
19:21:36 <notmyname> tdasilva: thanks for bringing it up?
19:21:41 <notmyname> s/?//
19:21:47 <tdasilva> notmyname: nothing from me
19:21:53 <notmyname> ok
19:21:57 <cutforth> sorry, i'm late folks
19:22:03 <notmyname> cutforth: welcome
19:22:07 <notmyname> #topic python-swiftclient
19:22:22 <notmyname> there's been a lot of stuff that's landed on swiftclient since the last release was tagged
19:22:37 <zaitcev> I think MD5 is the biggest, the rest is meh.
19:22:53 * cschwede agrees with zaitcev
19:22:53 <notmyname> for those of you who have been looking at it, what are your thoughts on tagging a new release?
19:23:24 <notmyname> zaitcev: you had the pseudo dirs crash fix. that's good
19:23:39 <acoles> notmyname: there's one of jeol's patches might be good to land
19:23:46 <notmyname> acoles: ok. got a link?
19:23:52 <acoles> min
19:24:09 <zaitcev> oh, that was a while ago. I also fixed Darrell's crash. But those were safe from regressions, relatively speaking.
19:24:33 <clayg> #link https://review.openstack.org/#/c/130339/
19:24:34 <acoles> sorry gerrit slow https://review.openstack.org/130339
19:24:43 <notmyname> clayg: acoles: thanks
19:24:44 <zaitcev> Right, the shuffle
19:24:47 <acoles> clayg: beat me :)
19:25:08 <acoles> been a while since i looked at it so need to revisit
19:25:28 <zaitcev> I think I have a question, if anyone understands. We used to shuffle everything. Now we shuffle inside a page (even if it's enabled). So... How big is the page?
19:25:33 <notmyname> ok. I'll track that one and look at a release after it lands
19:25:34 <notmyname> thanks
19:25:45 <clayg> zaitcev: ~10K
19:26:19 <zaitcev> That patch has surprisingly big benefit, if you run "time swift download blah" it shows hundreds of kilobytes less even for rather small downloads.
19:26:37 <notmyname> cool
19:26:54 <cschwede> wait a second, is the latest release from september 2014? ok, there are a lot of interesting commits then
19:27:27 <notmyname> :-)
19:27:55 <zaitcev> Probably might as well cut it before the shuffle patch, to be safe. I'll package it into Fedora right away.
19:28:07 <notmyname> before? why?
19:28:26 <zaitcev> well, to have a known-good release...
19:29:36 <notmyname> if it's reviewed and tested and landed, I'd call it known-good
19:30:05 <notmyname> unless there's a big gotcha in this one. anyone else have strong thoughts? acoles? clayg?
19:30:40 <notmyname> I see
19:30:41 <notmyname> ;-)
19:30:46 <acoles> iirc from when i last looked the only issue i had was how to verify the download sequencing
19:30:54 <acoles> in unit tests
19:31:29 <notmyname> this is the part of the meeting that you make fun of me about: "anything? anyone?" <wait 30 secs> "ok, moving on...."
19:32:30 <acoles> i'll look at it again tomorrow
19:32:32 <notmyname> zaitcev: if you've got some strong concerns, i'm fine with cutting one before. but if it's reviewed and tested, I'm fine with including it in a release
19:32:34 <notmyname> acoles: thanks
19:32:48 <acoles> notmyname: sorry, memory has faded a bit on that one
19:32:54 <notmyname> no worries
19:33:02 <zaitcev> notmyname: I don't have strong concerns.
19:33:06 <notmyname> ok
19:33:07 <notmyname> #topic open discussion
19:33:10 <notmyname> anything else?
19:33:20 <notmyname> that's all I've got for this week
19:33:47 <notmyname> <wait X seconds>
19:33:57 <acoles> tick - tock
19:34:02 <notmyname> :-)
19:34:02 <notmyname> ok
19:34:12 <notmyname> thanks everyone for attending and working on swift!
19:34:20 <notmyname> #endmeeting