21:00:05 <notmyname> #startmeeting swift 21:00:05 <openstack> Meeting started Wed Dec 9 21:00:05 2015 UTC and is due to finish in 60 minutes. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:06 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:09 <openstack> The meeting name has been set to 'swift' 21:00:14 <notmyname> who's here for the swift team meeting? 21:00:20 <blmartin> o/ 21:00:21 <jrichli> hola 21:00:24 <gmmaha> o/ 21:00:25 <jlhinson> o/ 21:00:29 <sgundur1> hi 21:00:30 <cutforth> yo! 21:00:36 <ho> o/ 21:00:54 <tdasilva> hello 21:00:56 <Zyric1> hello 21:00:57 <mattoliverau> o/ 21:00:59 <kota_> o/ 21:00:59 <siva_krishnan> o/ 21:01:08 <wbhuber> p/ 21:01:13 <notmyname> welcome everyone :-) 21:01:20 <travisn> o/ 21:01:27 <acoles> hi 21:01:49 <notmyname> big group today, so let's get started 21:02:04 <notmyname> ah! eranrom is here. I'll come back to you soon since it's so late 21:02:14 <notmyname> first general info 21:02:14 <eranrom> :-) 21:02:44 <notmyname> you may have seen Zyric1 in the -swift channel. she's joining us for the next few months as part of the https://outreachy.gnome.org/ internship program 21:02:58 <notmyname> please welcome her and help her out 21:03:07 <tdasilva> cool, welcome Zyric1! 21:03:13 <jrichli> welcome, Zyric1! 21:03:23 <gmmaha> welcome Zyric1 21:03:25 <torgomatic> meeting meeting 21:03:32 <Zyric1> Thanks everyone! Really excited and happy to be working with you :) 21:03:40 <acoles> Zyric1: hi! 21:03:50 <notmyname> she'll be working on one of those things that came up in the ops feedback session in tokyo: providing a method to get a list of accounts found in a swift cluster 21:04:02 <mattoliverau> Zyric1: \o/ 21:04:09 <cutforth> welcome Zyric1 21:04:10 <sgundur1> Zyric1: hi 21:04:19 <blmartin> Zyric1: welcome! 21:04:21 <ho> welcome! 21:04:27 <jlhinson> hi! 21:04:31 <notmyname> Zyric1: I'm glad to have you with us making swift better :-) 21:04:32 <kota_> Zyric1: welcome 21:04:50 <pdardeau-> Zyric1: welcome! 21:04:59 <notmyname> here's our agenda for today (yes, there's a lot. we'll cover what we can) 21:05:03 <notmyname> #link https://wiki.openstack.org/wiki/Meetings/Swift 21:05:23 <notmyname> for the sake of timezones, I want to jump briefly to the end and ask eranrom a question 21:05:28 <notmyname> #topic container sync status update 21:05:34 <notmyname> eranrom: what's going on with container sync? 21:05:38 <notmyname> any blockers? what do you need? 21:05:47 <eranrom> notmyname: thanks! 21:06:17 <eranrom> Well, the first (out of three patches) has been reviewed by clayg and acoles getting +2/-1 21:06:30 <eranrom> Fixed all comments and awaiting to be re-review 21:06:37 <notmyname> ok 21:06:39 <acoles> eranrom: you are on my todo list 21:06:40 <notmyname> what's the link? 21:06:55 <eranrom> https://review.openstack.org/#/c/205803/ 21:07:02 <notmyname> #link https://review.openstack.org/#/c/205803/ 21:07:06 <notmyname> (for meetbot) 21:07:28 <eranrom> In fact IBM wants to beta this pretty much as soon as it gets upstream 21:07:33 <notmyname> awesome 21:07:44 <eranrom> so would be great to have it in 2.5.1 21:07:49 <notmyname> so acoles will rereview it 21:08:00 <eranrom> gr8 thanks!!! 21:08:02 <notmyname> and we'll need another core to look too 21:08:13 <aerwin3> o/ 21:08:43 <eranrom> you mean other then clayg? 21:08:57 <mattoliverau> I'm in Sydney office and thus meetings this week but will try and get eyes on it 21:09:07 <notmyname> clayg had a family emergency come up this week and will be out for a bit 21:09:14 <notmyname> thanks mattoliverau 21:09:23 <eranrom> oh. 21:09:33 <eranrom> hope all is/will be fine. 21:09:36 <jrichli> thoughts go to clayg 21:09:46 <eranrom> mattoliverau: Thanks! 21:09:51 <notmyname> eranrom: anything else you need there? 21:10:05 <eranrom> not now. Thanks! 21:10:10 <notmyname> great! 21:10:17 <notmyname> let's get it landed :-) 21:10:27 <eranrom> :-) 21:10:27 <notmyname> #topic patch (landable or nearly so) 21:11:01 <notmyname> hurricanerix: started the journey for patch 214206 to port functional tests to use testr (testrepository) 21:11:01 <patchbot> notmyname: https://review.openstack.org/#/c/214206/ - Modify functional tests to use testr 21:11:10 <notmyname> and we've had quite a few iterations there 21:11:37 <notmyname> the reason this is important is to better integrate with tempest, the openstack fucntional testing project 21:12:04 <notmyname> so it's a Big Deal for defcore and what can be called "swift" or not. (technicall, "openstack object storage", but whatever) 21:12:05 <hurricanerix> notmyname: hehe, i think it was more than a journey. =) 21:12:08 <notmyname> heh 21:12:15 <notmyname> but I think we're very nearly at the end! 21:12:23 <notmyname> thanks to mattoliverau and you and gmmaha! 21:12:50 <notmyname> so right now we've got a patch that passes everything and looks pretty good in our normal "run the tests" dev workflow 21:13:15 <notmyname> the only thing is that the in-process tests end up spewing all of the logging to stdout (or at least the tty) now 21:13:27 <notmyname> which makes running the tests that way pretty unusable 21:13:37 <notmyname> guess which way they are run in the gate? ;-) 21:14:00 <notmyname> so gmmaha has a follow-on patch (may be rolled in?) to suppress the logging 21:14:13 <notmyname> I think if that one looks good we should be able to land it 21:14:19 <gmmaha> notmyname: patch 255457 21:14:20 <patchbot> gmmaha: https://review.openstack.org/#/c/255457/ - Option to supress logs for in_process test 21:14:24 <notmyname> thanks 21:14:31 <blmartin> n 21:14:56 <notmyname> so take a look, if not for a review then at least to see what the functional test output looks like now. 21:15:00 <mattoliverau> I think if we go back to pipe through unittest we can hide the extra outout 21:15:05 <mattoliverau> Output even 21:15:23 <mattoliverau> *I think 21:15:54 <notmyname> that would be nice 21:16:04 <notmyname> ok, next nearly landable patch. 21:16:04 <acoles> notmyname: gate runs both forms, in-process and normal right? 21:16:21 <notmyname> acoles: perhaps in different jobs, yes 21:16:24 <notmyname> patch 241571 21:16:24 <patchbot> notmyname: https://review.openstack.org/#/c/241571/ - Put part-replicas where they go 21:16:29 <notmyname> this is the ring patch 21:16:48 <notmyname> actually, based on all the of the comments from everyone, I think this one is ready to land. today. 21:17:37 <notmyname> thanks to ho and acoles/HPE and kota_ and briancline and dfg/rax and torgomatic and of course clayg for running all those tests and rings through this patch 21:18:38 <notmyname> are there any objects to landing this patch now? 21:19:14 <notmyname> we've got a +2 from sam, a +1 from another core, 2 more +1s from other deployers. and tons of comments (positive) in the review comments 21:19:29 <acoles> notmyname: no objections from us 21:20:14 <acoles> notmyname: lorcan had +1 but looks like it got lost on an update 21:20:40 <mattoliverau> So acoles is your other +2 :) 21:20:45 <notmyname> :-) 21:21:02 <notmyname> I added mine too, with a +A 21:21:09 <acoles> phew 21:21:13 <notmyname> thanks everyone. this one is a pretty big deal :-) 21:21:23 <notmyname> acoles: lol 21:21:34 <notmyname> ok, moving on 21:21:41 <acoles> great work clayg, for the record 21:21:43 <notmyname> #topic patches, status updates 21:21:59 <notmyname> pyeclib version update patch has landed 21:22:26 <notmyname> next step is to raise the min version up. that just be much less painful 21:22:48 <notmyname> since all the packages (liberasurecode) are in the distros and on the jenkins slaves now 21:23:27 <notmyname> other one to look at is patch 206105 21:23:27 <patchbot> notmyname: https://review.openstack.org/#/c/206105/ - Use entrypoints for storage policy implementation ... 21:23:35 <notmyname> this one is getting good feedback 21:23:53 <notmyname> torgomatic and acoles make very good arguments :-) 21:24:28 <notmyname> this one is a small code change but bigger from a philosophy POV 21:24:30 <notmyname> sortof 21:24:35 <notmyname> so please take a look 21:24:55 <notmyname> any questions on it? 21:25:29 <notmyname> ok 21:25:37 <notmyname> #topic patches that need reviews 21:25:39 <notmyname> all of them. 21:25:42 <notmyname> ;-) 21:25:45 <notmyname> but... 21:25:45 <mattoliverau> Lol 21:26:01 <notmyname> patch 156293 is blocking crypto work (and is a good idea anyway) 21:26:02 <patchbot> notmyname: https://review.openstack.org/#/c/156293/ - Use correct regex for EC2 tempest gating (MERGED) 21:26:06 <notmyname> this is the copy middleware patch 21:26:30 <tdasilva> wrong patch number i think 21:26:43 <jrichli> patch 156923 21:26:43 <patchbot> jrichli: https://review.openstack.org/#/c/156923/ - Refactor server side copy as middleware 21:26:44 <notmyname> oops 21:26:47 <notmyname> thanks :-) 21:27:27 <notmyname> the biggest question I have is around the TODO in the commit message 21:27:49 <wbhuber> ho and i have been looking into it. i've hit a snag when trying to come up with a file that handles as a common helper function for both copy as middleware and versioned_write middleware. 21:27:50 <notmyname> anyone care to give an update on this patch? what does it need? is there more work to be done before reviews? 21:27:56 <notmyname> ok 21:28:12 <notmyname> wbhuber: how can we help? 21:28:22 <wbhuber> notmyname: still thinking of right questions to ask. =] 21:28:31 <ho> I would like to know background of place for the middleware in pipeline (before *lo) 21:28:32 <notmyname> heh ok 21:28:35 <acoles> doing the commit message todo would also fix a bug with SLO post i think 21:28:47 <notmyname> ah, good 21:29:06 <notmyname> yeah, there's several things going on there. COPY middleware, fast-POST, and SLO post stuff 21:29:11 <notmyname> all kinda tangled togetehr 21:29:22 <wbhuber> fast-POST -> SLO -> copy as middlware -> encryption? 21:29:37 <acoles> i think it would obviate need for patch https://review.openstack.org/248219 21:30:05 <notmyname> acoles: doing the todo in the copy middleware patch? 21:30:27 <tdasilva> yeah...I can work with wbhuber to make copy middleware happen 21:30:35 <notmyname> the copy middleware patch has bounced around several times? who's the current "owner"? wbhuber? ho? tdasilva? 21:30:44 <notmyname> tdasilva: awesome. thanks 21:30:47 <wbhuber> ppai is no longer working on it? 21:30:51 <acoles> notmyname: yes, but maybe there is a related issue around copying the internal manifest format? 21:31:03 <wbhuber> tdasilva: that'd be terrific. 21:31:03 * acoles needs to study it some more tbh 21:31:17 <tdasilva> acoles: that patch refers to copy hook, since that would be removed is the patch still valid? 21:31:28 <jrichli> thanks, tdasilva! 21:31:58 <tdasilva> jrichli, wbhuber: sorry my time has been bouncing around and difficult to predict 21:32:16 <jrichli> tdasilva: no worries :-) 21:32:27 <wbhuber> tdasilva: understood. 21:32:33 <acoles> tdasilva: sorry, let me be clearer, bug 1260446 is what i hope will be fixed by COPY moving to left of SLO 21:32:33 <openstack> bug 1260446 in OpenStack Object Storage (swift) "when you copy a slo the destination doesn't get its content-length and etag set correctly in the container listings" [Undecided,In progress] https://launchpad.net/bugs/1260446 - Assigned to Alistair Coles (alistair-coles) 21:32:45 <tdasilva> acoles: oh yeah, ok 21:32:58 <acoles> patch 248219 was my attempt on master but its not pretty 21:32:59 <patchbot> acoles: https://review.openstack.org/#/c/248219/ - Fix listing of SLO manifest bytes after POST or COPY 21:34:17 <tdasilva> there's an issue with the original slo manifest getting modified on a PUT, so still need to think how to fix that, but we can have that conversation later 21:34:18 <notmyname> so what I'm hearing is wbhuber and tdasilva will get the patch into better shape, then ho and acoles will look at it? does that sound right? 21:34:28 <notmyname> the copy middleware patch 21:34:36 <notmyname> acoles: or do we need to get fast-post first? 21:34:43 <notmyname> would that make things easier? 21:36:23 <acoles> notmyname: copy does not depend on fast post. fixing the content length bug for poast-as-copy would mean i can test it in fast-post, so there's a weak dependency there 21:36:29 <acoles> post* 21:36:43 <tdasilva> sounds good 21:36:58 <notmyname> ok, great 21:37:00 <acoles> notmyname: so plan sounds ok 21:37:13 <notmyname> thanks 21:37:22 <notmyname> next. patch 202411 21:37:23 <patchbot> notmyname: https://review.openstack.org/#/c/202411/ - Add functional test for access control (RBAC) with... 21:37:29 <notmyname> this is ho's RBAC tests patch 21:37:52 <notmyname> it's long but formulaic for adding a bunch of new functional tests 21:38:28 <notmyname> ho: is there anything other than reviews you need on this one? 21:39:09 <ho> notmyname: i need performance improvement on this and i tried it so i would like to have re-review. 21:39:17 <notmyname> ok 21:39:40 <notmyname> next. patch 117710 for concurrent reads 21:39:40 <patchbot> notmyname: https://review.openstack.org/#/c/117710/ - Add concurrent reads option to proxy 21:39:49 <notmyname> definitely a performance improvement for swift 21:40:02 <notmyname> mattoliverau: your last comment there implies some further work or discovery 21:40:15 <notmyname> anything you need from the rest of us? 21:40:31 <notmyname> hoping mattoliverau is still here ;-) 21:41:05 <notmyname> ok, he said he might have to step out 21:41:22 <notmyname> last one I wanted to mention is much smaller in scope than all the rest! 21:41:25 <notmyname> so it's easy! 21:41:31 <notmyname> patch 252096 21:41:31 <patchbot> notmyname: https://review.openstack.org/#/c/252096/ - Allow smaller segments in static large objects 21:42:07 <notmyname> not that we have ranges in SLOs, the hard 1MB limit per segment doesn't make as much sense. torgomatic improved it to be a rate limit instead of a hard min segment size 21:42:21 <notmyname> it should improve the life of anyone using SLOs 21:42:35 <notmyname> and I think that's it for patches! 21:42:37 <notmyname> whew 21:42:47 <torgomatic> and hopefully keep operators' lives roughly the same difficulty 21:42:58 <notmyname> :-) 21:43:02 <notmyname> #topic next releases 21:43:22 <notmyname> swiftclient release has been in the works all week. AFAICT everything is now good from our side 21:43:35 <notmyname> it had to be 2.7.0 since it's the first at the start of the mitaka cycle 21:43:44 <notmyname> but otherwise, it's a very small release 21:43:59 <notmyname> the most significant thing being "last py26 supported release 21:44:14 <notmyname> so I hope to see that be tagged very soon 21:44:28 <notmyname> for swift, I'd like to do a release soon as well 21:44:47 <notmyname> the ring change will be there. hopefully the container sync improvements :-) 21:44:49 <mattoliverau> Sorry in commute, re: concurrent reads, yeah clays last change meant there were some greenlet craziness, I have a diff that fixes it but causes more issues. I'll need to work on it a little more 21:44:53 <notmyname> and all the other stuff we've been working on 21:45:11 <notmyname> mattoliverau: ok, thanks. can you set WIP until you have somethign you want us to look at? 21:45:21 <mattoliverau> Sure can! 21:45:29 <notmyname> thanks 21:45:42 <notmyname> any questions on releases? 21:46:06 <notmyname> ok, last up is general stuff going on 21:46:12 <notmyname> #topic other status updates 21:46:35 <notmyname> acoles: where are with with the EC improvement patches? I feel like we've dropped the ball 21:46:39 <notmyname> (my fault) 21:47:49 <notmyname> acoles: you an I can talk about that later and come up with a plan to land some patches, if you want 21:47:53 <acoles> notmyname: no ball dropped. so there are three that depend on https://review.openstack.org/231121 which clayg and peluse have reviewed 21:48:14 <acoles> and imho its in good shape i.e. patch 231121 21:48:15 <patchbot> acoles: https://review.openstack.org/#/c/231121/ - Make ECDiskFile report all fragments found on disk 21:48:21 <notmyname> ok 21:48:24 <notmyname> good 21:48:43 <kota_> will review 21:48:47 <notmyname> thanks kota_ 21:48:52 <acoles> the the others are linked as dependent patches from there, and they bring the real value 21:48:55 <kota_> because my patch depends on that 21:49:13 <acoles> 231121 is just getting supporting things sorted in diskfile 21:49:20 <notmyname> ok 21:49:21 <acoles> thanks kota_ ! 21:49:32 <notmyname> mattoliverau: anything to report on container sharding? 21:49:55 <mattoliverau> Yeah, were making good progress 21:50:18 <mattoliverau> IBM have given a few bodies to help test and do some dev work 21:50:32 <notmyname> great! 21:51:08 <mattoliverau> I'm thinking we might be getting to a point where we might need to think about making a feature brach so all the people can more easily work on it upstream 21:51:09 <notmyname> mattoliverau: what do you need to make progress there? any other eyes or hands? just time to do the typing and thinking? 21:51:17 <notmyname> mattoliverau: ah? 21:51:43 <mattoliverau> But really ATM, I think it's more thinking and working time 21:51:47 <notmyname> ok 21:51:51 <notmyname> thanks for the update 21:52:01 <notmyname> we should talk more later about a feature branch or not 21:52:09 <notmyname> jrichli: any updates for us on crypto work? 21:52:28 <mattoliverau> Sure, we might want to wait until IBM testing and benching comes back 21:52:33 <jrichli> just chugging along ... most of the trello cards are assigned. just need to finish them 21:52:46 <notmyname> #link https://trello.com/b/63l5zQhq/swift-encryption 21:52:57 <notmyname> jrichli: and you need copy middleware ;-) 21:53:05 <jrichli> Yes :-) 21:53:08 <tdasilva> heh 21:53:11 <acoles> yes! 21:53:22 <notmyname> ok :-) 21:53:31 <tdasilva> got the msg 21:53:37 <notmyname> wow. we got through all the things on the agenda! 21:53:43 <notmyname> #topic hackathon 21:53:44 <acoles> tdasilva: no pressure there :) 21:53:49 <notmyname> so the next hackathon 21:54:12 <kota_> yey 21:54:20 <notmyname> acoles and HPE are hosting in bristol (UK) in early february next year 21:54:29 <jrichli> YAY! 21:54:29 <tdasilva> early? 21:54:43 <notmyname> tdasilva: oh. oops 21:54:46 <notmyname> late february! 21:55:03 <notmyname> he and I are working on logistics, and we'll be getting that available asap 21:55:29 <notmyname> very late february 21:55:34 <notmyname> week of feb 28 21:55:35 <tdasilva> ;) 21:55:41 <notmyname> practically early march 21:55:58 <acoles> almost summer in fact 21:56:04 <notmyname> lol 21:56:08 <acoles> http://tinyurl.com/npssaf4 21:56:08 <ho> lol 21:56:08 <kota_> lol 21:56:39 <notmyname> acoles: I expect good conversation around a table in a warm pub with dark beer! 21:56:53 <wbhuber> acoles: will we have elvenses? =] 21:56:55 <eranrom> I will drink for that 21:57:01 <tdasilva> shouldn't the hackathon be hosted at a pub? 21:57:02 <acoles> you mean a dark pub with warm beer surely? 21:57:14 <eranrom> for that too 21:57:18 <jrichli> lol 21:57:37 <notmyname> anything else to bring up in the meeting this week? 21:57:47 <tdasilva> just a quick question on the entrypoints patch: eikke mentioned on a comment that there's still work to do, should it be marked as WIP and do we know if he is still working on it? 21:58:33 <notmyname> tdasilva: he's definitely still working on it and interested in it 21:59:15 <acoles> can I just mention bug 1487791 - there's a discussion to be had (on launchpad) about what the right fix is for that, I'd appreciate others' opinions 21:59:15 <openstack> bug 1487791 in OpenStack Object Storage (swift) "POST to DLO squashes data without fast-POST" [Undecided,New] https://launchpad.net/bugs/1487791 21:59:38 <notmyname> acoles: right. you had a good comment today. I want to respond this afternoon 21:59:38 <acoles> on launchpad, not here. 21:59:45 <acoles> notmyname: thanks 21:59:50 <notmyname> we're out of time here ;-) 22:00:02 <notmyname> thanks for coming today. thanks for working on swift! 22:00:07 <notmyname> #endmeeting