21:00:05 #startmeeting swift 21:00:05 Meeting started Wed Dec 9 21:00:05 2015 UTC and is due to finish in 60 minutes. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:06 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:09 The meeting name has been set to 'swift' 21:00:14 who's here for the swift team meeting? 21:00:20 o/ 21:00:21 hola 21:00:24 o/ 21:00:25 o/ 21:00:29 hi 21:00:30 yo! 21:00:36 o/ 21:00:54 hello 21:00:56 hello 21:00:57 o/ 21:00:59 o/ 21:00:59 o/ 21:01:08 p/ 21:01:13 welcome everyone :-) 21:01:20 o/ 21:01:27 hi 21:01:49 big group today, so let's get started 21:02:04 ah! eranrom is here. I'll come back to you soon since it's so late 21:02:14 first general info 21:02:14 :-) 21:02:44 you may have seen Zyric1 in the -swift channel. she's joining us for the next few months as part of the https://outreachy.gnome.org/ internship program 21:02:58 please welcome her and help her out 21:03:07 cool, welcome Zyric1! 21:03:13 welcome, Zyric1! 21:03:23 welcome Zyric1 21:03:25 meeting meeting 21:03:32 Thanks everyone! Really excited and happy to be working with you :) 21:03:40 Zyric1: hi! 21:03:50 she'll be working on one of those things that came up in the ops feedback session in tokyo: providing a method to get a list of accounts found in a swift cluster 21:04:02 Zyric1: \o/ 21:04:09 welcome Zyric1 21:04:10 Zyric1: hi 21:04:19 Zyric1: welcome! 21:04:21 welcome! 21:04:27 hi! 21:04:31 Zyric1: I'm glad to have you with us making swift better :-) 21:04:32 Zyric1: welcome 21:04:50 Zyric1: welcome! 21:04:59 here's our agenda for today (yes, there's a lot. we'll cover what we can) 21:05:03 #link https://wiki.openstack.org/wiki/Meetings/Swift 21:05:23 for the sake of timezones, I want to jump briefly to the end and ask eranrom a question 21:05:28 #topic container sync status update 21:05:34 eranrom: what's going on with container sync? 21:05:38 any blockers? what do you need? 21:05:47 notmyname: thanks! 21:06:17 Well, the first (out of three patches) has been reviewed by clayg and acoles getting +2/-1 21:06:30 Fixed all comments and awaiting to be re-review 21:06:37 ok 21:06:39 eranrom: you are on my todo list 21:06:40 what's the link? 21:06:55 https://review.openstack.org/#/c/205803/ 21:07:02 #link https://review.openstack.org/#/c/205803/ 21:07:06 (for meetbot) 21:07:28 In fact IBM wants to beta this pretty much as soon as it gets upstream 21:07:33 awesome 21:07:44 so would be great to have it in 2.5.1 21:07:49 so acoles will rereview it 21:08:00 gr8 thanks!!! 21:08:02 and we'll need another core to look too 21:08:13 o/ 21:08:43 you mean other then clayg? 21:08:57 I'm in Sydney office and thus meetings this week but will try and get eyes on it 21:09:07 clayg had a family emergency come up this week and will be out for a bit 21:09:14 thanks mattoliverau 21:09:23 oh. 21:09:33 hope all is/will be fine. 21:09:36 thoughts go to clayg 21:09:46 mattoliverau: Thanks! 21:09:51 eranrom: anything else you need there? 21:10:05 not now. Thanks! 21:10:10 great! 21:10:17 let's get it landed :-) 21:10:27 :-) 21:10:27 #topic patch (landable or nearly so) 21:11:01 hurricanerix: started the journey for patch 214206 to port functional tests to use testr (testrepository) 21:11:01 notmyname: https://review.openstack.org/#/c/214206/ - Modify functional tests to use testr 21:11:10 and we've had quite a few iterations there 21:11:37 the reason this is important is to better integrate with tempest, the openstack fucntional testing project 21:12:04 so it's a Big Deal for defcore and what can be called "swift" or not. (technicall, "openstack object storage", but whatever) 21:12:05 notmyname: hehe, i think it was more than a journey. =) 21:12:08 heh 21:12:15 but I think we're very nearly at the end! 21:12:23 thanks to mattoliverau and you and gmmaha! 21:12:50 so right now we've got a patch that passes everything and looks pretty good in our normal "run the tests" dev workflow 21:13:15 the only thing is that the in-process tests end up spewing all of the logging to stdout (or at least the tty) now 21:13:27 which makes running the tests that way pretty unusable 21:13:37 guess which way they are run in the gate? ;-) 21:14:00 so gmmaha has a follow-on patch (may be rolled in?) to suppress the logging 21:14:13 I think if that one looks good we should be able to land it 21:14:19 notmyname: patch 255457 21:14:20 gmmaha: https://review.openstack.org/#/c/255457/ - Option to supress logs for in_process test 21:14:24 thanks 21:14:31 n 21:14:56 so take a look, if not for a review then at least to see what the functional test output looks like now. 21:15:00 I think if we go back to pipe through unittest we can hide the extra outout 21:15:05 Output even 21:15:23 *I think 21:15:54 that would be nice 21:16:04 ok, next nearly landable patch. 21:16:04 notmyname: gate runs both forms, in-process and normal right? 21:16:21 acoles: perhaps in different jobs, yes 21:16:24 patch 241571 21:16:24 notmyname: https://review.openstack.org/#/c/241571/ - Put part-replicas where they go 21:16:29 this is the ring patch 21:16:48 actually, based on all the of the comments from everyone, I think this one is ready to land. today. 21:17:37 thanks to ho and acoles/HPE and kota_ and briancline and dfg/rax and torgomatic and of course clayg for running all those tests and rings through this patch 21:18:38 are there any objects to landing this patch now? 21:19:14 we've got a +2 from sam, a +1 from another core, 2 more +1s from other deployers. and tons of comments (positive) in the review comments 21:19:29 notmyname: no objections from us 21:20:14 notmyname: lorcan had +1 but looks like it got lost on an update 21:20:40 So acoles is your other +2 :) 21:20:45 :-) 21:21:02 I added mine too, with a +A 21:21:09 phew 21:21:13 thanks everyone. this one is a pretty big deal :-) 21:21:23 acoles: lol 21:21:34 ok, moving on 21:21:41 great work clayg, for the record 21:21:43 #topic patches, status updates 21:21:59 pyeclib version update patch has landed 21:22:26 next step is to raise the min version up. that just be much less painful 21:22:48 since all the packages (liberasurecode) are in the distros and on the jenkins slaves now 21:23:27 other one to look at is patch 206105 21:23:27 notmyname: https://review.openstack.org/#/c/206105/ - Use entrypoints for storage policy implementation ... 21:23:35 this one is getting good feedback 21:23:53 torgomatic and acoles make very good arguments :-) 21:24:28 this one is a small code change but bigger from a philosophy POV 21:24:30 sortof 21:24:35 so please take a look 21:24:55 any questions on it? 21:25:29 ok 21:25:37 #topic patches that need reviews 21:25:39 all of them. 21:25:42 ;-) 21:25:45 but... 21:25:45 Lol 21:26:01 patch 156293 is blocking crypto work (and is a good idea anyway) 21:26:02 notmyname: https://review.openstack.org/#/c/156293/ - Use correct regex for EC2 tempest gating (MERGED) 21:26:06 this is the copy middleware patch 21:26:30 wrong patch number i think 21:26:43 patch 156923 21:26:43 jrichli: https://review.openstack.org/#/c/156923/ - Refactor server side copy as middleware 21:26:44 oops 21:26:47 thanks :-) 21:27:27 the biggest question I have is around the TODO in the commit message 21:27:49 ho and i have been looking into it. i've hit a snag when trying to come up with a file that handles as a common helper function for both copy as middleware and versioned_write middleware. 21:27:50 anyone care to give an update on this patch? what does it need? is there more work to be done before reviews? 21:27:56 ok 21:28:12 wbhuber: how can we help? 21:28:22 notmyname: still thinking of right questions to ask. =] 21:28:31 I would like to know background of place for the middleware in pipeline (before *lo) 21:28:32 heh ok 21:28:35 doing the commit message todo would also fix a bug with SLO post i think 21:28:47 ah, good 21:29:06 yeah, there's several things going on there. COPY middleware, fast-POST, and SLO post stuff 21:29:11 all kinda tangled togetehr 21:29:22 fast-POST -> SLO -> copy as middlware -> encryption? 21:29:37 i think it would obviate need for patch https://review.openstack.org/248219 21:30:05 acoles: doing the todo in the copy middleware patch? 21:30:27 yeah...I can work with wbhuber to make copy middleware happen 21:30:35 the copy middleware patch has bounced around several times? who's the current "owner"? wbhuber? ho? tdasilva? 21:30:44 tdasilva: awesome. thanks 21:30:47 ppai is no longer working on it? 21:30:51 notmyname: yes, but maybe there is a related issue around copying the internal manifest format? 21:31:03 tdasilva: that'd be terrific. 21:31:03 * acoles needs to study it some more tbh 21:31:17 acoles: that patch refers to copy hook, since that would be removed is the patch still valid? 21:31:28 thanks, tdasilva! 21:31:58 jrichli, wbhuber: sorry my time has been bouncing around and difficult to predict 21:32:16 tdasilva: no worries :-) 21:32:27 tdasilva: understood. 21:32:33 tdasilva: sorry, let me be clearer, bug 1260446 is what i hope will be fixed by COPY moving to left of SLO 21:32:33 bug 1260446 in OpenStack Object Storage (swift) "when you copy a slo the destination doesn't get its content-length and etag set correctly in the container listings" [Undecided,In progress] https://launchpad.net/bugs/1260446 - Assigned to Alistair Coles (alistair-coles) 21:32:45 acoles: oh yeah, ok 21:32:58 patch 248219 was my attempt on master but its not pretty 21:32:59 acoles: https://review.openstack.org/#/c/248219/ - Fix listing of SLO manifest bytes after POST or COPY 21:34:17 there's an issue with the original slo manifest getting modified on a PUT, so still need to think how to fix that, but we can have that conversation later 21:34:18 so what I'm hearing is wbhuber and tdasilva will get the patch into better shape, then ho and acoles will look at it? does that sound right? 21:34:28 the copy middleware patch 21:34:36 acoles: or do we need to get fast-post first? 21:34:43 would that make things easier? 21:36:23 notmyname: copy does not depend on fast post. fixing the content length bug for poast-as-copy would mean i can test it in fast-post, so there's a weak dependency there 21:36:29 post* 21:36:43 sounds good 21:36:58 ok, great 21:37:00 notmyname: so plan sounds ok 21:37:13 thanks 21:37:22 next. patch 202411 21:37:23 notmyname: https://review.openstack.org/#/c/202411/ - Add functional test for access control (RBAC) with... 21:37:29 this is ho's RBAC tests patch 21:37:52 it's long but formulaic for adding a bunch of new functional tests 21:38:28 ho: is there anything other than reviews you need on this one? 21:39:09 notmyname: i need performance improvement on this and i tried it so i would like to have re-review. 21:39:17 ok 21:39:40 next. patch 117710 for concurrent reads 21:39:40 notmyname: https://review.openstack.org/#/c/117710/ - Add concurrent reads option to proxy 21:39:49 definitely a performance improvement for swift 21:40:02 mattoliverau: your last comment there implies some further work or discovery 21:40:15 anything you need from the rest of us? 21:40:31 hoping mattoliverau is still here ;-) 21:41:05 ok, he said he might have to step out 21:41:22 last one I wanted to mention is much smaller in scope than all the rest! 21:41:25 so it's easy! 21:41:31 patch 252096 21:41:31 notmyname: https://review.openstack.org/#/c/252096/ - Allow smaller segments in static large objects 21:42:07 not that we have ranges in SLOs, the hard 1MB limit per segment doesn't make as much sense. torgomatic improved it to be a rate limit instead of a hard min segment size 21:42:21 it should improve the life of anyone using SLOs 21:42:35 and I think that's it for patches! 21:42:37 whew 21:42:47 and hopefully keep operators' lives roughly the same difficulty 21:42:58 :-) 21:43:02 #topic next releases 21:43:22 swiftclient release has been in the works all week. AFAICT everything is now good from our side 21:43:35 it had to be 2.7.0 since it's the first at the start of the mitaka cycle 21:43:44 but otherwise, it's a very small release 21:43:59 the most significant thing being "last py26 supported release 21:44:14 so I hope to see that be tagged very soon 21:44:28 for swift, I'd like to do a release soon as well 21:44:47 the ring change will be there. hopefully the container sync improvements :-) 21:44:49 Sorry in commute, re: concurrent reads, yeah clays last change meant there were some greenlet craziness, I have a diff that fixes it but causes more issues. I'll need to work on it a little more 21:44:53 and all the other stuff we've been working on 21:45:11 mattoliverau: ok, thanks. can you set WIP until you have somethign you want us to look at? 21:45:21 Sure can! 21:45:29 thanks 21:45:42 any questions on releases? 21:46:06 ok, last up is general stuff going on 21:46:12 #topic other status updates 21:46:35 acoles: where are with with the EC improvement patches? I feel like we've dropped the ball 21:46:39 (my fault) 21:47:49 acoles: you an I can talk about that later and come up with a plan to land some patches, if you want 21:47:53 notmyname: no ball dropped. so there are three that depend on https://review.openstack.org/231121 which clayg and peluse have reviewed 21:48:14 and imho its in good shape i.e. patch 231121 21:48:15 acoles: https://review.openstack.org/#/c/231121/ - Make ECDiskFile report all fragments found on disk 21:48:21 ok 21:48:24 good 21:48:43 will review 21:48:47 thanks kota_ 21:48:52 the the others are linked as dependent patches from there, and they bring the real value 21:48:55 because my patch depends on that 21:49:13 231121 is just getting supporting things sorted in diskfile 21:49:20 ok 21:49:21 thanks kota_ ! 21:49:32 mattoliverau: anything to report on container sharding? 21:49:55 Yeah, were making good progress 21:50:18 IBM have given a few bodies to help test and do some dev work 21:50:32 great! 21:51:08 I'm thinking we might be getting to a point where we might need to think about making a feature brach so all the people can more easily work on it upstream 21:51:09 mattoliverau: what do you need to make progress there? any other eyes or hands? just time to do the typing and thinking? 21:51:17 mattoliverau: ah? 21:51:43 But really ATM, I think it's more thinking and working time 21:51:47 ok 21:51:51 thanks for the update 21:52:01 we should talk more later about a feature branch or not 21:52:09 jrichli: any updates for us on crypto work? 21:52:28 Sure, we might want to wait until IBM testing and benching comes back 21:52:33 just chugging along ... most of the trello cards are assigned. just need to finish them 21:52:46 #link https://trello.com/b/63l5zQhq/swift-encryption 21:52:57 jrichli: and you need copy middleware ;-) 21:53:05 Yes :-) 21:53:08 heh 21:53:11 yes! 21:53:22 ok :-) 21:53:31 got the msg 21:53:37 wow. we got through all the things on the agenda! 21:53:43 #topic hackathon 21:53:44 tdasilva: no pressure there :) 21:53:49 so the next hackathon 21:54:12 yey 21:54:20 acoles and HPE are hosting in bristol (UK) in early february next year 21:54:29 YAY! 21:54:29 early? 21:54:43 tdasilva: oh. oops 21:54:46 late february! 21:55:03 he and I are working on logistics, and we'll be getting that available asap 21:55:29 very late february 21:55:34 week of feb 28 21:55:35 ;) 21:55:41 practically early march 21:55:58 almost summer in fact 21:56:04 lol 21:56:08 http://tinyurl.com/npssaf4 21:56:08 lol 21:56:08 lol 21:56:39 acoles: I expect good conversation around a table in a warm pub with dark beer! 21:56:53 acoles: will we have elvenses? =] 21:56:55 I will drink for that 21:57:01 shouldn't the hackathon be hosted at a pub? 21:57:02 you mean a dark pub with warm beer surely? 21:57:14 for that too 21:57:18 lol 21:57:37 anything else to bring up in the meeting this week? 21:57:47 just a quick question on the entrypoints patch: eikke mentioned on a comment that there's still work to do, should it be marked as WIP and do we know if he is still working on it? 21:58:33 tdasilva: he's definitely still working on it and interested in it 21:59:15 can I just mention bug 1487791 - there's a discussion to be had (on launchpad) about what the right fix is for that, I'd appreciate others' opinions 21:59:15 bug 1487791 in OpenStack Object Storage (swift) "POST to DLO squashes data without fast-POST" [Undecided,New] https://launchpad.net/bugs/1487791 21:59:38 acoles: right. you had a good comment today. I want to respond this afternoon 21:59:38 on launchpad, not here. 21:59:45 notmyname: thanks 21:59:50 we're out of time here ;-) 22:00:02 thanks for coming today. thanks for working on swift! 22:00:07 #endmeeting