21:00:40 #startmeeting swift 21:00:41 Meeting started Wed Jan 10 21:00:40 2018 UTC and is due to finish in 60 minutes. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:42 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:45 The meeting name has been set to 'swift' 21:00:46 who's here for the swift team meeting? 21:00:47 o/ 21:00:50 o. 21:00:51 o/ 21:00:57 o/ 21:00:58 o/ 21:01:08 I know torgomatic won't be here today 21:01:17 sorry I'm late 21:01:27 not late. just starting :-) 21:01:41 cschwede: clayg: ping 21:01:50 thanks! 21:01:56 o/ 21:02:01 oh hi mathiasb! 21:02:04 o/ 21:02:12 Happy New Year! 21:02:42 hi notmyname! 21:02:44 tdasilva: ping 21:02:52 hello 21:02:54 all right, let's get started 21:02:59 agenda is at 21:03:04 #link https://wiki.openstack.org/wiki/Meetings/Swift 21:03:19 starting not-from-the-top because of time zones 21:03:33 #topic swift-ring-builder exit code 21:03:36 #link https://bugs.launchpad.net/swift/+bug/1742417 21:03:37 Launchpad bug 1742417 in OpenStack Object Storage (swift) "swift-ring-builder exit code changed from 0 to 1 for empty rings" [High,Confirmed] 21:03:43 what's up here? cschwede? clayg? 21:03:52 hi o/ 21:04:00 i think we were waiting on the gate - then there's a pep8 thing 21:04:22 which patch? 21:04:24 if cschwede is off once mattoliverau 's patch lands tim and I can land the other one and then we're done 21:04:32 cool 21:04:40 and response codes stay as they were? 21:04:45 matt's patch is 530839 21:04:48 s/response/exit 21:04:49 oh my patch hasn't landed yet. 21:04:49 exit codes 21:04:53 yeah :-) 21:04:59 nope :-( 21:04:59 patch 530839 21:05:00 https://review.openstack.org/#/c/530839/ - swift - Show devices marked as deleted on empty rings 21:05:17 notmyname: well pete pointed out that exit code will change for the place we were thorwing stack - but that was a different bug - not an intentional reason to error 21:05:26 that one's at least making its way through the gate now 21:05:27 ok 21:05:44 then there's the meta point that two different states resulting in similar output and exit codes somewhat reduces the overall level of entropy in the universe - which you can argue for or against 21:05:52 IIRC infra was doing some reboots and stuff for the recent kernel bugs, so I guess the gate is a little slow right now 21:06:01 stupid meltdown 21:06:16 at least it's well-named 21:06:49 sounds like this topic is all sorted then. I thought cschwede would be here (but it's late for him) 21:07:10 so let's assume he's out for now, and timburke and clayg will take care of the follow-on to mattoliverau's patch 21:07:18 anything else to address on this one? 21:07:45 s'ok - he knows we got his back (sorry to but you up cschwede - thanks for the quick fixup!) 21:07:56 *bust you up 21:07:57 go team swift! :-) 21:08:06 \o/ 21:08:11 ok. on to the next topic 21:08:20 #topic SLO data segments 21:08:24 is it merged yet? 21:08:25 no :-( 21:08:30 https://review.openstack.org/#/c/365371/ 21:08:30 :( 21:08:30 patch 365371 - swift - Add support for data segments to SLO and Segmented... 21:08:40 however, it's probably better than ever now! 21:08:53 I started looking at it yesterday some, and it looks like kota_ added some comments 21:09:01 Kota left some comments, and it's definitely getting better! 21:09:03 yup 21:09:37 kota_: do you have oustanding questions in your review? doesn't look like it 21:10:14 looked like _kota was going away for a proper look 21:10:21 ah. you're still looking and ... yeah 21:10:30 kota_ even 21:10:33 ok, great. thanks kota_ 21:10:48 notmyname: yeah, i need to get more detail on the imple 21:11:07 yeah, I'm in the same boat 21:11:22 (yay american? idioms in an international meeting!) 21:11:32 #topic s3api work status 21:11:34 the base idea looks good to me but i feel like i want to cleanup the code to generalize on the inline data / normal case. 21:11:43 kota_: tdasilva: what's going on with the s3api work? 21:11:50 tdasilva: looked like you'd been doing some work there this week 21:11:52 i'd just like to point out that poor joeljwright has had this open for more than a year now -- and while it'd be nice if we didn't need to shove bytes into a var named "seg_req" and the like, it might be better to address some things in follow-ups 21:12:10 timburke: +1 21:12:24 thanks timburke :) 21:12:41 notmyname, kota_ I really haven't and i'm sorry :( but today I proposed the WIP I was doing to merge func test 21:12:45 hoping i can get back to it 21:13:06 tdasilva: never mind and thx to be working on it 21:13:24 ok. because of people temporarily stepping away from some upstream work over the last couple of months, we expected that this s3api work would slip 21:13:34 i also have small progress... 1 sec 21:13:36 so it's not a surprise (or shouldn't be to anyone) 21:13:40 kota_: great! 21:13:58 #link https://review.openstack.org/#/c/529268/ 21:13:59 patch 529268 - swift (feature/s3api) - Avoid global LOGGER instance 21:14:32 and i noticed timburke already commented to it, so i should circle back to address. 21:14:44 nice :) 21:15:25 that's all since the last meeting in the last year. 21:15:31 ok 21:15:52 #topic general task queue 21:16:00 m_kazuhiro: rledisez: any updates on this right now? 21:16:15 not on my side 21:16:20 We've landed the internal client required patch 21:16:36 nice 21:16:37 I'm making unit tests now. 21:16:50 I promised to look at the current state of the expirer patch.. which I'll try and get done today.. if work gives me the time 21:17:00 ok 21:17:08 that all sounds good 21:17:13 I will finish to make all tests in this week. 21:17:26 m_kazuhiro: great! 21:17:29 #topic container sharding 21:17:37 another update thing. acoles? mattoliverau? 21:18:15 progress has been a little slow over the holiday period. But things are starting to pick up again. 21:18:16 this is what I think is the top priority Big Thing in swift right now. where are we? 21:18:24 mattoliverau: what's next? 21:18:43 we've worked on some api changes that makes things simpler and easier to grok 21:19:09 ok. internal api changes? 21:19:12 changes on what? 21:19:13 acoles has been awesome at adding more tests for things like his shrinking alg and some race conditions 21:19:22 notmyname: also I just wrote a proposed simplification tradeoff https://etherpad.openstack.org/p/deep-container-listing 21:19:51 the way we ask for shardranges that include an given object 21:20:08 we've fixed some races between sharding and shrinking, fixed some listing issues 21:20:22 sounds good 21:20:25 acoles: was going over this with me this am - i love the plan - phase 1 is to leverage stale container listings during cleaving FTW! we should focus on the data model and persistence layer for correctness 21:20:33 acoles: on that etherpad, what do you need fromt he rest of us? 21:20:38 clayg: ah ok 21:20:40 changing the names for root containers and sharding shard containers. so an op can tell what are suppose to be root containers and what should only be short lived sharding containers 21:20:56 mattoliverau: that sounds good 21:21:38 notmyname: basically the trade off comes down to simpler code if we can live with container listings being a little behind *while sharding* 21:22:15 (vs them being a little behind because your container is so large it can't eat all the updates) 21:22:15 sounds good to me ;-) 21:22:20 ironically, sounds identical to what I did originally.. ie use stale before I added the complicated stuff, which is hard to grok :P 21:22:26 but yeah. that's a good thing to think on 21:22:28 acoles: which is basically the same as "do you want it to work or do you have puppies" 21:22:38 rats... *hate puppies 21:22:53 clayg: either works for me :O 21:22:56 but am totally cool with that.. we need it less hard to grok, and acoles is much smarter then i ;) 21:23:04 not 21:23:48 acoles: do you need feedback on that etherpad? or is this the plan and it will be morphed into some docs page and the rest of us can use it to keep up? 21:23:58 so opinions on that etherpad if anyone feels this is dumbing down too much, but TBH its more about the process of steps we take to get something simpler and robust, then build in more smarts 21:24:29 ok 21:24:49 notmyname: its design choice not docs, feedback from mattoliverau timburke torgomatic if they have time which an be as simple as 'ok' (or not) 21:25:22 ok 21:25:25 well sharding/cleaving very large containers would be a pain.. but they should only happen once.. once sharding is on, sharding will be much quicker, so we shohuld really cater for the "normal" smaller sharding case. So KISS makes so much sense. Big containers can just lag while we shard 21:25:40 is there anything else we need to cover on this topic in the meeting today? 21:25:51 and, I also have a monster patch chain on feature/deep with head being https://review.openstack.org/532604 that at some point would be good to get reviewed 21:25:52 patch 532604 - swift (feature/deep) - WIP Don't attempt to merge root and shard listings... 21:25:55 I for one will look at the etherpad today 21:26:36 the chain is partly because I have broken changes into digestible chunks since it encompasses quite significant changes overall 21:26:37 acoles: that big "WIP" at the top is a "don't review me sign", right? ;-) 21:26:57 notmyname: yes but only on that last patch in the chain 21:27:03 ok :-) 21:27:03 the rest is ready 21:27:07 sounds good 21:27:31 two more topics, I think 21:27:41 #topic next swift release (2.17.0) 21:28:03 bundling all this stuff we talked about, we shoudl release soon(-ish) 21:28:15 #link https://wiki.openstack.org/wiki/Swift/PriorityReviews 21:28:26 ^^ that page is updated with some important patches 21:28:43 it also has listed a couple of high priority bug that don't have patches. they need triaged at least 21:29:18 patch 531527 and patch 530411 are the most important, I think (bugs affecting production) 21:29:19 https://review.openstack.org/#/c/531527/ - swift - proxy: make the right number of container updates 21:29:21 https://review.openstack.org/#/c/530411/ - swift - Fix socket leak on 416 EC GET responses. 21:29:24 that should probably also include a slew of torgomatic's recent patches to do with how terrible x-delete-after + EC is 21:29:28 notmyname: do any more of torgomatics patches need to be on there? 21:29:31 yes! 21:29:46 timburke: acoles: can one of you add them please? 21:29:57 I'd *really* like to see the SLO data segments patch land too 21:30:04 notmyname: yep will do tomorrow 21:30:07 thank you 21:30:22 thinking of this release, it reminds us to look at the openstack release cycle schedule 21:30:28 the queens release is sneaking up on is 21:30:29 p 532595 p 532383 p 532342 p 531527 21:30:30 https://review.openstack.org/#/c/532595/ - swift - Fix object POST of X-Delete-At with skewed clocks 21:30:31 https://review.openstack.org/#/c/532383/ - swift - Don't make async_pendings during object expiration 21:30:33 https://review.openstack.org/#/c/532342/ - swift - Limit object-expirer queue updates on object DELET... 21:30:35 https://review.openstack.org/#/c/531527/ - swift - proxy: make the right number of container updates 21:30:46 our deadline for the last tag of queens is February 19 21:30:49 speaking of swift releases, we haven't had a *client* release since july. not much has landed in the meantime, but we might want to thing about landing https://review.openstack.org/#/c/478611/ and cutting a release there, too 21:30:50 patch 478611 - python-swiftclient - Allow for object uploads > 5GB from stdin. 21:31:27 so a 2.17.0 release has a reasonable chance of being the last of the queens cycle. there's a chance we could do a .1 release closer to the deadline 21:31:52 timburke: agreed. I'll look in to it. thanks for bringing it up 21:32:34 I'm hoping to tag a swift release next week 21:32:46 there's not a hard deadline on that, but it's what i'm hoping for 21:32:55 is there any question about an upcoming release? 21:33:21 if there's other stuff that needs to land (like torgomatic's expiry patches), please let me know or add them to the priority reviews page 21:34:05 only cschwede's ringbuilder fix, but lets try and get that in before he wakes up as a nice surprise ;) 21:34:11 right :-) 21:34:13 ok, let's move on then 21:34:18 #topic PTG in dublin 21:34:22 last topic (from me) 21:34:31 the PTG is coming soon, as well 21:34:44 does anyone know for sure yet if they are NOT coming? 21:35:18 good. I'll expect to see everyone there, then ;-) 21:35:22 \o/ everyone _might_ be coming :P 21:35:26 ALL OF YOU 21:35:26 heh 21:35:40 lol 21:36:00 I think we'll start thinking about topics in a few weeks 21:36:05 i'm guessing joeljwright not being there is a fairly safe bet? 21:36:14 probably :-( 21:36:18 yeah, sorry 21:36:36 yay joeljwright ! 21:37:00 I'm hoping to get some support people from swiftstack to come, too. I'd love to put them in a room with tdasilva and rledisez and onovy and see what kind of magic happens :-) 21:37:08 hey, gotta store all those ML training sets *somewhere* though, right? 21:37:17 hahahaha 21:37:27 #topic open discussion 21:37:33 anythign else to bring up in the meeting this week? 21:37:39 notmyname: can't wait to meet your support team ;) 21:37:42 I do keep mentioning swift when we need checkpoint storage 21:37:45 who knows... 21:37:55 rledisez: I really do hope it happens :-) 21:38:03 in looking at https://review.openstack.org/#/c/528106/ (which is a GREAT plan, i LOVE this idea), i've become appalled by our lack of coverage for CORS. i'm working on throwing together some single-page-app-style test suite where you run a bunch of bash, host the page with `python -m SimpleHTTPServer`, then visit the page and verify that everything's green 21:38:04 patch 528106 - swift - Move CORS to middleware. 21:38:10 soo... that'll be a thing 21:38:45 actually having some functional tests for CORS will be fantastic 21:39:02 notmyname: priority review page updated with expiring object patches 21:39:03 timburke: you're gunna host what *where* now!? 21:39:06 is that a probetest thing? 21:39:09 timburke: cool, honestly CORs confuses me, so thank you for looking into it.. you too Sam ;) 21:39:17 ^ +1 21:39:17 acoles: thanks 21:39:39 clayg: *you're* gonna host it! on localhost! it'll be great 21:40:05 and then we'll have javascript in our repo, and we'll all cry about it 21:40:12 lol 21:40:14 oh dear 21:40:33 there's not much that demonstrates the different skill sets of a "front-end engineer" from a "back-end engineer" than trying to understand the CORS spec and implement it 21:40:44 or, it'll be some external project and even *less* likely to be run 21:41:09 all right. anything else, or are we done for this week? 21:42:05 it's really great to see everyone back from holidays and have everyone back working on swift! thanks for all the work you do to make it a great project and great community 21:42:09 #endmeeting