19:00:13 #startmeeting swift 19:00:14 Meeting started Wed Dec 10 19:00:13 2014 UTC and is due to finish in 60 minutes. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:15 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:00:17 The meeting name has been set to 'swift' 19:00:26 who's here for the swift meeting? 19:00:35 hello 19:00:36 aye 19:00:37 hello 19:00:43 hi 19:00:45 o/ 19:00:46 yo 19:00:48 here 19:01:04 welcome! 19:01:07 present 19:01:29 I gave a swift overview at a meetup last night and got to brag on all of you. easily my favorite part of telling people about swift 19:01:42 agenda for today's meeting 19:01:42 #link https://wiki.openstack.org/wiki/Meetings/Swift 19:02:09 ok, let's switch things up 19:02:20 I think gvernik's topic is pretty short (but a great question 19:02:27 #topic SWLite WAL 19:02:32 "WAL support for SQLite. This blueprint https://blueprints.launchpad.net/swift/+spec/cactus-sqlite-wal was closed by John, in 2011. Would like to hear what community thinks about it. What were the issues in Swift with WAL? Are they still exists? Recent SQLite versions are much better than of 2011 so I wonder if recent SQLlite version with WAL mode can provide better SQLlite response times for Swift." 19:02:47 gvernik: any specific question beyond that? 19:02:56 yes..here is the story: 19:03:16 I did series of benchmarks. One was to upload 500K objects of 15KB size to single, empty container.I saw that container server could process X OPS. If I put middleware on the container server that does nothing, (response with 200 OK), than container server can 'process' 6*X requests... Trying to understand what is wrong. 19:03:27 so i saw that WAL can greatly improve SQLite 19:03:39 and want to know your thoughts about 19:03:57 I'm not super familiar with it, what the size/scope of the actual changes? 19:04:27 gvernik: we looked into it a long time ago, but I don't know if anyone has tried it recently 19:04:48 today WAL has great support in SQLite, much better than in 2011 19:05:09 * peluse wonders if his microphone is turned on 19:05:15 and it suppose to improve response times of SQLite 19:05:17 gvernik: but my opinion is that I'd love for someone to look in to it. peluse's question is pretty important when it get's down to the code 19:05:34 but as far as basic investigation goes, have at it 19:05:39 notmyname,: what were the issues in 2011? 19:05:51 gvernik: have you had any chance to look into an implementation? 19:06:14 gvernik: I don't remember exactly. I don't know if it was replicating it or performance or what? I don't remember 19:06:41 gvernik: if you see him online, you might ask dfg. IIRC he's the one who looked at it back then 19:06:49 (doesn't look like he's on now) 19:07:39 notmyname: ok... 19:07:45 gvernik, seems like if its easy to enable then a pre-req for real consideration would be to put it through its paces and see what if anything breaks perf wise or feature wise 19:08:08 peluse: ++ 19:08:16 For those playing along at home, this is WAL (I think): https://www.sqlite.org/wal.html 19:08:28 mattoliverau: yup. thanks 19:08:50 peluse: I know that WAL will improve much response times of container servers, it looks like it will be able to handle more ops 19:09:21 gvernik: maybe a good first step is to write a toy app that compares WAL and not. then try it out in the container server. then figure out any durability/migration issues that have to be sovled 19:10:41 notmyname,: ok, sort of did it already... but than container data become corrupted at some point.. but performance was great :) 19:10:45 gvernik: are you able to look into it and work on Swift + WAL? 19:10:51 lol 19:11:03 sweet :) 19:11:17 gvernik: there's always a trade-off ;) 19:11:31 notmyname,: I will look at it, gladly. wanted to know if you have some strong objections, but seems you are open mind to check this feature 19:11:32 lol 19:12:38 gvernik: AFAIK there's no "there be dragons here" danger. I'd love to have a better answer. either it won't work because X or it does work so let's use it 19:12:47 gvernik: thanks for bringing it up 19:13:19 gvernik: any other questions you have there? 19:13:44 notmyname,: no 19:13:52 ok 19:13:56 #topic EC status 19:14:02 and this will get into priority reviews 19:14:17 so reconstructor is one thing... pretty good progress and should be ready for review within a week or so 19:14:20 peluse: what's up with EC right now? tons of (big) patches are there 19:14:32 tsg and/or torgomatic will ahve to comment on PUT path (still needs .durable stuff) 19:14:33 peluse: great! 19:14:54 and clayg has started on GET but I'm not sure how much is in his head versus in the patch right now :) 19:15:02 and clayg has a GET path patch (admittedly "for the lolz") 19:15:05 peluse, notmyname: I was finishing up some eventlet stuff so that can get released (a dependency we have for the .durable stuff) 19:15:08 priority reviews should be up to date and there are a few that need review 19:15:10 good 19:15:35 also trello should be up to date now as well. there are some open questions up there for a few folks working on EC that I don't think are answered yet though 19:15:55 tsg, ahh cool thanks. ETA? 19:16:07 eventlet - some time next 19:16:10 notmyname: pyeclib stuff also has gone through some changes (that I pushed an hour ago) - will get back to the PUT stuff today 19:16:21 peluse: ^^^ 19:16:22 tsg: ok 19:16:30 peluse: I need to also update the latest on Trello 19:16:30 tsg: some time next...? 19:16:38 note: when I say reconstructo patch it will have everything except the actual reconstruction. Sounds funny but that's the easy part :) Need PUT/GET to really be done before it has the final touches 19:16:40 sorry :) next week 19:16:48 :-) 19:16:51 heh, could have been next year :) 19:16:55 heh 19:17:27 peluse: I am down to 2 eventlet issues - so next week is the best bet 19:17:29 peluse: ya, I'm getting the impression that there's a ton of foundational work that's been done and currently proposed (nearly done). then the visible stuff is relatively small to tie it all together 19:17:49 peluse: not saying we'll be done next week, but basically that there's light at the end of the tunnel 19:17:55 peluse: would you agree? 19:18:22 notmyname, yes with the exception of probetests.... still kinda dark there at least for me 19:18:27 ok 19:18:56 we can talk more about that offline also, there's a trello card there bringing up the issue(s) 19:19:09 notmyname, peluse: can we get #link https://review.openstack.org/#/c/132389 reviewed? simple patch - but this will be one thing out of the way 19:19:22 hi 19:19:23 whoops 19:19:27 tsg: ya. I'll put it on the priority reviews 19:19:35 notmyname: thank you 19:20:06 clayg: welcome to the meeting :P 19:20:15 anything else peluse? 19:20:31 peluse: the biggest thign right now for those of us not coding a feature is to review, right? 19:20:37 Sounds like the ec release target could be kilo-2 or kilo-3? 19:20:42 right, reviews please 19:20:49 is there a kilo-4? :) 19:20:50 kota_: let's not get ahead of ourselves yet ;-) 19:21:01 peluse: there can be kilo-whateveryouwant 19:21:07 I see 19:21:43 so...speaking of releases.... 19:21:45 mattoliverau: i have different idea of "early" than you do 19:21:51 #topic next release of swift 19:21:54 swift 2.2.1 19:22:01 yeah, I think its too early to call... we need PUT/GET/recon done and line of sight on tests before we should make release statements 19:22:04 clayg: lol 19:22:22 final hurrah for the year: getting some of the landed bug fixes released 19:23:09 there's a few things listed under "stuff for 2.2.1" on https://wiki.openstack.org/wiki/Swift/PriorityReviews 19:23:17 peluse: yup, thanks 19:23:18 none of these I feel are critical 19:23:27 but very nice to have in a bugfix release 19:23:39 hi 19:23:53 * cschwede is late too… timezones 19:23:58 cschwede's thing with the ring warning looks interesting 19:23:58 cschwede: o/ 19:24:00 notmyname: should i add https://review.openstack.org/#/c/140478/ too? 19:24:01 however, just yesterday there were some things discussed by cschwede and swifterdarrell that would be nice 19:24:11 clayg: cschwede: yes. you are typing faster than me 19:24:15 #link https://review.openstack.org/#/c/140478/ 19:24:25 that one is important 19:25:07 there's nothing particularly broken about swift, but the current placement methodology can cause very imbalanced clusters to do something unexpected, and we need warnings for that 19:25:55 i added it to the important reviews 19:26:28 cschwede: thanks 19:26:38 tsg: and I added your EC one 19:26:53 notmyname: great .. thx! 19:26:55 ok, so for 2.2.1, anything else? 19:27:08 this one, maybe? https://review.openstack.org/#/c/140178/ 19:27:09 I think it should mostly be a non-event (and that's good) 19:27:43 torgomatic: self promoting much? 19:27:49 :-) 19:27:54 it's a good patch 19:28:00 * torgomatic has not yet begun to self-promote :) 19:28:01 lol 19:28:20 idk, there's lots of good patches up that need reviews, everytime I don't do a reviews for two days I feel like the entire thing gets away from me :'( 19:28:25 ya 19:28:51 like, this bug really scares me -> https://bugs.launchpad.net/swift/+bug/1328735 19:28:53 Launchpad bug 1328735 in swift "Object-updater gives up updating container with no success if all containers are placed at handoff" [Undecided,In progress] 19:30:23 clayg: I'll look at it and see what we can do for progress there 19:30:27 https://review.openstack.org/#/c/135319/ has one +2 already (rework splice and tee) 19:30:30 needs one more 19:30:53 https://review.openstack.org/#/c/126923/ fsync on dirs has some more work to do, so I'm going to drop it from 2.2.1 19:31:15 notmyname: a) I don't really know how to test that b) isn't that a newish feature anyway c) is anyone using it d) is it a bug or a footgun? 19:31:18 anything have any details on https://review.openstack.org/#/c/136258/ (fix GET if there is a missing segment in LOs) 19:31:24 notmyname: re: splice and tee 19:31:44 I like the missing-segment patch, but it now contains a significant portion of code written by me, so I've not +2ed it 19:31:55 torgomatic: ack 19:32:00 bah, i disagree with everything today, ya'll review whatever :P 19:32:08 we can argue over whether 409 is the right status code to return, but anything beats a 500 IMO 19:32:29 clayg: kota_: what's the current status of https://review.openstack.org/#/c/99824/ (global replication). IIRC clayg had some concerns 19:32:50 torgomatic: especially when you're incentivized (by SLA) to not return 5xx! 19:32:52 notmyname: it's all rebased now - needs a good review - probably makes ssync better so it's high on my list! 19:33:01 notmyname: srsly :) 19:33:01 clayg: ack. cool 19:33:33 ok, so for me, top of the list is cschwede's patch for reporting warnings on ring balance 19:33:37 notmyname: I rebased according to clayg suggestion 19:33:50 kota_: thanks 19:33:56 notmyname: I am waiting for another review, write now 19:34:10 s/write/right/ 19:34:16 then global cluster and splice each need one more +2. then GET on LOs. 19:34:25 I'll reorder the list to reflect that 19:34:33 any arguments with that order? 19:35:30 ok, I'll update after the meeting 19:35:47 #topic other reviews / other things 19:35:54 what else is on your mind? 19:36:06 notmyname: should the swift-ring-builder warning go on top of that list? 19:36:19 tdasilva: ya, it will 19:37:16 I'm working on some possibilities for the next swift hackathon. I'll let you know as soon as I know something 19:37:27 I want to give you enough time to look at travel, etc. 19:37:49 +1 19:37:50 notmyname: do you have a time in mind for when the hackathon would happen? even ballpark... 19:38:15 but the general thing is: yes, we'll do one. before the next summit (probably feb/march). probably (but not at all definite) in the eastern half of the US 19:38:43 brrrr 19:38:59 peluse: maybe not ;-) 19:39:11 peluse: you act like your from AZ 19:39:22 reminder: weather in AZ in Jan/Feb can't be beat! 19:39:44 peluse: it can by being here :) 19:40:00 Thats the middle of summer 19:40:03 mattoliverau: +1 19:40:03 ok, next meetings 19:40:16 yes to meeting next week. no to meeting on the 24th 19:40:33 also I propose no to meeting on the 31st 19:40:46 notmyname: +1 19:40:49 sounds good 19:40:50 sweet swift holiday break! 19:41:03 clayg: you still have to code/review. just no meeting ;-) 19:41:13 +1 :) 19:41:16 shoot 19:41:22 merry christmas! I got you a code review! 19:41:30 lol 19:41:34 lol 19:41:38 sounds like a meme 19:42:09 thanks everyone from coming. thanks for working on swift! 19:42:15 #endmeeting