Wednesday, 2014-01-29

*** byeager has quit IRC00:03
*** byeager has joined #openstack-swift00:03
*** byeager has quit IRC00:08
*** judd7 has quit IRC00:09
*** judd7 has joined #openstack-swift00:12
*** Midnightmyth has quit IRC00:22
*** matsuhashi has joined #openstack-swift00:30
*** NM has joined #openstack-swift01:17
*** hurricanerix has quit IRC01:25
*** zaitcev has quit IRC01:30
openstackgerritJon Snitow proposed a change to openstack/swift: Privileged acct ACL header, new ACL syntax, TempAuth impl.
*** nosnos has joined #openstack-swift01:37
*** csd has quit IRC01:43
*** shri has quit IRC01:43
*** NM has left #openstack-swift01:59
*** zackf has quit IRC02:28
*** chandankumar_ has joined #openstack-swift02:36
*** lpabon has quit IRC02:42
*** zackf has joined #openstack-swift02:43
*** chandankumar_ has quit IRC02:48
*** hugokuo has quit IRC02:51
*** minnear has quit IRC02:52
*** hugokuo has joined #openstack-swift02:52
*** minnear has joined #openstack-swift02:52
*** byeager has joined #openstack-swift03:03
*** chandankumar_ has joined #openstack-swift03:09
*** chandankumar_ has quit IRC03:16
*** matsuhashi has quit IRC03:30
*** matsuhashi has joined #openstack-swift03:31
*** zackf has quit IRC03:34
*** matsuhashi has quit IRC03:35
*** dencaval has quit IRC03:38
*** dencaval has joined #openstack-swift03:38
*** peluse has quit IRC03:40
*** peluse has joined #openstack-swift03:40
*** gyee has quit IRC03:44
*** byeager has quit IRC03:57
*** raies has joined #openstack-swift04:46
raiesnotmyname: can you please suggest best material on cross origin resource sharing ??04:47
*** matsuhashi has joined #openstack-swift04:55
*** zackf has joined #openstack-swift05:00
*** zackf has quit IRC05:08
*** jasondotstar has quit IRC05:10
*** ppai has joined #openstack-swift05:10
*** nshaikh has joined #openstack-swift05:11
*** matsuhashi has quit IRC05:34
*** matsuhashi has joined #openstack-swift05:35
*** krast has quit IRC05:35
*** nosnos has quit IRC05:38
*** nosnos has joined #openstack-swift05:41
*** zackf has joined #openstack-swift05:45
*** SkyRocknRoll has joined #openstack-swift05:45
*** Anju has joined #openstack-swift06:14
Anjuhii all, which version of python swift needs06:15
hugokuoAnju: 2.6~2.706:17
Anjuhugokuo:  can I use 2.7.1?06:18
hugokuoAnju: As I know.. Yes. We are running 2.7.3 on our box06:18
Anjuhugokuo:  can you tell me06:18
Anjuwhere is is mentioned ?06:18
Anjuhugokuo:  is 2.7 backward compatible to 2.406:20
hugokuoAnju: I don't really sure about if there's any conflict with 2.4 sry06:21
*** zackf has quit IRC06:21
Anjuokk hugokuo : but please tell me where the python requirement is mentioned06:21
hugokuoAnju: As a note in developer guide. It requires 2.6+
*** matsuhashi has quit IRC06:23
Anjuhugokuo:  thanks a lot06:23
*** matsuhas_ has joined #openstack-swift06:25
*** sourabh__ has quit IRC06:28
*** early has quit IRC06:39
*** early has joined #openstack-swift06:47
raiesHi.. can someone tell me about CORS practical implementation ??06:52
*** psharma has joined #openstack-swift07:09
raiesnotmyname: it is possible to give a fake Origin header in the request during Cross origin resource sharing, then how it is effecient to handle these requests.07:19
*** madhuri has joined #openstack-swift07:22
madhuriHi, can anyone tell me what does fast POST mean?07:25
*** mjseger has quit IRC07:28
*** SkyRocknRoll_ has joined #openstack-swift07:43
SkyRocknRollraies, here you go
*** SkyRocknRoll has quit IRC07:44
*** SkyRocknRoll_ is now known as SkyRocknRoll07:44
*** SkyRocknRoll has quit IRC07:45
*** SkyRocknRoll has joined #openstack-swift07:48
*** Midnightmyth has joined #openstack-swift07:52
*** markd has quit IRC07:52
*** chandankumar_ has joined #openstack-swift07:59
*** chandankumar_ has quit IRC08:03
*** chandankumar_ has joined #openstack-swift08:03
*** xga has joined #openstack-swift08:10
*** thomaschaaf has joined #openstack-swift08:10
*** Midnightmyth has quit IRC08:12
*** chandankumar_ has quit IRC08:15
*** thomaschaaf has quit IRC08:16
*** dencaval has quit IRC08:19
*** dencaval has joined #openstack-swift08:20
*** thomaschaaf has joined #openstack-swift08:20
*** chandankumar_ has joined #openstack-swift08:40
*** chandankumar_ has quit IRC08:46
*** nacim has joined #openstack-swift08:47
*** thomaschaaf has quit IRC08:53
openstackgerritTristan Cacqueray proposed a change to openstack/python-swiftclient: Add SSL certificate verification by default
tristanCHello folks, this last changeset should be good for review09:01
tristanCAnd now I wonder what kind of ssl tests should be added, as most of the work is done by the requests module09:03
*** matsuhas_ has quit IRC09:04
*** matsuhashi has joined #openstack-swift09:04
*** xga_ has joined #openstack-swift09:07
*** xga has quit IRC09:07
*** mlipchuk has joined #openstack-swift09:10
*** matsuhashi has quit IRC09:17
*** matsuhashi has joined #openstack-swift09:19
*** matsuhashi has quit IRC09:51
*** nosnos has quit IRC09:51
*** mehmetali has joined #openstack-swift09:56
*** xga_ has quit IRC09:56
*** xga has joined #openstack-swift09:57
*** _bluev has joined #openstack-swift10:00
*** xga_ has joined #openstack-swift10:04
*** xga has quit IRC10:07
acolesdfg: sorry, just seen your message.10:33
*** gvernik has joined #openstack-swift10:58
openstackgerritBalazs KOSSOVICS proposed a change to openstack/swift: 503 instead of 404 if account server fails
*** xga_ has quit IRC11:08
*** zul has quit IRC11:20
*** zul has joined #openstack-swift11:24
*** gvernik has quit IRC11:34
*** tristanC has quit IRC11:36
*** tristanC has joined #openstack-swift11:36
*** marcusvrn has joined #openstack-swift11:41
erlonmarcusvrn: ping11:53
erlonmarcusvrn: ;)11:53
marcusvrnerlon: nice! :)11:54
marcusvrnportante: hey all, I'm trying to adapt the existing in-memory diskfile interface example to Swift, but it still saves containers and objects on sdb1 and not in memory, could someone help me to understand what is happening?11:57
openstackgerritA change was merged to openstack/swift: Merge branch 'milestone-proposed' (1.12.0) into merge_branch
koolhead17Can someone take this question
*** ppai has quit IRC12:04
*** Trixboxer has joined #openstack-swift12:16
*** haomaiwang has quit IRC12:18
*** haomaiwang has joined #openstack-swift12:18
*** gvernik has joined #openstack-swift12:18
*** ppai has joined #openstack-swift12:22
portantemarcusvrn: I'll be able to help later when I get into the office, around 9:30 AM EST12:22
marcusvrnportante: ok! thks!!12:25
*** mehmetali has quit IRC12:47
*** AbyssOne__ is now known as a1|away12:54
*** chandankumar_ has joined #openstack-swift13:01
*** chandankumar_ has quit IRC13:05
*** chandankumar_ has joined #openstack-swift13:06
*** gvernik has quit IRC13:10
*** chandankumar_ has quit IRC13:30
*** dencaval has quit IRC13:33
*** ppai has quit IRC13:34
*** dencaval has joined #openstack-swift13:34
*** ppai has joined #openstack-swift13:51
*** chandankumar_ has joined #openstack-swift13:51
*** chandankumar_ has quit IRC13:51
*** haomaiwang has quit IRC14:10
*** ppai has quit IRC14:10
*** haomaiwang has joined #openstack-swift14:16
*** gvernik has joined #openstack-swift14:18
*** IRCInfoCustomer has joined #openstack-swift14:22
*** IRCInfoCustomer has left #openstack-swift14:22
*** gvernik has quit IRC14:28
*** nacim has quit IRC14:36
*** tongli has joined #openstack-swift14:37
*** byeager has joined #openstack-swift14:37
*** haomaiwang has quit IRC14:42
*** haomaiwang has joined #openstack-swift14:43
*** chandankumar_ has joined #openstack-swift14:48
*** nacim has joined #openstack-swift14:49
*** Midnightmyth has joined #openstack-swift14:51
*** chandankumar_ has quit IRC14:51
*** chandankumar_ has joined #openstack-swift14:53
*** chandankumar_ has quit IRC14:57
*** chandankumar_ has joined #openstack-swift14:58
portantemarcusvrn: back14:58
*** psharma has quit IRC14:58
*** Midnightmyth has quit IRC15:01
*** chandankumar_ has quit IRC15:02
*** chandankumar_ has joined #openstack-swift15:03
marcusvrnportante: hey!15:07
portantegood morning (or afternoon)15:07
*** chandankumar_ has quit IRC15:07
marcusvrnportante: good morning/afternoon (afternoon here)15:09
portanteso how are things going?15:09
portanteyou mentioned having some issues earlier with the in-memory object server15:09
marcusvrnportante: yep....I intended to run swift with in-memory diskfile, but I don't know if I did sth wrong...because when I upload an object, it is saved on sdb115:11
portantewhat does your object server .conf file look like and what version of swift are you using?15:12
marcusvrnportante: I'm using master swift and my .conf is like object-server.conf-sample15:16
portanteso can you post your "use = " line for the object-server section?15:16
marcusvrnportante: for sure! "use = egg:swift#object"15:18
portantechange that to "use = egg:swift#mem_object"15:19
portanteright now the mechanism for using a different object server is to change the use line to load the correct server code15:20
marcusvrnhmmm...nice! but changing it, I don't need to change obj/, right?15:21
portanteit is using the obj/ instead15:21
marcusvrnand if I create a new diskfile, which parameter i need to write in "use=" line?15:23
marcusvrnwhere do i need to define the parameter, in this case "mem_object"?15:28
*** zul has quit IRC15:30
portanteso you should write your new DiskFile class, and then follow the file, cloning that to use your new diskfile15:30
*** zul has joined #openstack-swift15:30
dfgacoles: its alright- i want to add some account lvl ratelimiting. Basically to prevent a problem user from taking too much of the system's resources. I thought a good place to add the flag was using your sysmeta stuff. I think it would work but wanted your opinion on in.15:32
dfgacoles: in your patch you didn't set up any specific means of setting this data did you?  I was thinking of just adding in the functionality and if anybody would need to use it they'd just set it with a direct client or something but I'm pretty sure that won't go over very well. idk15:33
acolesdfg: sounds like it could be a candidate for sysmeta15:33
*** chandankumar_ has joined #openstack-swift15:34
*** hurricanerix has joined #openstack-swift15:34
acolesdfg: when you say "means of settingthis data" do you mean a function to add a sysmeta header?15:35
*** hurricanerix has quit IRC15:35
acolesdfg: i did put a convenience method in like add_sysmeta(key, value) but it got reviewed-away because nothing was calling it :)15:35
*** hurricanerix has joined #openstack-swift15:36
*** byeager has quit IRC15:36
*** byeager has joined #openstack-swift15:36
dfgthats fine- i just didn't know what your thoughts were on how it'd be used. We could set it with a direct client, or there could be some weird backdoor middleware or something. i kinda like the direct client better15:37
acolesdfg: so currently you need to add to req.headers - there is a get_sysmeta_prefix function to avoid typing x-account-sysmeta- or whatever, in request_helpers.py15:37
dfgacoles: but you'd have to add those headers in some middleware beneath the gatekeeper right?15:37
marcusvrnportante: alright! i'll try to run in-memory object server, changing "use=" line and after that, i'll try to write my diskfile!! tks so much!!!!!15:37
acolesdfg: clients can't set sysmeta though - gatekeeper prevents that, deliberately15:37
*** chandankumar_ has quit IRC15:38
dfgacoles: ya- i'd use like swiftly -D- skip all the middleware. this would just be for sys admin use for example15:38
portantemarcusvrn: welcome15:38
dfgor direct client15:38
acolesdfg: so yes, pattern would be client sends x-foo, middleware validates and copies to x-account-sysmeta-foo, which is then automaticaaly persisted (assuming its an account request)15:38
dfgya- thats what i was thinking. hmmm15:39
*** chandankumar_ has joined #openstack-swift15:39
acolesdfg: oh, so if you have a backdoor for sysadmin client and send straight to proxy_Server then you could just send the sysmeta header. the gatekeeper is there to stop users ever tinkering15:40
*** byeager has quit IRC15:41
*** IRTermite has left #openstack-swift15:41
*** chandankumar_ has quit IRC15:45
*** chandankumar_ has joined #openstack-swift15:46
*** mjseger has joined #openstack-swift15:47
dfgacoles: ya- thats the idea. it would work. how would that sound as a general way of setting those flags. assuming people are going to start using them for different types of sysadmin type things.15:47
*** chandankumar_ has quit IRC15:49
*** chandankumar_ has joined #openstack-swift15:50
acolesdfg: i guess if its only ever sysadmin then its ok to subvert the gatekeeper.15:50
acolesdfg: maybe others will scream objections...15:51
acolesdfg: btw, here's the function that didn't make it, in case its of any use (was in :
portantewe are using git not subversion15:52
*** zackf has joined #openstack-swift15:52
*** chandankumar_ has quit IRC15:52
acolesportante: :)15:53
marcusvrnportante: sry...just one more question....i'm trying to run servers ("startmain"), but it returns me an error: "KeyError: 'getpwnam(): name not found: swift'"15:55
mjsegerawhile back I brought up the fact that 1K PUTs were slower then 2K PUTs and after others confirmed it was because of the nagel algorithm, I thought we confirmed that too and someone said they were making the change to set TCP_NODELAY client-side15:56
mjsegerbut when I clone the latest version of swiftclient and grep for TCP, I don't see that set anywhere.  is this still on someone's todo list?15:56
dfgacoles: alright cool. i'll code something up and see what people say about it. i'm sure it'll go just great... :)15:59
*** byeager has joined #openstack-swift16:01
marcusvrnportante: i fixed it! Sry for that question! :)16:04
*** lpabon has joined #openstack-swift16:06
*** nshaikh has left #openstack-swift16:07
portantemarcusvrn: no problem, just got back to my desk and saw your question16:08
portantemjseger: we set NDELAY server side, for sure, not sure about the client16:10
mjsegeroooooh, I don't thnk that will solve the problem, will it?16:10
mjsegerit's the client-side where nagel is causing the extra 0.04 sec delay, at least I think so16:11
portanteI don't think so either16:11
mjsegerportante: so doesn't that mean users will continue to see slower 1k puts?16:12
mjsegerthe thing that concerns me is people love to run benchmarks (me too ;)) using 1k, 10k etc object sizes and swift does NOT do well with 1k16:13
mjsegerand I don't want to see if get a bum rap16:13
Dieterbeany tips on rsync max_connections settings?16:15
Dieterbei see we still have it set to 2, and also been seeing "max connections reached" errors.16:15
marcusvrnportante: now it's working, but when i upload an object, in log file i got this:[29/Jan/2014:16:06:37 +0000] "PUT /sdb1/802/AUTH_test/teste1"[29/Jan/2014:16:06:37 +0000] "HEAD /sdb1/135/AUTH_test/teste1/test.txt" it still saving object on sdb, right?16:15
Dieterbesince we have beefy machines, i wonder if i can crank this up to 20 or something?16:15
portanteso that is because the proxy server does not know that it is only in-memory, and the rings you are using are setup to use disks from before16:16
*** otherjon_ has joined #openstack-swift16:16
portantemjseger: there seems to be two things to consider here16:17
portante1. behavior of the python-swiftclient code in the face of small objects as it relates to the nagel algorithm16:18
portante2. behavior of the swift server in the face of small objects16:18
mjsegerso it sounds like someone has address the server-side but not the client side16:19
portantesomeone can address the swift client as well, but I would be careful to not lump them together16:19
mjsegerI was pretty sure when we had this conversation notmyname; suggested a patch which i tried and it worked. then he (or someone else) said they would commit it.16:20
portanteI would consider doing benchmarks against the swift server using a client on which you have total control (or near total control) of the messages being sent16:20
portantethat notmyname!16:20
*** haomaiwa_ has joined #openstack-swift16:22
*** zaitcev has joined #openstack-swift16:24
*** haomaiwang has quit IRC16:26
marcusvrnportante: i'm looking for documentation that explain how ring works and how configure it, but  actually i didn't find sth about that.16:32
pelusemarcusvrn:  check out this video:
portantemarcusvrn: and
*** gyee has joined #openstack-swift16:46
*** byeager_ has joined #openstack-swift16:56
*** byeager has quit IRC16:56
*** acoles_ has joined #openstack-swift16:59
*** acoles_ has quit IRC16:59
*** byeager has joined #openstack-swift17:20
*** byeager_ has quit IRC17:23
*** mlipchuk has quit IRC17:29
marcusvrnpeluse: portante  thks for the links17:32
*** nacim has quit IRC17:34
peluseyup, you bet17:36
*** nacim has joined #openstack-swift17:42
*** csd has joined #openstack-swift17:43
*** pheadron has joined #openstack-swift17:48
*** pheadron1 has quit IRC17:50
*** nacim has quit IRC17:52
*** Midnightmyth has joined #openstack-swift17:54
*** fbo is now known as fbo_away18:04
*** byeager has quit IRC18:12
*** byeager has joined #openstack-swift18:13
*** _bluev has quit IRC18:13
*** byeager has quit IRC18:17
*** shri has joined #openstack-swift18:49
*** nacim has joined #openstack-swift18:57
*** zackf1 has joined #openstack-swift19:09
*** krtaylor has quit IRC19:10
*** zackf has quit IRC19:12
*** zackf1 has quit IRC19:23
*** dencaval has quit IRC19:28
*** dencaval has joined #openstack-swift19:28
*** dencaval has quit IRC19:32
*** byeager has joined #openstack-swift19:33
*** gvernik has joined #openstack-swift19:33
*** dencaval has joined #openstack-swift19:33
*** zackf has joined #openstack-swift19:48
*** krtaylor has joined #openstack-swift19:56
*** nacim has quit IRC20:01
*** byeager has quit IRC20:02
*** byeager has joined #openstack-swift20:03
*** byeager has quit IRC20:05
*** byeager has joined #openstack-swift20:05
*** wayneeseguin has joined #openstack-swift20:10
*** dencaval has quit IRC20:18
*** byeager has quit IRC20:18
*** dencaval has joined #openstack-swift20:18
*** byeager has joined #openstack-swift20:19
*** byeager has quit IRC20:24
*** csd has quit IRC20:24
*** byeager has joined #openstack-swift20:33
*** byeager has quit IRC20:42
*** byeager has joined #openstack-swift20:42
*** byeager_ has joined #openstack-swift20:45
*** byeager has quit IRC20:45
*** pberis_ has quit IRC20:53
*** pberis has quit IRC20:53
*** byeager_ has quit IRC20:57
*** byeager has joined #openstack-swift20:57
*** byeager has quit IRC21:01
*** byeager has joined #openstack-swift21:02
*** shri has quit IRC21:08
*** shri has joined #openstack-swift21:11
werhrm. I was just trying to be nice and push a node out before I removed it.... and adjusted the weight from 3000 to 10 on each of it's drives.  And now it is pulling tons of data.  meanwhile I have a node with the same weight as all the nodes that is getting no data.21:26
werthe whole reason I was pushing this node out in the the first place is that I was migrating to one zone in that rack.  So the node that is filling up in zone 1.  And the other nodes are in zone 0.  Before the change zone one's usage was higher.  which made sense.  But now it is going to fill up :/21:32
werso the answer is.... I must have done something stuupid.21:33
torgomaticwer: yeah, the partition placement algorithm prioritizes durability over even distribution, so if you've got two zones and three replicas, a minimum of 1/3 of your data will reside in the smaller of the two zones21:33
torgomaticif you want to take drives out, set their weights to 0, I think21:33
torgomaticthat should make it so they have no partitions allocated, so the data should then move off them21:34
werI'll try that in an hour :)  I knew I should have set them to 0, but I wanted to see it work how I expected.  Previous to pushing it out it didn't have 1/3 despite being the only node.  But perhaps I didn't rebalance enough between all these steps to know.21:35
torgomaticah yeah, you might have wound up in a situation where 1/3 of your partitions wanted to be in that zone, but due to min_part_hours, they couldn't move yet21:36
weryup :)21:36
torgomaticso after $min_part_hours hours, you rebalanced with lower weights, and the allocated data went *up*21:36
werAnd the choice to move to one zone was to avoid this scenario where that zone couldn't provide 1/3 of the space.  I would rather leverage the host/disk smartness given the size of my cluster :)21:37
werdoes a handoff occur when 1/3 is not available in a given zone?  Or does it start doing node/disk distribution at that point?21:38
torgomaticwer: if you want to stop it now, swift-ring-builder has a (deliberately) undocumented command called "pretend_min_part_hours_passed"21:38
torgomaticwer: there's no handoffs involved; it's all about the placement of primary nodes21:39
werso if that zone filled up what would I expect?21:39
Dieterbedoes tox insist on python 2.6 ? i'm running it on python 2.7.6 and i get: ERROR: InterpreterNotFound: python2.621:40
torgomaticbadness? things get pretty unhappy with full disks... you'd get failed object PUTs at a minimum, possibly lost container updates (can't write an async_pending if there's no space), and probably other things too21:40
wertorgomatic: nice!  jut reinforces my decision :)21:41
*** csd has joined #openstack-swift21:49
wersweet torgomatic now the ring looks like I expected.  The data stopped migrating into zone1 with all the weights set to 0.... letting it soak waiting for magic.21:50
wermy first lie to swift.21:51
*** lpabon has quit IRC21:55
*** tongli has quit IRC22:00
werSo I'll have a similar thing when I add a new region.  I suppose I should just add everything at once.  If I add a new zone, add it all at once.  If I remove a zone, set it to 0 first before removing nodes.22:02
weror scary.22:02
werWhat's crazy is watching a machine with 24 3TB disk fill at %2 and hour.....  btw the node set with a weight of 0 is not egressing at all.  But I'll let it have some time before I poke.22:04
wer*an hour :)22:04
claygnotmyname: can we like assign this but to pbr ->
*** byeager has quit IRC22:10
*** byeager has joined #openstack-swift22:10
openstackgerritDavid Goetz proposed a change to openstack/swift: Add a way to ratelimit all writes to an account
*** byeager has quit IRC22:15
clayglike the paste error when trying to load some middleware that is totally installed has come up before right?22:17
torgomaticyeah, it's what's in your egg-info being stale22:20
claygtorgomatic: yeah i've seen it like that in dev, but that normally means I cloned new stuff and didn't develop again22:44
claygyou think the issue in that bug was installed from source with develop?22:45
portantetorgomatic, clayg: i have also see that, where I did not properly develop the tree in my SAIO22:56
* portante above is missing an "n"22:56
openstackgerritpaul luse proposed a change to openstack/swift: Add Storage Policy Support for /Info
*** Midnightmyth has quit IRC22:58
*** shri1 has joined #openstack-swift23:03
*** _bluev has joined #openstack-swift23:05
*** shri has quit IRC23:05
*** krtaylor has quit IRC23:22
torgomaticclayg: I have no idea what that guy's problem is; it's probably either that or he's got a mix of packages and source installs23:23
openstackgerritpaul luse proposed a change to openstack/swift: Add Storage Policy Support for /Info
torgomaticI love how those get filed as bugs against Swift, though. It's not Swift's fault that someone doesn't know how Python works.23:37
*** shri1 has left #openstack-swift23:42
*** csd has quit IRC23:59

Generated by 2.14.0 by Marius Gedminas - find it at!