*** matsuhashi has joined #openstack-swift | 00:07 | |
*** jergerber has quit IRC | 00:11 | |
openstackgerrit | John Dickinson proposed a change to openstack/swift: Change the default token logged length to 16 https://review.openstack.org/94506 | 00:33 |
---|---|---|
*** csd has quit IRC | 00:34 | |
*** dmorita has joined #openstack-swift | 00:40 | |
*** shri has quit IRC | 01:02 | |
*** shakamunyi has quit IRC | 01:45 | |
*** shakamunyi has joined #openstack-swift | 01:59 | |
*** saschpe has quit IRC | 02:00 | |
*** saschpe has joined #openstack-swift | 02:02 | |
*** shakamunyi has quit IRC | 02:05 | |
*** dvas has joined #openstack-swift | 02:18 | |
*** dvas has quit IRC | 02:22 | |
*** gyee has quit IRC | 02:36 | |
openstackgerrit | John Dickinson proposed a change to openstack/swift: Change the default token logged length to 16 https://review.openstack.org/94506 | 02:46 |
*** baojg has joined #openstack-swift | 02:55 | |
*** hipster has quit IRC | 02:56 | |
*** kenhui has joined #openstack-swift | 02:58 | |
*** blazesurfer has joined #openstack-swift | 03:01 | |
*** baojg has quit IRC | 03:01 | |
blazesurfer | hi all | 03:02 |
*** baojg has joined #openstack-swift | 03:02 | |
*** baojg_ has joined #openstack-swift | 03:07 | |
*** baojg_ has quit IRC | 03:09 | |
*** baojg has quit IRC | 03:09 | |
*** baojg has joined #openstack-swift | 03:10 | |
*** baojg_ has joined #openstack-swift | 03:17 | |
*** dvas has joined #openstack-swift | 03:18 | |
*** baojg has quit IRC | 03:20 | |
*** kenhui has quit IRC | 03:21 | |
*** dvas has quit IRC | 03:23 | |
*** madhuri has joined #openstack-swift | 03:33 | |
*** kenhui has joined #openstack-swift | 03:35 | |
*** nosnos has joined #openstack-swift | 03:48 | |
*** kenhui1 has joined #openstack-swift | 03:50 | |
*** kenhui has quit IRC | 03:51 | |
blazesurfer | hey guys | 04:07 |
blazesurfer | Keystone and ssl swift as well would any one have some time to discuss how it all works | 04:08 |
*** ppai has joined #openstack-swift | 04:14 | |
*** dvas has joined #openstack-swift | 04:19 | |
*** haomaiwang has joined #openstack-swift | 04:23 | |
*** dvas has quit IRC | 04:23 | |
*** psharma has joined #openstack-swift | 04:52 | |
*** baojg has joined #openstack-swift | 04:59 | |
*** baojg_ has quit IRC | 05:00 | |
*** baojg has quit IRC | 05:00 | |
*** igor_ has quit IRC | 05:02 | |
*** kenhui has joined #openstack-swift | 05:04 | |
*** blazesurfer has quit IRC | 05:05 | |
*** igor has joined #openstack-swift | 05:06 | |
*** kenhui2 has joined #openstack-swift | 05:07 | |
*** kenhui has quit IRC | 05:07 | |
*** kenhui has joined #openstack-swift | 05:07 | |
*** kenhui1 has quit IRC | 05:08 | |
*** ppai has quit IRC | 05:10 | |
*** kenhui2 has quit IRC | 05:11 | |
*** dvas has joined #openstack-swift | 05:19 | |
*** dvas has quit IRC | 05:24 | |
*** kenhui has quit IRC | 05:27 | |
*** dvas has joined #openstack-swift | 05:30 | |
*** dvas has quit IRC | 05:31 | |
*** dvas has joined #openstack-swift | 05:40 | |
*** elambert has joined #openstack-swift | 06:39 | |
*** dvas has quit IRC | 06:41 | |
*** zaitcev has quit IRC | 06:41 | |
*** ppai has joined #openstack-swift | 06:41 | |
*** Honghui has joined #openstack-swift | 06:52 | |
*** nshaikh has joined #openstack-swift | 06:54 | |
*** dvas has joined #openstack-swift | 07:12 | |
*** dvas has quit IRC | 07:15 | |
*** Ju has joined #openstack-swift | 07:17 | |
*** sungju has quit IRC | 07:29 | |
*** dvas has joined #openstack-swift | 07:37 | |
openstackgerrit | A change was merged to openstack/swift: Make the new xprofile tests optional. https://review.openstack.org/94404 | 07:39 |
*** mlipchuk has joined #openstack-swift | 07:43 | |
*** mlipchuk has quit IRC | 07:47 | |
*** PVince81 has joined #openstack-swift | 07:50 | |
*** mlipchuk has joined #openstack-swift | 08:04 | |
*** nacim has joined #openstack-swift | 08:18 | |
chmouel | pconstantine_: cool glad you figured it out | 08:19 |
*** haomaiwang has quit IRC | 08:30 | |
*** haomaiwang has joined #openstack-swift | 08:30 | |
*** jamie_h has joined #openstack-swift | 08:31 | |
*** haomaiw__ has joined #openstack-swift | 08:33 | |
*** haomaiwang has quit IRC | 08:35 | |
*** pberis has quit IRC | 08:39 | |
*** pberis has joined #openstack-swift | 08:42 | |
*** PVince81 has left #openstack-swift | 08:56 | |
*** Honghui has quit IRC | 08:58 | |
*** mmcardle has joined #openstack-swift | 09:00 | |
*** mmcardle has quit IRC | 09:12 | |
*** mmcardle has joined #openstack-swift | 09:16 | |
*** sungju has joined #openstack-swift | 09:17 | |
*** mkollaro has joined #openstack-swift | 09:18 | |
*** acoles_away is now known as acoles | 09:20 | |
*** mlipchuk has quit IRC | 09:21 | |
*** mmcardle1 has joined #openstack-swift | 09:22 | |
*** sungju has quit IRC | 09:22 | |
*** mmcardle has quit IRC | 09:23 | |
*** mandarine has left #openstack-swift | 09:23 | |
*** mlipchuk has joined #openstack-swift | 09:40 | |
*** sungju has joined #openstack-swift | 09:49 | |
*** matsuhashi has quit IRC | 10:01 | |
*** sungju has quit IRC | 10:03 | |
*** sungju has joined #openstack-swift | 10:06 | |
*** haomaiw__ has quit IRC | 10:15 | |
*** haomaiwang has joined #openstack-swift | 10:16 | |
*** haomaiw__ has joined #openstack-swift | 10:23 | |
*** Honghui has joined #openstack-swift | 10:24 | |
*** haomaiwang has quit IRC | 10:26 | |
*** mmcardle1 has quit IRC | 10:34 | |
*** mmcardle has joined #openstack-swift | 10:34 | |
*** mlipchuk has quit IRC | 10:38 | |
*** hipster has joined #openstack-swift | 10:41 | |
*** nacim has quit IRC | 10:44 | |
*** Honghui has quit IRC | 10:44 | |
*** mmcardle has quit IRC | 10:45 | |
*** hipster has quit IRC | 10:49 | |
*** mlipchuk has joined #openstack-swift | 10:53 | |
*** nacim has joined #openstack-swift | 10:58 | |
openstackgerrit | Constantine Peresypkin proposed a change to openstack/swift: account to account copy implementation https://review.openstack.org/72157 | 11:04 |
*** ppai has quit IRC | 11:05 | |
*** dvas has quit IRC | 11:07 | |
*** dvas has joined #openstack-swift | 11:13 | |
openstackgerrit | Christian Schwede proposed a change to openstack/python-swiftclient: Fix Python3 bugs https://review.openstack.org/94347 | 11:13 |
*** nosnos has quit IRC | 11:16 | |
*** ppai has joined #openstack-swift | 11:18 | |
*** shague_ has quit IRC | 11:19 | |
*** shague_ has joined #openstack-swift | 11:19 | |
*** dmorita has quit IRC | 11:21 | |
*** otoolee has joined #openstack-swift | 11:30 | |
*** sungju has quit IRC | 11:37 | |
*** Honghui has joined #openstack-swift | 11:44 | |
*** psharma has quit IRC | 11:48 | |
*** ppai has quit IRC | 11:50 | |
*** bvandenh has joined #openstack-swift | 11:59 | |
acoles | dfg: short answer is 'yes' :) i just added comment to review - would still be good to avoid the possible exceptions when disallowed section paths aren't found. thx | 12:01 |
*** ppai has joined #openstack-swift | 12:04 | |
openstackgerrit | A change was merged to openstack/swift: Change the default token logged length to 16 https://review.openstack.org/94506 | 12:08 |
*** tdasilva has joined #openstack-swift | 12:11 | |
*** Honghui has quit IRC | 12:21 | |
*** shague_ has left #openstack-swift | 12:22 | |
*** ppai has quit IRC | 13:10 | |
acoles | cschwede_: hi | 13:15 |
cschwede_ | Hi Alistair! | 13:20 |
acoles | cschwede_: hi. re the py 3 bugs patch - the header encoding part looks fine but i just noticed there is no unit test for the change to set content-type='' | 13:22 |
acoles | cschwede_: are you happy if i push a ne wpatch set adding just that test? | 13:22 |
acoles | new patch | 13:23 |
cschwede_ | acoles: new patch or just update the current patchset? I’m fine if you want to add anything on top of it | 13:24 |
acoles | cschwede_: sorry, update. ok, will do. | 13:25 |
cschwede_ | acoles: of course i can also do this if you prefer | 13:27 |
acoles | cschwede_: i have it ready to go | 13:28 |
cschwede_ | acoles: great, thanks a lot! | 13:28 |
openstackgerrit | gholt proposed a change to openstack/swift: Container sync no longer sending swift_bytes value https://review.openstack.org/94465 | 13:32 |
*** hipster has joined #openstack-swift | 13:49 | |
*** dvas has quit IRC | 13:51 | |
*** dvas_ has joined #openstack-swift | 13:51 | |
openstackgerrit | Alistair Coles proposed a change to openstack/python-swiftclient: Fix Python3 bugs https://review.openstack.org/94347 | 13:54 |
acoles | cschwede_: ^^ i hope i didn't break anything! | 13:55 |
*** zaitcev has joined #openstack-swift | 14:00 | |
*** ChanServ sets mode: +v zaitcev | 14:00 | |
openstackgerrit | Alistair Coles proposed a change to openstack/python-swiftclient: Add keystone v3 auth support https://review.openstack.org/91788 | 14:09 |
*** dvas_ has quit IRC | 14:13 | |
cschwede_ | acoles: thanks, will have a look soon (currently in a meeting) | 14:15 |
*** ophuk has quit IRC | 14:20 | |
*** dvas_ has joined #openstack-swift | 14:29 | |
*** shakamunyi has joined #openstack-swift | 14:30 | |
*** dvas__ has joined #openstack-swift | 14:36 | |
*** dvas_ has quit IRC | 14:36 | |
*** haomaiw__ has quit IRC | 14:37 | |
*** haomaiwa_ has joined #openstack-swift | 14:37 | |
*** haomaiw__ has joined #openstack-swift | 14:39 | |
*** dvas__ has quit IRC | 14:42 | |
*** haomaiwa_ has quit IRC | 14:42 | |
*** dvas__ has joined #openstack-swift | 14:43 | |
*** dvas__ has quit IRC | 14:47 | |
*** kevinc_ has joined #openstack-swift | 15:01 | |
*** nacim has quit IRC | 15:05 | |
notmyname | good morning world | 15:13 |
*** dvas__ has joined #openstack-swift | 15:14 | |
*** lpabon has joined #openstack-swift | 15:15 | |
notmyname | ugh 16k keystone tokens. not a good way to start the day ;-) | 15:16 |
notmyname | creiht: I love the "if you *only* have 10k req/sec" to swift :-) | 15:17 |
notmyname | it's absolutely the right way to phrase it! | 15:17 |
creiht | :) | 15:18 |
creiht | notmyname: sorry to bring it up | 15:18 |
notmyname | creiht: actually, I'm glad you did post that to the ML | 15:18 |
creiht | but it becomes more concerning to me now that others have promised that we will be running keystone | 15:18 |
notmyname | it's the same story as "hey you are trying to make a one-size-fits-all solution, and you're completely ignoring a major use case" | 15:19 |
cschwede_ | hmm, can’t remember which talk it was, but for many deployments the average object size is 30-50kb - maybe? so 1/3 of the traffic would be only authentication :-/ | 15:19 |
creiht | yeah even for us, average object size is pretty small | 15:20 |
gholt | And our traffic is mostly reads, not writes. And probably half those reads are HEADs. | 15:21 |
creiht | right | 15:21 |
cschwede_ | gholt: ouch! | 15:21 |
creiht | makes it also really bad for all the libs/sdks that do silly things like do a head before every single operation | 15:22 |
notmyname | PKI is a nice idea, as long as it isn't the current keystone variant of "encrypt the entire service catalog for every request". singed requests are a pretty cool thing (see tempurl) | 15:22 |
creiht | lol | 15:22 |
notmyname | signed requests, too :-) | 15:22 |
creiht | I like my requests slightly singed :) | 15:22 |
redbo | don't worry, soon they'll just require kerberos auth for all clients | 15:23 |
notmyname | redbo: that's not a completely horrible idea, actually ;-) | 15:23 |
scotticus | hey its worked since the 70s right? | 15:23 |
notmyname | shoot keystone in the head and say "just use kerberos" :-) | 15:23 |
redbo | ahh! | 15:24 |
scotticus | i'm going to rename my go keystone bindings KeithStone. | 15:24 |
portante | three headed monster vs. hydra | 15:24 |
notmyname | scotticus: heh | 15:24 |
notmyname | redbo: note that I said it wasn't completely horrible. not that it was good :-) | 15:25 |
scotticus | there are still horrible aspects? | 15:25 |
*** bvandenh has quit IRC | 15:25 | |
gholt | My guess is that Keystone just can't handle very many requests per second and that's what the bloated tokens are for. Seems the wrong way to scale Keystone though. Heh | 15:26 |
notmyname | ok, before I respond and say something stupid on the ML, how can you compress PKI tokens and get any benefit? isnt' crypto data non-compressable? | 15:31 |
cschwede_ | notmyname: i would be very concerned if it is compressable | 15:31 |
creiht | notmyname: redbo just mentioned that it is likely because it is then base64 encoded or something like that to put in the header | 15:32 |
creiht | I did a quick test of an example token from the docs | 15:32 |
cschwede_ | notmyname: i’m no keystone expert, but maybe pki tokens with a subset of services (ie only swift) might be possible= | 15:32 |
cschwede_ | s/=/? | 15:32 |
creiht | but then wouldn't you have to encode that back again | 15:32 |
creiht | so yeah not sure how that would actually help | 15:32 |
notmyname | cschwede_: not really. see RAX and the fact that "just" swift services has today at least 6 entries in the catalog | 15:32 |
creiht | more than that | 15:33 |
creiht | because of cdn stuff | 15:33 |
notmyname | ah yes | 15:33 |
cschwede_ | ok, would have been too easy | 15:34 |
cschwede_ | hmm, maybe i’m still too jetlagged. „we“ are in charge of the keystone middleware for swift, right? | 15:36 |
notmyname | cschwede_: half of it | 15:36 |
cschwede_ | so - (probably i miss something here) - let me draft an idea | 15:37 |
notmyname | cschwede_: the keystone part basically builds an identity data structure and puts it in the wsgi env. the swift part implements the authorize() and interprets the identity data structure in the context of the swift ACL info | 15:38 |
redbo | I'm still holding a grudge because keystone gets to store things in memcache with pickles and for us it was some huge security problem. | 15:38 |
notmyname | I like that pattern, actually. we use the same one for the swiftstack AD and LDAP integration | 15:38 |
cschwede_ | 1. user sends data to swift for auth 2. swift asks (via middleware) keystone 3. keystone responds with a mb-sized token 4. middleware hashes that token, uses the hash as key for memcache and stores token as memcache value 5. middleware returns hashed token to user 6. user uses only the short token | 15:38 |
*** krtaylor has quit IRC | 15:38 | |
cschwede_ | of course client needs to be aware of that too. hmm. and caching introduces maybe problems due to expired tokens… | 15:39 |
gholt | redbo: You should file a bug for that! Under an alias of course. ;) | 15:39 |
notmyname | cschwede_: ya, you've just described tempauth | 15:39 |
cschwede_ | notmyname: ah, i’ve seen that before ;) | 15:39 |
notmyname | cschwede_: the problem is that for swift the client would first auth against swift and not keystone | 15:39 |
redbo | in fact, just forget keystone | 15:40 |
notmyname | cschwede_: which works, of course. just that it's different that all the existing docs and client behavior | 15:40 |
cschwede_ | notmyname: yes, that’s the problem (the different behavior). | 15:40 |
gholt | With compression, you compress before you encrypt. But anyway, stupid non-solution to the real problem. | 15:42 |
creiht | gholt: yeah | 15:42 |
notmyname | gholt: isn't that the basis of some recent attack? BEAST or CRIME or something | 15:42 |
redbo | we haven't even mentioned that validating tokens requires shelling out to openssl.. how many times per second? | 15:42 |
creiht | heh that's a different issue | 15:43 |
*** ophuk has joined #openstack-swift | 15:43 | |
notmyname | redbo: switch to libressl? /troll | 15:43 |
gholt | notmyname: Probably (on the crypto compression attack), heh | 15:45 |
notmyname | looks like my patch I threw over the wall last night was popular and is already merged (to by default limit the number of chars logged on the auth token) | 15:45 |
notmyname | ...speaking of auth tokens ^^ | 15:45 |
notmyname | cschwede_: do you know if there can be any sort of shared secret with keystone? how possible would it be to move from bearer tokens to signed requests for keystone+swift? | 15:47 |
notmyname | cause I feel we have the worst of both right now | 15:48 |
openstackgerrit | David Goetz proposed a change to openstack/swift: Add ability to remove subsections from /info https://review.openstack.org/94458 | 15:49 |
cschwede_ | notmyname: sorry, don’t know (yet). i think i need to digg deeper into keystone | 15:49 |
cschwede_ | notmyname: with signed requests you have something similar to s3 in mind? | 15:50 |
*** gyee has joined #openstack-swift | 15:50 | |
*** igor has quit IRC | 15:50 | |
notmyname | cschwede_: yes. or tempurl. same concept | 15:51 |
*** igor__ has joined #openstack-swift | 15:51 | |
notmyname | redbo: https://twitter.com/johnleach/status/469142817781338112 <<-- let's solve the overhead problem by using SOAP! | 15:51 |
cschwede_ | notmyname: well, it all depends on the client, right? so if we use a different way on the client side we would be able to use what we want, aren’t we? | 15:52 |
redbo | unfortunately when we do things that are different but we think better, we get in trouble | 15:52 |
notmyname | cschwede_: ya. so "use signed requests" probably reduces down to "make everyone use signed requests". which sounds hard | 15:52 |
redbo | I'm glad nobody is paying attention, they'd probably get mad about tempurl. | 15:53 |
notmyname | http://adam.younglogic.com/2014/02/compressed-tokens/ | 15:54 |
*** kevinc_ has quit IRC | 15:59 | |
*** kevinc_ has joined #openstack-swift | 16:01 | |
creiht | notmyname: so I guess that brings us to the point of having to ask, at what point are tokens too big? | 16:12 |
notmyname | creiht: 128 bytes should be enough for anyone, right? | 16:13 |
creiht | I think 10% encrypted size is a bit optimistic, but that's another thing | 16:13 |
notmyname | (actually sercious) | 16:13 |
creiht | heh | 16:13 |
notmyname | ya, I'm writing a response now | 16:13 |
creiht | I'm installing devstack to play with pki tokens a bit | 16:13 |
notmyname | cool | 16:13 |
creiht | will probably be a bit though | 16:14 |
creiht | and it is lunch time :) | 16:14 |
ophuk | so I moved my mount point from / to the normal default mount point and everything is working. Is there anywhere else other than account-server that you would need to set devices to tell it where to look for the mount point? | 16:15 |
notmyname | ophuk: there's a "devices" config setting in all of the storage server configs (account, container, object) | 16:19 |
*** nshaikh has quit IRC | 16:20 | |
ophuk | notmyname: oh - must of missed it when I looked for it on the example configs. | 16:21 |
openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/python-swiftclient: Updated from global requirements https://review.openstack.org/89250 | 16:28 |
cschwede_ | notmyname: I added two links to https://wiki.openstack.org/wiki/Swift/PriorityReviews#Looking_for_something_to_review.3F that might be useful - think it’s a nice addition to Gerrit. Feel free to edit if this is not the right place, misleading etc. | 16:29 |
notmyname | cschwede_: thanks | 16:29 |
openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 16:29 |
notmyname | cschwede_: ah, cool. those are great | 16:29 |
openstackgerrit | Christian Schwede proposed a change to openstack/python-swiftclient: fixed several pep8 issues https://review.openstack.org/93519 | 16:32 |
notmyname | I like how openstack global requirements requires a newer version of swiftclient than swift itself does | 16:34 |
openstackgerrit | John Dickinson proposed a change to openstack/swift: taking the global reqs that we can https://review.openstack.org/94669 | 16:37 |
notmyname | can someone double check that for me? ^ | 16:37 |
openstackgerrit | Mark Seger proposed a change to openstack/swift: added Benchmarks/Load Generators section + link to getput https://review.openstack.org/94670 | 16:39 |
*** mlipchuk has quit IRC | 16:40 | |
*** miqui has quit IRC | 16:47 | |
openstackgerrit | John Dickinson proposed a change to openstack/swift: added Benchmarks/Load Generators section https://review.openstack.org/94670 | 16:49 |
*** miqui has joined #openstack-swift | 16:50 | |
notmyname | reminder that the swift team meeting is in 2 hours in #openstack-meeting. we'll do a summit recap and look at the TODO things that came from it: https://wiki.openstack.org/wiki/Meetings/Swift | 16:53 |
*** haomaiw__ has quit IRC | 17:02 | |
*** saschpe has quit IRC | 17:08 | |
*** kenhui has joined #openstack-swift | 17:09 | |
*** saschpe has joined #openstack-swift | 17:10 | |
*** kenhui has quit IRC | 17:21 | |
*** kenhui has joined #openstack-swift | 17:22 | |
*** shri has joined #openstack-swift | 17:26 | |
*** kenhui has quit IRC | 17:28 | |
*** kenhui has joined #openstack-swift | 17:29 | |
openstackgerrit | A change was merged to openstack/swift: Add targeted config loading to swift-init https://review.openstack.org/92933 | 17:43 |
*** kenhui has quit IRC | 17:44 | |
*** kevinc_ has quit IRC | 17:48 | |
anticw | clayg / notmyname: the object-server middleware has timeout handling ... i assume this is needed because it's at the top/left of the pipeline? | 18:00 |
anticw | other middleware stages simply won't get __call__ invoked in cases where there is no data? | 18:00 |
notmyname | anticw: object server middleware? | 18:00 |
anticw | https://github.com/openstack/swift/blob/master/swift/obj/server.py#L412 | 18:01 |
anticw | ie. in the PUT method there is handling for .read timeouts - is that required for any middleware stage which consumes data via .read - or only just the top/left most? | 18:01 |
anticw | (and is it even required for swob at all? i can see it might be possible for the caller to deal with timeouts itself) | 18:02 |
notmyname | anticw: that code is in the server, not the middleware. so it's on the bottom/right. still thinking about your other question | 18:02 |
anticw | notmyname: it's the only-stage isn't it? | 18:02 |
anticw | so top/left = bottom/right surely? | 18:02 |
notmyname | storage nodes can (and often, at least with new cluster) have healthcheck and recon | 18:03 |
*** kenhui has joined #openstack-swift | 18:03 | |
anticw | notmyname: i'm wondering if that timeout code predates swob and is still needed though | 18:04 |
notmyname | anticw: I don't remember if swob does it or not. but in general if you're pulling data off the wsgi.input, then you should handle timeouts in case the client disconnects. I think. | 18:05 |
anticw | notmyname: we're pulling data from wsgi.input ... which is why i was asking the other day for a .peek vs .read | 18:05 |
notmyname | :-) | 18:05 |
anticw | the solution seems to be dat = .read(...) then later on wsgi.input = StringIO(data) | 18:05 |
anticw | i'd love to know there is a cleaner way though | 18:06 |
*** erlon has joined #openstack-swift | 18:06 | |
notmyname | redbo: is normally the person I ask for wsgi/swob stuff :-) | 18:06 |
*** acoles is now known as acoles_away | 18:08 | |
anticw | i'm going to suggest for now they leave the timeout handling there and then see if i can tickle a slow-write/timeout later today | 18:08 |
notmyname | anticw: remind me of the peek use case? look at the data but leave it in the bufer? | 18:08 |
anticw | NSA hooks | 18:09 |
notmyname | of course | 18:09 |
notmyname | /me kickbans anticw ;-) | 18:09 |
anticw | actually, different checksums :) | 18:09 |
creiht | lol | 18:09 |
notmyname | creiht: you'd probably have a good perspective on this | 18:10 |
creiht | heh | 18:10 |
notmyname | anticw: can you compute it without the whole buffer (is it a streaming thing?) | 18:10 |
anticw | well, you can do it with the whole bugger using body.req ... but then you consume a lot of ram and fall over for large PUTs | 18:11 |
anticw | so it really has to be incremental | 18:11 |
notmyname | right | 18:11 |
anticw | there isn't anything amazing here going on, let me ask them to just push the code out for the world to see | 18:11 |
notmyname | creiht: what do you think about a .peek()? | 18:11 |
creiht | seems problematic | 18:12 |
creiht | but somewhat related | 18:12 |
anticw | .peek isn't wsgi spec | 18:12 |
anticw | so i'm not sure it makes sense | 18:12 |
creiht | well let me rethink that | 18:13 |
notmyname | basically you need read() + tell() within the buffered chunk | 18:13 |
notmyname | right? | 18:13 |
creiht | it has been a while since I have messed with | 18:13 |
creiht | notmyname: yeah I was kinda thinking it might be nice to have a generic "observer" middleware that others could subclass to do things that want to watch the body or whatever | 18:13 |
notmyname | if you have to rebuild the wsgi.input do you have to create a new wsgi request? | 18:13 |
creiht | because otherwise it is kind of a pain to get right | 18:13 |
anticw | lemme quickly pastebin what i think they are doing ... i checked the code they have done and it's a bit hard to follow | 18:14 |
creiht | and yeah I'm not sure off the top of my head how doable it is | 18:14 |
notmyname | creiht: whoa. look at you going all "let's do some pluggable extensible framework" think ;-) | 18:14 |
notmyname | *thing | 18:14 |
notmyname | creiht: can it be middleware-middleware? | 18:14 |
creiht | yeah not sure you can even make it work as I'm not sure if you can send the chunks through the middleware pipeline | 18:15 |
notmyname | yo need a tee middleware | 18:15 |
creiht | it has been a while since I have played with it | 18:15 |
*** dvas__ has quit IRC | 18:16 | |
anticw | creiht: http://pastebin.com/XhrN1KcM | 18:17 |
creiht | but that's why I was saying that it might need a separate hook | 18:17 |
anticw | pseudo-pseudo code to explain what i think should be going on | 18:17 |
creiht | anticw: the usual problem with that type of pattern is you end up putting all of the object in memory | 18:20 |
anticw | creiht: originally that's what it did and blew up | 18:21 |
anticw | i'm hoping in this case data is only a limited amount of data ... a few kB to mB | 18:21 |
anticw | i just asked and am told this works - but it's not clear from their code if it is holding references to all of this | 18:22 |
anticw | oh, and in related/other news --- if you get a MemoryError exception in swift pipelines ... you get no useful logs :) | 18:22 |
creiht | heh | 18:23 |
anticw | creiht: https://github.com/openstack/swift/blob/master/swift/obj/server.py#L418 | 18:24 |
creiht | yeah sorry I'm not much help here | 18:24 |
creiht | I'm kinda in the middle of something myself | 18:24 |
openstackgerrit | Pete Zaitcev proposed a change to openstack/swift: Pluggable Back-ends for account and container servers https://review.openstack.org/47713 | 18:25 |
anticw | creiht: np, much appreciated all the same ... notmyname too! | 18:25 |
creiht | I would have to take some time to do some experimentation myself | 18:25 |
*** gvernik has joined #openstack-swift | 18:28 | |
notmyname | meeting in 30 minutes | 18:31 |
*** igor__ has quit IRC | 18:34 | |
*** igor_ has joined #openstack-swift | 18:35 | |
creiht | notmyname: so a quick hack test on devstack with compression token goes from 5974 bytes to 1160 bytes | 18:36 |
creiht | just fyi | 18:36 |
notmyname | creiht: ah, good info. thanks | 18:36 |
anticw | pki tokens? | 18:36 |
notmyname | so 1:5 (ish) | 18:36 |
creiht | yeah | 18:36 |
anticw | rackspace and swiftstack don't use those... | 18:36 |
anticw | and those people who do end up with a lot of complexity when it comes to revocation | 18:37 |
creiht | heh yeah there are still a ton of other issues | 18:37 |
anticw | i'm more or less convinced pki tokens are a solution in search of even more things to break | 18:37 |
creiht | lol | 18:37 |
notmyname | anticw: did you see the ML thread creiht started this morning? http://lists.openstack.org/pipermail/openstack-dev/2014-May/035463.html | 18:38 |
anticw | once you start dealing with some of the revocation and other stuff various things require, you end up doing excatly the things pki token advocates claim you avoid | 18:38 |
anticw | notmyname: i didn't reading now | 18:38 |
notmyname | anticw: and Rackspace, during a keynote at the openstack summit, promised to start using "real" keystone | 18:38 |
anticw | but since this came up | 18:38 |
anticw | let me put this gently | 18:38 |
creiht | anticw: yeah thanks for the input | 18:39 |
anticw | PKI TOKENS ARE RETARDED AND SHOULD DIE | 18:39 |
notmyname | :-) | 18:39 |
creiht | lol | 18:39 |
anticw | notmyname: i talked to joearnold about this in passing | 18:39 |
anticw | i want to talk to you more ... about a new mac-based auth scheme | 18:39 |
anticw | similar to what aws does | 18:39 |
anticw | ... are you free for lunch? | 18:39 |
notmyname | anticw: this week? maybe friday. | 18:40 |
anticw | the goal would be to avoid replay-attacks and concerns about tokens leaking | 18:40 |
notmyname | right | 18:40 |
notmyname | so bearer tokens are out | 18:40 |
anticw | like in 20 minutes :) my date cancelled on me! | 18:40 |
notmyname | anticw: heh. no, not today. swift team meeting (and I'm sitting at home with a broken collar bone instead of being in the office) | 18:41 |
ctennis | was your date someone on the keystone team? | 18:41 |
zaitcev | what happened man, this sounds crazy | 18:41 |
anticw | that sucks ... i hope there is a good story to that? | 18:41 |
anticw | ctennis: unlike most people around SF ... i do not luvs the keystone | 18:42 |
notmyname | zaitcev: anticw: unfortunately no. biking home and my feet slipped off the pedals. cracked my helmet and broke a bone | 18:42 |
zaitcev | aww | 18:42 |
*** acoles_away is now known as acoles | 18:42 | |
zaitcev | and my doc called me with MRI results and referred to surgeon, set after hackathon | 18:42 |
anticw | notmyname: ok, that's a crappy story ... never tell it again ... make up some stuff about defending the nation from rogue terrorists | 18:42 |
notmyname | heh. ya, my wife said something similar :-) | 18:43 |
ctennis | some of us choose to believe there was a disagreement on the storage policy implementation and you lost | 18:43 |
notmyname | lol | 18:43 |
notmyname | peluse_...persuaded....me to choose his implementation :-) | 18:43 |
notmyname | speaking of...time to go take more advil | 18:44 |
zaitcev | yeah, ibuprofen has anti-swelling properties | 18:45 |
openstackgerrit | A change was merged to openstack/swift: added Benchmarks/Load Generators section https://review.openstack.org/94670 | 18:54 |
openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 18:55 |
*** krtaylor has joined #openstack-swift | 18:55 | |
*** shakamunyi has quit IRC | 18:57 | |
zaitcev | clayg: I implemented the spirit of your "missive", I think, but there's a side effect: I cannot split up the big patch as I hoped anymore, so I'll ask you to review the https://review.openstack.org/47713 | 18:58 |
zaitcev | clayg: before, there was inheritance, so I hoped to do Step 1: just create the class, Step 2: change methods for HEAD, Step 3: change methods for PUT, etc. | 19:00 |
notmyname | meeting time in #openstack-meeting | 19:00 |
zaitcev | umm, I need to run | 19:00 |
*** cds has quit IRC | 19:00 | |
zaitcev | I'll read the meeting log later | 19:00 |
notmyname | zaitcev: kk | 19:00 |
*** kevinc_ has joined #openstack-swift | 19:04 | |
*** kenhui has quit IRC | 19:07 | |
*** kenhui has joined #openstack-swift | 19:07 | |
*** ChanServ changes topic to ""Swift: It's not just another object storage" -- cschwede" | 19:15 | |
*** kevinc_ has quit IRC | 19:17 | |
*** kenhui has quit IRC | 19:20 | |
*** kenhui has joined #openstack-swift | 19:22 | |
*** schofield has joined #openstack-swift | 19:28 | |
*** dvas___ has joined #openstack-swift | 19:34 | |
*** gvernik has left #openstack-swift | 19:47 | |
*** kenhui has quit IRC | 19:48 | |
*** kevinc_ has joined #openstack-swift | 19:49 | |
*** acoles is now known as acoles_away | 19:50 | |
dfg | clayg: you there? | 19:55 |
notmyname | dfg: he's at gluecon today so is only in and out online | 19:58 |
dfg | gluecon? | 20:00 |
dfg | is that where horses go? is clay that old? | 20:00 |
dfg | alright- anyway- i'll talk to him later | 20:00 |
glange | they shoot horses, don't they? | 20:03 |
*** tdasilva has left #openstack-swift | 20:03 | |
*** lpabon has quit IRC | 20:04 | |
*** schofield has left #openstack-swift | 20:04 | |
*** jeblair has joined #openstack-swift | 20:16 | |
openstackgerrit | David Goetz proposed a change to openstack/swift: xLO bug with auth tokens expiring during download. https://review.openstack.org/92165 | 20:25 |
*** r-daneel has joined #openstack-swift | 20:39 | |
*** pberis has quit IRC | 20:52 | |
*** fifieldt has joined #openstack-swift | 20:57 | |
*** kevinc_ has quit IRC | 20:59 | |
*** pberis has joined #openstack-swift | 21:10 | |
*** dvas___ has quit IRC | 21:38 | |
*** dvas has joined #openstack-swift | 21:38 | |
*** kevinc_ has joined #openstack-swift | 21:39 | |
*** jamie_h has quit IRC | 21:42 | |
*** kenhui has joined #openstack-swift | 21:44 | |
*** dvas has quit IRC | 21:45 | |
*** pberis has quit IRC | 21:49 | |
*** hipster has quit IRC | 21:55 | |
*** krtaylor has quit IRC | 22:05 | |
*** cihhan has joined #openstack-swift | 22:07 | |
cihhan | Hi all! I have installed keystone and swift using: http://docs.openstack.org/havana/install-guide/install/apt/content/index.html -- However, I have two questions: How can I create multiple users? And how can I use SSL between the client and the proxy server (also between the proxy server and the storage nodes)? | 22:14 |
*** anticw_ has joined #openstack-swift | 22:27 | |
*** fbo_away has joined #openstack-swift | 22:29 | |
*** ondergetekende has quit IRC | 22:29 | |
*** Mikalv has quit IRC | 22:31 | |
*** Mikalv has joined #openstack-swift | 22:32 | |
*** portante has quit IRC | 22:33 | |
*** ondergetekende has joined #openstack-swift | 22:33 | |
*** notmyname_ has joined #openstack-swift | 22:33 | |
*** ChanServ sets mode: +v notmyname_ | 22:33 | |
*** anticw has quit IRC | 22:33 | |
*** gholt has quit IRC | 22:33 | |
*** fbo has quit IRC | 22:33 | |
*** notmyname has quit IRC | 22:33 | |
*** fbo_away is now known as fbo | 22:33 | |
*** notmyname_ is now known as notmyname | 22:34 | |
*** igor__ has joined #openstack-swift | 22:34 | |
*** acolesz has joined #openstack-swift | 22:35 | |
*** acolesz is now known as acoles | 22:35 | |
*** ChanServ sets mode: +v acoles | 22:35 | |
*** portante has joined #openstack-swift | 22:35 | |
*** ChanServ sets mode: +v portante | 22:35 | |
*** acoles_away has quit IRC | 22:38 | |
*** igor_ has quit IRC | 22:38 | |
*** gholt has joined #openstack-swift | 22:38 | |
*** ChanServ sets mode: +v gholt | 22:38 | |
*** kenhui has quit IRC | 22:40 | |
*** r-daneel has quit IRC | 22:41 | |
notmyname | cihhan: still here? I can help with that | 22:41 |
notmyname | zaitcev: isnt' WIP the -1 workflow now? | 22:42 |
zaitcev | notmyname: no idea, sorry. could be. | 22:42 |
cihhan | notmyname, yep still around :) that would be really great | 22:44 |
*** pberis has joined #openstack-swift | 22:45 | |
notmyname | cihhan: both work, but why are you using havana docs to install and not icehouse docs? | 22:46 |
cihhan | notmyname: i think i gave the wrong one, i used icehouse documentation. | 22:46 |
notmyname | cihhan: ok, just checking that there aren't special circumstances | 22:47 |
cihhan | notmyname: btw, i think both are the same | 22:47 |
notmyname | cihhan: did you get everything installed ok? your 2 questions seemed to be post-install questions | 22:47 |
*** pberis has quit IRC | 22:48 | |
cihhan | notmyname: one one VM, i have installed keystone and swift proxy and on 2 other VMs, i have installed storage (im using VMs bcs i can snapshot if i make some big mistake) | 22:48 |
*** pconstantine_ has quit IRC | 22:49 | |
*** omame has quit IRC | 22:49 | |
cihhan | notmyname, do i need anything more? | 22:50 |
cihhan | notmyname, based on the documentation, that should be all; right? | 22:50 |
notmyname | cihhan: when you say you have installed "storage" what do you mean? | 22:52 |
notmyname | on the 2 other VMs | 22:52 |
cihhan | notmyname, i mean this part: http://docs.openstack.org/icehouse/install-guide/install/apt/content/installing-and-configuring-storage-nodes.html | 22:53 |
notmyname | cihhan: cool | 22:53 |
notmyname | cihhan: how you done any api requests to it? is it working? | 22:54 |
cihhan | from proxy i can do 'swift list/upload/download', is that what you mean? | 22:55 |
notmyname | cihhan: ya. or even `curl -i http://swift.whatever/healthcheck` | 22:56 |
cihhan | i havent checked curl -- let me try it now | 22:56 |
cihhan | for curl, on proxy, i m trying this 'curl -i http://PROXY_IP/healthcheck' | 22:58 |
cihhan | but it s waiting | 22:58 |
notmyname | cihhan: did you configure ssl? | 22:59 |
cihhan | notmyname, nope not yet | 22:59 |
notmyname | cihhan: what port is it listening on? | 23:02 |
cihhan | based on proxy-server.conf, it is 8080 | 23:03 |
notmyname | cihhan: then that's your issue above. do `curl -i http://swift:8080/healthcheck` | 23:04 |
cihhan | notmyname, i think there is something wrong with my configuration. when i do 'netstat -a | grep swift', i dont get anything | 23:05 |
cihhan | notmyname, is that usual? | 23:07 |
notmyname | cihhan: are your swift processes running? | 23:08 |
cihhan | notmyname, yes seems to be | 23:08 |
cihhan | at least i can see swift-proxy in ps and swift list myfiles give me the right output | 23:09 |
notmyname | did you try the /healthcheck request to the right port? | 23:09 |
notmyname | oh, ok. then you're good | 23:09 |
cihhan | notmyname, i wonder if i did everything correctly since curl didnt work | 23:10 |
*** pconstantine has joined #openstack-swift | 23:10 | |
*** nosnos has joined #openstack-swift | 23:10 | |
notmyname | cihhan: can you pastebin what you are seeing? | 23:11 |
notmyname | what's the IP and port of your proxy server? | 23:11 |
cihhan | notmyname, i cant share the IP since it would cause me headache, but let me pastebin the proxy-server.conf -- would that be enough? | 23:11 |
notmyname | cihhan: not right now | 23:12 |
notmyname | cihhan: if the swift CLI tool is giving you good output, then it works. I want to make sure the curl stuff works too (since that's the "real" API) | 23:12 |
*** omame has joined #openstack-swift | 23:14 | |
notmyname | cihhan: did you set up the envvars for the CLI? | 23:14 |
notmyname | if so, then `swift stat -v` should also work | 23:15 |
cihhan | notmyname, when i try 'curl -i http://swift.PROXY_IP_ADDRESS:8080/healthcheck', i get 'curl: (6) Could not resolve host: swift.PROXY_IP_ADDRESS' | 23:15 |
cihhan | notmyname, yeah swift stat works | 23:15 |
cihhan | but im trying it on the proxy | 23:16 |
notmyname | cihhan: are you using a hostname or the IP address. 'cause eg "swift.192.168.100.100" doesn't make any sense | 23:16 |
cihhan | im using IP | 23:16 |
notmyname | ok, so don't put a subdomain in front of the IP | 23:16 |
cihhan | in that case, it s just waiting forever | 23:17 |
notmyname | do you have the IP in an envvar? I'd like to be able to give you commands you can copy and paste without having to translate | 23:17 |
cihhan | i can write it in /etc/hosts and export as IP_ADDRESS if u wish | 23:17 |
notmyname | I think you'd only do one of those things. not both | 23:18 |
cihhan | notmyname, hang on | 23:18 |
cihhan | on proxy, when i try 'curl -i proxy:8080/healthcheck' it worked fine | 23:19 |
cihhan | by mistake i tried it on a client VM i created | 23:19 |
cihhan | that s why it took forever | 23:19 |
notmyname | ok | 23:19 |
cihhan | but on proxy it works fine | 23:19 |
notmyname | cihhan: ok. now I think we can get to your questions :-) | 23:19 |
notmyname | cihhan: first is about SSL, right? | 23:19 |
cihhan | notmyname, i think before that i have another question: how can i connect using a client? | 23:20 |
cihhan | by client, i mean another machine | 23:20 |
notmyname | cihhan: didn't you just do that? curl and the swift cli are clients | 23:20 |
notmyname | that sounds like a routing issue in your setup. if you can ping the server from another box, you should be able to connect | 23:21 |
cihhan | notmyname, as i mentioned, curl worked fine if i do it on the proxy. when i do it on the client vm (which is a vm without anything) it waits and says connection timed out | 23:21 |
notmyname | cihhan: right. so can you ping the proxy from the client vm? | 23:21 |
cihhan | yeah i can ping but somehow i cant ssh... there is something wrong with the routing as u mentioned i think | 23:22 |
cihhan | let me meanwhile check it | 23:22 |
cihhan | notmyname, corrected -- it was a small configuration issue -- now it s working fine | 23:23 |
notmyname | ok, great | 23:24 |
cihhan | notmyname, thanks a lot :) | 23:24 |
cihhan | notmyname, now we can check SSL and multiple users? :) | 23:24 |
notmyname | cihhan: ya. SSL is "easy" | 23:24 |
cihhan | notmyname, fingers crossed and listening to u :) | 23:24 |
notmyname | swift doesn't do ssl, so use an external tool to do that. it could be a load balancer (like haproxy), or a dedicated tool like stud or stunnel | 23:25 |
notmyname | cihhan: this is how every production swift cluster does it. terminate the ssl external to the swift process and the forward the connection to swift (either on the same box or not) | 23:25 |
cihhan | notmyname, hmmmm i saw some documents talking about ssl but they were not or at least i couldnt make them work :) | 23:26 |
cihhan | ok that makes sense. | 23:26 |
notmyname | cihhan: any other questions around that? | 23:26 |
cihhan | yes | 23:27 |
cihhan | i want to have multiple users if possible | 23:27 |
cihhan | can i do that? | 23:27 |
cihhan | im thinking like dropbox -- every user has a username and passwd | 23:27 |
notmyname | of course. | 23:28 |
notmyname | what auth system are you using? | 23:28 |
cihhan | keystone | 23:28 |
notmyname | (note that unless you are using other openstack components, there is not requirement in swift to use keystone) | 23:29 |
notmyname | cihhan: then it's simply a matter of adding the users to keystone, identifying with keystone, and using the creds passed back | 23:30 |
cihhan | i have tried that but couldnt do it | 23:30 |
notmyname | cihhan: and for that I'd point you to the keystone docs, since I don't know how to do that off the top of my head | 23:30 |
cihhan | maybe it was some misconf | 23:30 |
cihhan | what do u suggest other than keystone? | 23:30 |
cihhan | if keystone is not suggested in my case | 23:31 |
cihhan | i have installed keystone just for authentication services | 23:31 |
notmyname | cihhan: are you using other openstack components or just swift? | 23:31 |
notmyname | cihhan: and how are you integrating it in to your existing infrastructure? | 23:31 |
cihhan | im using just swift | 23:32 |
cihhan | i have vmware vsphere for hypervisor and im creating VMs to install swift on them -- i dont have multiple physical machines but i have a server available | 23:32 |
notmyname | cihhan: do you have requirements to integrate into any other identity system or is this a standalone service? | 23:32 |
notmyname | hmm..../me isn't a fan of running swift on virtual machines | 23:33 |
cihhan | this is mainly for testing purpose -- maybe later i can move to the phsyical machines | 23:33 |
notmyname | cihhan: if it's just for testing, then the simplest thing would be to use the included tempauth system | 23:34 |
notmyname | assuming you won't have much churn in the users nor have a lot of them (where "a lot" ~= 50) | 23:34 |
cihhan | well i dont think that i will have 50 clients -- but in case of large number, what should i do? | 23:35 |
notmyname | cihhan: again, depends if you need to integrate in to existing systems or not (eg AD or LDAP). if not, use something like swauth. if so, use something like swiftstack | 23:36 |
notmyname | (disclosure, I work for swiftstack) | 23:36 |
notmyname | cihhan: but for today, if you are looking at some kicking-the-tires sort of testing, then use the included tempauth | 23:39 |
cihhan | notmyname, the truth is im a student and swift will be used for the research purpose mainly and i dont have expertise in it. | 23:39 |
cihhan | well at least for now :) | 23:39 |
notmyname | cihhan: cool. what sort of research? | 23:39 |
cihhan | security :) | 23:40 |
*** jogo has joined #openstack-swift | 23:41 | |
cihhan | anyway, so i ll try tempauth first | 23:41 |
notmyname | cihhan: that's a broad subject. what specifically are you looking at? are you looking at swift or just uswing swift to store data? | 23:41 |
jogo | I got a question about your memcache client | 23:42 |
cihhan | data storage security and we will use swift for the storage | 23:42 |
jogo | any plans on spinning that out into a library as it would be a nice thing for other projects to use | 23:42 |
notmyname | jogo: "plans" is such a strong word :-) | 23:42 |
cihhan | notmyname, im not sure how much im supposed to say as a student without my advisor's approval, so im sort of trying to keep it abstract here | 23:43 |
notmyname | jogo: it works for us, and we haven't heard a clamoring for it in other places, so I don't think it's at the top of anyone's todo list | 23:43 |
*** kevinc_ has quit IRC | 23:44 | |
* jogo clamors | 23:44 | |
jogo | notmyname: I ask because I was thinking we should make devstack closer to production -- and for production we (I) recomend using a real memcache server instead of oslos junk | 23:45 |
notmyname | cihhan: that's ok. you should be able to set up tempauth by looking at the sample proxy config file. it's very well commented | 23:45 |
cihhan | notmyname, thanks a lot for the suggestions and your help :) | 23:45 |
jogo | so I noticed we already install memcached in devstack for swift and then I saw your nice memcache client | 23:45 |
jogo | which sounds really useful | 23:45 |
notmyname | jogo: in general, we're probably not too opposed to putting it in an external library. but you'll get some pushback if "external library" comes with hooks to other things. that is, we're not going to like replacing a simple py module with something that adds a lot of dependencies. or any, actually | 23:47 |
jogo | notmyname: no objections from me on that | 23:47 |
jogo | notmyname: I just want to use it in nova/oslo | 23:47 |
notmyname | jogo: "use it" is better than "include it in" | 23:48 |
notmyname | :-) | 23:48 |
jogo | notmyname: amen to that | 23:48 |
notmyname | jogo: so I guess I'm trying to say, "yes that might be something we'd like" :-) | 23:49 |
notmyname | or at least not be too opposed to | 23:49 |
notmyname | cihhan: glad to see you coming by with questions. don't be afraid to ask | 23:49 |
*** openstackgerrit has quit IRC | 23:49 | |
*** openstackgerrit has joined #openstack-swift | 23:50 | |
cihhan | notmyname, thanks a lot :) | 23:50 |
notmyname | cihhan: np. glad to help | 23:50 |
zaitcev | I'm having a problem with Devstack, apparently... http://logs.openstack.org/13/47713/16/check/check-swift-dsvm-functional/9b6998f/console.html | 23:50 |
*** csd has joined #openstack-swift | 23:53 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!