*** SkyRocknRoll has quit IRC | 00:04 | |
*** m_kazuhiro has joined #openstack-swift | 00:52 | |
m_kazuhiro | good morning | 00:52 |
---|---|---|
mattoliverau | m_kazuhiro: morning | 00:53 |
m_kazuhiro | mattoliverau: morning! | 00:53 |
*** dja has quit IRC | 01:04 | |
*** jamielennox|away is now known as jamielennox | 01:06 | |
*** stradling has quit IRC | 01:22 | |
*** rcernin has joined #openstack-swift | 01:58 | |
kota_ | morning | 02:13 |
*** vint_bra has joined #openstack-swift | 02:39 | |
*** klrmn has quit IRC | 02:41 | |
*** rcernin has quit IRC | 02:45 | |
*** catintheroof has joined #openstack-swift | 02:55 | |
*** klrmn has joined #openstack-swift | 03:03 | |
*** vint_bra has quit IRC | 03:12 | |
mattoliverau | kota_: morning | 03:15 |
kota_ | mattoliverau: o/ | 03:16 |
*** dja has joined #openstack-swift | 03:16 | |
*** dja has quit IRC | 03:38 | |
*** klrmn has quit IRC | 03:45 | |
*** chsc has joined #openstack-swift | 03:53 | |
*** chsc has joined #openstack-swift | 03:53 | |
*** dja has joined #openstack-swift | 03:55 | |
*** dja has quit IRC | 04:01 | |
*** Dinesh_Bhor has joined #openstack-swift | 04:01 | |
*** gkadam has joined #openstack-swift | 04:03 | |
*** catintheroof has quit IRC | 04:10 | |
*** chsc has quit IRC | 04:12 | |
*** psachin has joined #openstack-swift | 04:26 | |
*** rcernin has joined #openstack-swift | 04:57 | |
*** rcernin has quit IRC | 05:18 | |
*** gkadam has quit IRC | 05:20 | |
*** gkadam has joined #openstack-swift | 05:21 | |
*** rcernin has joined #openstack-swift | 05:31 | |
*** IRCFrEAK has joined #openstack-swift | 05:51 | |
*** IRCFrEAK has left #openstack-swift | 05:53 | |
*** adriant has quit IRC | 05:55 | |
*** ChubYann has quit IRC | 06:20 | |
*** cshastri has joined #openstack-swift | 06:25 | |
*** sputnik13 has quit IRC | 06:32 | |
*** sputnik13 has joined #openstack-swift | 06:34 | |
openstackgerrit | Matthew Oliver proposed openstack/swift master: Add container sharding to Swift containers https://review.openstack.org/423906 | 06:34 |
openstackgerrit | Matthew Oliver proposed openstack/swift master: Container sharding docs https://review.openstack.org/438322 | 06:34 |
*** tesseract has joined #openstack-swift | 06:34 | |
*** hoonetorg has quit IRC | 06:37 | |
*** dja has joined #openstack-swift | 06:41 | |
*** hseipp has joined #openstack-swift | 06:42 | |
*** oshritf has joined #openstack-swift | 06:47 | |
openstackgerrit | Merged openstack/swift master: Fix intermittent test_unlink_* failures https://review.openstack.org/451593 | 06:55 |
*** dja has quit IRC | 06:57 | |
*** pcaruana has joined #openstack-swift | 06:57 | |
*** jaosorior has joined #openstack-swift | 06:57 | |
*** geaaru has joined #openstack-swift | 07:15 | |
*** amoralej|off is now known as amoralej | 07:20 | |
acoles | good morning | 07:51 |
mattoliverau | acoles: morning o/ | 08:00 |
acoles | mattoliverau: o/ welcome to (your) winter time :) | 08:00 |
*** openstackgerrit has quit IRC | 08:03 | |
*** jordanP has joined #openstack-swift | 08:06 | |
*** cbartz has joined #openstack-swift | 08:15 | |
*** openstackgerrit has joined #openstack-swift | 08:24 | |
openstackgerrit | Mahati Chamarthy proposed openstack/swift master: Limit number of revert tombstone SSYNC requests https://review.openstack.org/439572 | 08:24 |
mattoliverau | acoles: thanks and it's been raining ll day so kinda feels like it too | 08:30 |
mattoliverau | acoles: I meged your sharding config changes into the parent. Still want to play with misplaced objects. | 08:30 |
mattoliverau | also thanks so much for all that reviewing | 08:31 |
acoles | mattoliverau: :( rain | 08:31 |
acoles | mattoliverau: ok cool. I ran out of time last week re misplaced objects, I think the test is sufficient to demonstrate the problem but I had no time to look t how easy it might be to workaround. Also the test could probably be fleshed out some more to cover more cases/more objects/more pivots??? consider it "scaffolding"! | 08:33 |
*** openstackgerrit has quit IRC | 08:33 | |
acoles | got me thinking about probe tests for this stuff | 08:33 |
mattoliverau | acoles: yeah totally, I figure scaffolding it a great start. | 08:34 |
acoles | mattoliverau: I hate to add to your gloomy day but it is beautiful spring sunshine here :P | 08:34 |
mattoliverau | yeah, I've been meaning to make a basic probe test that would get us containers in the sharding and sharding state so we can start testing more edge cases (there is a trello card for it) | 08:35 |
mattoliverau | acoles: lol, damn you :P | 08:35 |
acoles | mattoliverau: actually I think this past weekend has been what we get to call "summer" in its entireity | 08:35 |
mattoliverau | We've had a hurrican (way up north) but the weather has pushed alot of rain | 08:35 |
acoles | mattoliverau: I should check out the trello board, hope to give this more time over coming weeks | 08:36 |
mattoliverau | lol | 08:36 |
mattoliverau | I try and update it with things as we think of them, or as people (you and tim) bring up in reviews. | 08:36 |
acoles | good idea | 08:36 |
acoles | with big features like this its easy to lose track of comments/todos on gerrit as the patch versions roll on, best to keep a list somewhere else | 08:37 |
acoles | mattoliverau: looks like you and kota were making progress on patch 448480? | 08:38 |
mattoliverau | yeah, there are things I said 'great idea' too in earlier patchsets that I think I may have already forgotten about.. so yeah can see how easy that is to happen. | 08:38 |
patchbot | https://review.openstack.org/#/c/448480/ - swift - Container drive error results double space usage o... | 08:38 |
mattoliverau | yup, I wrote some probetests if you need an easy way to play with it. | 08:39 |
kota_ | acoles: o/ | 08:39 |
acoles | hi kota_ | 08:39 |
mattoliverau | there should be a paste linked in the review. | 08:40 |
* kota_ was concentrating another patch | 08:40 | |
mattoliverau | A spend a bit of this arvo looking at the composit ring stuff (want to have more of a play with that too) | 08:40 |
mattoliverau | acoles: seeing as you have a UUID patch for the ringbulder (re:composit rings) did you want to add a rebalance timestamp or something (last rebalance) at the same time.. that is if we want to compare when last rebalanced too.. tho maybe we only want to check if different no necessarily newer. | 08:40 |
kota_ | oops, sounds thunder in Tokyo | 08:40 |
acoles | mattoliverau: oic, a probe test? | 08:41 |
mattoliverau | 2, one to demonstraight the problem. the other to play with a edgecase that kota_ and I talked about in channel last Friday. | 08:41 |
acoles | mattoliverau: ok. | 08:42 |
mattoliverau | it should apply to the review patch. | 08:42 |
mattoliverau | k need to go do baby night rutine things now, bath, story and bed.. bbl. | 08:44 |
acoles | mattoliverau: rebalance timestamp or some other way to compare the state of rings might be useful (e.g. md5 of replica2part2dev). I'm hoping we can make progress (i.e. land something) and follow on with those ideas. TBH I am not convinced how much we should care about re-composing rings that have not changed now it is just an offline process, but it would be a nice check to have | 08:44 |
acoles | mattoliverau: k have a good evening | 08:44 |
mattoliverau | acoles: yeah, good point (re:being offline). Still want to make it as fool proof as possible, cause I can be pretty foolish ;) | 08:45 |
acoles | hehe | 08:45 |
* acoles going for coffee | 08:45 | |
mahatic | acoles: mattoliverau o/ | 08:49 |
mahatic | kota_: o/ | 08:49 |
mattoliverau | mahatic: o/ | 08:52 |
*** gkadam_ has joined #openstack-swift | 08:52 | |
*** gkadam has quit IRC | 08:53 | |
kota_ | mahatic: o/ | 08:53 |
*** mtreinish has quit IRC | 08:53 | |
*** mtreinish has joined #openstack-swift | 08:57 | |
acoles | mahatic: welcome back, did you have a good break? | 09:00 |
mahatic | acoles: thanks. yup, did some hiking in the hills. away from the Bangalore summer :) | 09:03 |
acoles | nice | 09:03 |
*** szaher has quit IRC | 09:25 | |
*** szaher has joined #openstack-swift | 09:30 | |
cbartz | Anyone here who can review patch 414232 ? Thx.... | 09:31 |
patchbot | https://review.openstack.org/#/c/414232/ - swift - Exclude containers for account quota | 09:31 |
*** szaher has quit IRC | 09:33 | |
*** szaher has joined #openstack-swift | 09:39 | |
onovy | funny bug: https://bugs.launchpad.net/swift/+bug/1678976 | 09:51 |
openstack | Launchpad bug 1678976 in OpenStack Object Storage (swift) "POST with Range header corrupts data without fast-POST" [Undecided,New] | 09:51 |
*** silor has joined #openstack-swift | 09:55 | |
*** sams-gleb has joined #openstack-swift | 09:59 | |
*** m_kazuhiro has quit IRC | 10:00 | |
*** szaher has quit IRC | 10:01 | |
*** silor has quit IRC | 10:03 | |
*** silor has joined #openstack-swift | 10:04 | |
*** szaher has joined #openstack-swift | 10:04 | |
*** rcernin has quit IRC | 10:05 | |
*** sams-gleb has quit IRC | 10:05 | |
*** oshritf has quit IRC | 10:05 | |
*** sams-gleb has joined #openstack-swift | 10:06 | |
*** hoonetorg has joined #openstack-swift | 10:07 | |
*** oshritf has joined #openstack-swift | 10:07 | |
*** szaher has quit IRC | 10:10 | |
*** sams-gleb has quit IRC | 10:10 | |
*** szaher has joined #openstack-swift | 10:11 | |
*** rcernin has joined #openstack-swift | 10:24 | |
*** oshritf has quit IRC | 10:31 | |
*** psachin has quit IRC | 10:34 | |
*** oshritf has joined #openstack-swift | 10:50 | |
*** psachin has joined #openstack-swift | 10:51 | |
*** oshritf has quit IRC | 10:52 | |
*** dja has joined #openstack-swift | 10:56 | |
*** oshritf has joined #openstack-swift | 11:01 | |
*** kei_yama has quit IRC | 11:04 | |
*** oshritf has quit IRC | 11:10 | |
*** oshritf has joined #openstack-swift | 11:32 | |
*** ma9_ has joined #openstack-swift | 11:45 | |
*** dja has quit IRC | 11:48 | |
*** oshritf has quit IRC | 11:48 | |
*** openstackgerrit has joined #openstack-swift | 11:49 | |
openstackgerrit | Ondřej Nový proposed openstack/swift master: Remove Range header from non-GET requests https://review.openstack.org/452726 | 11:49 |
*** oshritf has joined #openstack-swift | 11:52 | |
openstackgerrit | Ondřej Nový proposed openstack/swift master: Remove Range header from non-GET requests https://review.openstack.org/452726 | 11:54 |
*** sams-gleb has joined #openstack-swift | 11:56 | |
*** psachin has quit IRC | 11:58 | |
*** hseipp has quit IRC | 12:05 | |
*** psachin has joined #openstack-swift | 12:11 | |
*** silor1 has joined #openstack-swift | 12:15 | |
*** silor has quit IRC | 12:19 | |
*** silor1 is now known as silor | 12:19 | |
*** vint_bra has joined #openstack-swift | 12:27 | |
*** vint_bra has quit IRC | 12:29 | |
*** jaosorior is now known as jaosorior_brb | 12:36 | |
*** amoralej is now known as amoralej|lunch | 12:54 | |
*** adu has quit IRC | 13:13 | |
openstackgerrit | Alistair Coles proposed openstack/swift master: Fix UnicodeDecodeError in reconstructor _full_path function https://review.openstack.org/452750 | 13:14 |
acoles | clayg: ^^ I think you found a different bug to rledisez | 13:15 |
*** gkadam_ is now known as gkadam | 13:15 | |
acoles | rledisez: what version were you running when you saw bug 1678018 ? | 13:15 |
openstack | bug 1678018 in OpenStack Object Storage (swift) "ssync_sender encoding issue" [Undecided,Confirmed] https://launchpad.net/bugs/1678018 | 13:15 |
*** gkadam has quit IRC | 13:17 | |
*** stradling has joined #openstack-swift | 13:18 | |
rledisez | acoles: it was on swift 2.12. in the past we also ran 2.7 with the policy | 13:18 |
*** catintheroof has joined #openstack-swift | 13:31 | |
*** jaosorior_brb is now known as jaosorior | 13:33 | |
*** ma9_1 has joined #openstack-swift | 13:37 | |
*** ma9_ has quit IRC | 13:39 | |
*** stradling has quit IRC | 13:48 | |
*** vint_bra has joined #openstack-swift | 14:15 | |
*** amoralej|lunch is now known as amoralej | 14:20 | |
*** cshastri has quit IRC | 14:33 | |
*** thurloat[m] has joined #openstack-swift | 14:36 | |
*** chlong has joined #openstack-swift | 14:45 | |
*** Guest56308 has joined #openstack-swift | 15:01 | |
*** stradling has joined #openstack-swift | 15:01 | |
*** Guest56308 has quit IRC | 15:06 | |
*** chsc has joined #openstack-swift | 15:07 | |
*** klrmn has joined #openstack-swift | 15:15 | |
*** oshritf has quit IRC | 15:20 | |
*** tesseract has quit IRC | 15:31 | |
notmyname | good morning | 15:42 |
notmyname | the TC has https://review.openstack.org/#/c/451524/ on the agenda for their meeting tomorrow. /cc tdasilva | 15:43 |
notmyname | patch 451524 | 15:43 |
patchbot | https://review.openstack.org/#/c/451524/ - governance - Use case analysis for Golang addition to Openstack | 15:43 |
*** jordanP has quit IRC | 15:43 | |
notmyname | acoles: do you remember when joel gets back? | 15:45 |
acoles | notmyname: I believe he is back - I saw him leave some review comments late last week | 15:45 |
notmyname | ok | 15:45 |
notmyname | oh! you had time change this past weekend, right? | 15:46 |
acoles | unless he did that from a mountain side somewhere | 15:46 |
acoles | notmyname: nope we've been on BST for a whole week now :) | 15:46 |
acoles | notmyname: however mattoliverau did just change clocks | 15:46 |
notmyname | ah | 15:47 |
*** mvk has quit IRC | 15:47 | |
*** pcaruana has quit IRC | 15:53 | |
*** sams-gleb has quit IRC | 15:54 | |
*** sams-gleb has joined #openstack-swift | 15:55 | |
*** sams-gleb has quit IRC | 15:59 | |
*** stradling has quit IRC | 16:00 | |
*** _JZ_ has joined #openstack-swift | 16:05 | |
*** JimCheung has joined #openstack-swift | 16:07 | |
*** psachin has quit IRC | 16:10 | |
*** klrmn has quit IRC | 16:13 | |
*** ma9_1 has quit IRC | 16:17 | |
*** ma9_ has joined #openstack-swift | 16:18 | |
*** d0ugal has quit IRC | 16:19 | |
*** ma9_ has quit IRC | 16:22 | |
*** sams-gleb has joined #openstack-swift | 16:24 | |
openstackgerrit | Alistair Coles proposed openstack/swift master: sharding: Test and refactor for _misplaced_objects https://review.openstack.org/451896 | 16:28 |
*** ma9_ has joined #openstack-swift | 16:28 | |
acoles | mattoliverau: ^^ the test now passes, I figured out how to get the list_iter_objects to return the full row info | 16:29 |
acoles | mattoliverau: the interface into the broker is a bit crufty, we might want to make a variant of list_objects_iter that returns a dict like you use generate_list_objects to produce, save on an iteration at same time | 16:32 |
tdasilva | notmyname: ack, got an email from ttx | 16:35 |
*** stradling has joined #openstack-swift | 16:35 | |
notmyname | tdasilva: anything I should know? | 16:36 |
tdasilva | notmyname: it was just the meeting agenda | 16:36 |
tdasilva | he sent it to tc list and cc'd me on it | 16:36 |
*** cbartz has left #openstack-swift | 16:39 | |
*** ma9_ has quit IRC | 16:55 | |
*** stradling has quit IRC | 16:57 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/swift master: Updated from global requirements https://review.openstack.org/88736 | 16:59 |
*** chlong has quit IRC | 17:05 | |
*** stradling has joined #openstack-swift | 17:09 | |
*** klrmn has joined #openstack-swift | 17:13 | |
*** d0ugal has joined #openstack-swift | 17:15 | |
*** chsc has quit IRC | 17:16 | |
*** chlong has joined #openstack-swift | 17:20 | |
*** jaosorior is now known as jaosorior_away | 17:29 | |
*** chosafine has joined #openstack-swift | 17:46 | |
*** stradling has quit IRC | 17:49 | |
*** jamielennox is now known as jamielennox|away | 17:50 | |
*** SkyRocknRoll has joined #openstack-swift | 17:51 | |
*** chosafine has quit IRC | 17:53 | |
*** amoralej is now known as amoralej|off | 17:57 | |
*** d0ugal has quit IRC | 17:59 | |
*** ChubYann has joined #openstack-swift | 18:02 | |
openstackgerrit | Christian Schwede proposed openstack/swift master: Add commands to ring-builder to change region / zone https://review.openstack.org/291990 | 18:14 |
*** silor1 has joined #openstack-swift | 18:17 | |
*** silor has quit IRC | 18:20 | |
*** silor1 is now known as silor | 18:20 | |
*** d0ugal has joined #openstack-swift | 18:20 | |
*** tonanhngo has joined #openstack-swift | 18:43 | |
*** silor1 has joined #openstack-swift | 18:49 | |
*** silor has quit IRC | 18:49 | |
*** silor1 is now known as silor | 18:49 | |
*** silor has quit IRC | 18:53 | |
*** JimCheung has quit IRC | 18:54 | |
*** JimCheung has joined #openstack-swift | 18:55 | |
*** d0ugal has quit IRC | 19:23 | |
mgagne | what's the best way to bulk delete a container with millions of objects? :D | 19:25 |
*** d0ugal has joined #openstack-swift | 19:25 | |
*** stradling has joined #openstack-swift | 19:28 | |
tdasilva | mgagne: one way is to use the bulk delete api: https://docs.openstack.org/developer/swift/middleware.html#bulk-delete ;) | 19:30 |
mgagne | tdasilva: thanks, will have to check if bulk is used. The openstackclient is used and multiple objects are passed. But at first glance, it looks like it doesn't translate to bulk :-/ | 19:32 |
mgagne | looks like "native" swift client has more options at the CLI | 19:34 |
*** d0ugal has quit IRC | 19:36 | |
tdasilva | mgagne: yeah, using either cli is a different approach that can also be useful because you could potentially be spreading the load across many proxies | 19:36 |
tdasilva | as opposed to the bulk api where you are sending a request to one proxy and that one proxy is sending all delete requests to the storage servers | 19:37 |
mgagne | true, will check that, thanks =) | 19:37 |
*** chlong has quit IRC | 19:37 | |
mgagne | we aren't using the bulk API and so far, it's gonna take weeks to delete =) | 19:38 |
*** chsc has joined #openstack-swift | 19:39 | |
*** chlong has joined #openstack-swift | 19:51 | |
*** jamielennox|away is now known as jamielennox | 20:05 | |
clayg | steps to fix a bug in swift #1 report the bug #2 understand what's broke #3 write the trivial and obvious fix #4 ship it - when we try to do #1-3 at the sametime things get gummed up in boring and useless ways - please report issues as early as possilbe and make sure the bug is triaged before commiting to writing tests/change - if you happen to have a patch that womm you can attach it to the bug report to help with the triage. | 20:36 |
notmyname | I sense frustration | 20:36 |
notmyname | but you're not wrong! | 20:37 |
*** zaitcev has joined #openstack-swift | 20:37 | |
*** ChanServ sets mode: +v zaitcev | 20:37 | |
*** dja has joined #openstack-swift | 20:42 | |
*** stradling has quit IRC | 20:43 | |
*** mvk has joined #openstack-swift | 20:52 | |
*** sams-gleb has quit IRC | 20:55 | |
*** d0ugal has joined #openstack-swift | 21:01 | |
*** dja has quit IRC | 21:16 | |
*** mattoliverau has quit IRC | 21:24 | |
*** darrenc has quit IRC | 21:24 | |
*** darrenc has joined #openstack-swift | 21:24 | |
*** matt6434 has joined #openstack-swift | 21:25 | |
*** dja has joined #openstack-swift | 21:44 | |
*** dja has quit IRC | 21:49 | |
clayg | notmyname: nah, it's fine - but I don't feel great about setting a bug from in progress to incomplete - or debating internally if my notes from investigation should go on the bug report or the patch - and I feel like I've done that twice in the two days back from vacation | 21:53 |
clayg | I think the solution is to stop writing bugs - process of not having any bugs to fix is more similar to vacation ;) | 21:54 |
*** dja has joined #openstack-swift | 22:12 | |
*** rcernin has quit IRC | 22:12 | |
zaitcev | "In one of our installations, we are using Fernet tokens with Keystone V3. In that context, I was wondering if we need the keystone revocation thread in RGW to run by default." -- Ceph users, they do curious things. | 22:19 |
*** matt6434 is now known as mattoliverau | 22:22 | |
*** ChanServ sets mode: +v mattoliverau | 22:22 | |
mattoliverau | Morning | 22:24 |
*** stradling has joined #openstack-swift | 22:25 | |
*** stradling has quit IRC | 22:27 | |
*** chlong has quit IRC | 22:38 | |
*** geaaru has quit IRC | 22:38 | |
*** catintheroof has quit IRC | 22:44 | |
mattoliverau | acoles: ta, yeah, that's a good point. that would clean it up alot. | 22:51 |
*** vint_bra has quit IRC | 22:57 | |
*** adriant has joined #openstack-swift | 23:01 | |
*** _JZ_ has quit IRC | 23:06 | |
notmyname | good morning mattoliverau | 23:07 |
notmyname | mattoliverau: I was wondering what's going on with stable gate tests | 23:08 |
mattoliverau | notmyname: not too much, I have it running, but haven't checked it much.. I'll make sure I spend some time today spending a few cycles playing with it some more :) | 23:10 |
notmyname | mattoliverau: https://review.openstack.org/#/c/448854/ seems to be passing the gate now, but wasn't there another issue too? (sorry, I had a 4 day weekend, so my memory is a little fuzzy) | 23:10 |
notmyname | mattoliverau: yeah, thanks. I was looking at other backports, and that's when I remembered something something something big problem matt's fixing it | 23:10 |
mattoliverau | lol | 23:10 |
mattoliverau | notmyname: so tldr, ther doesn't seem to be any big problems, as each intermittent error is mostly different. so I think it's just flaky testing edge cases in general. | 23:12 |
mattoliverau | and sometimes it takes a while on an SAIO to hit one. | 23:12 |
notmyname | yeah, but timburke had mentioned that they'd gotten a lot more frequent lately. i think that was the only warning flag | 23:13 |
mattoliverau | But I want to keep an eye, because as we solve each one (over time) we can learn the courses and teach us all how to write better tests :) | 23:13 |
mattoliverau | notmyname: yeah, the periodic jobs in gerrit seem to hit alot more. though that do run once a day. | 23:14 |
notmyname | yeah, if it's so rare in the saio, I'd expect it to be even more rare in a once-daily job | 23:14 |
mattoliverau | so what 1 a week (pulling a number out of there air) is alot more regualar that what I'm seeing. | 23:15 |
mattoliverau | ^ so what you said :P | 23:15 |
mattoliverau | notmyname: is it a higher contended bunch of servers, causing more slips in say dealing with timestamps then what happens to us locally, or the way nodepool builds test nodes? | 23:16 |
mattoliverau | hmm, maybe I could go build a node pool image, and then run some tests on that. | 23:16 |
mattoliverau | or however they do that nowdays | 23:16 |
mattoliverau | I might ask Josh in our meeting today :) | 23:17 |
mattoliverau | ^ I mean the Australian rpc standup | 23:18 |
mattoliverau | k, email read, /me is going to grab a coffee. bbs | 23:19 |
*** tonanhngo has quit IRC | 23:24 | |
*** kei_yama has joined #openstack-swift | 23:36 | |
*** tonanhngo has joined #openstack-swift | 23:37 | |
*** tonanhngo has quit IRC | 23:41 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!