| *** markvoelker has joined #openstack-performance | 00:11 | |
| *** Deng has joined #openstack-performance | 00:46 | |
| *** dimtruck is now known as zz_dimtruck | 01:22 | |
| *** rfolco has quit IRC | 01:24 | |
| *** zz_dimtruck is now known as dimtruck | 02:00 | |
| *** pcaruana has joined #openstack-performance | 06:39 | |
| *** rmart04 has joined #openstack-performance | 06:50 | |
| *** rmart04 has quit IRC | 06:56 | |
| *** dimtruck is now known as zz_dimtruck | 07:11 | |
| *** rmart04 has joined #openstack-performance | 07:28 | |
| *** markvoelker has quit IRC | 07:59 | |
| *** markvoelker has joined #openstack-performance | 09:00 | |
| *** zz_dimtruck is now known as dimtruck | 09:02 | |
| *** markvoelker has quit IRC | 09:05 | |
| *** dimtruck is now known as zz_dimtruck | 09:16 | |
| *** markvoelker has joined #openstack-performance | 10:01 | |
| *** markvoelker has quit IRC | 10:05 | |
| *** markvoelker has joined #openstack-performance | 11:02 | |
| *** markvoelker has quit IRC | 11:06 | |
| *** bvanhav has joined #openstack-performance | 11:16 | |
| *** rfolco has joined #openstack-performance | 11:45 | |
| *** markvoelker has joined #openstack-performance | 12:02 | |
| *** markvoelker has quit IRC | 12:07 | |
| *** markvoelker has joined #openstack-performance | 12:23 | |
| *** catintheroof has joined #openstack-performance | 12:46 | |
| *** rook_ is now known as rook | 13:01 | |
| *** permalac has joined #openstack-performance | 13:18 | |
| *** rfolco has quit IRC | 13:48 | |
| *** rfolco has joined #openstack-performance | 13:49 | |
| *** hieulq_ has joined #openstack-performance | 14:06 | |
| *** zz_dimtruck is now known as dimtruck | 14:06 | |
| *** subscope has joined #openstack-performance | 14:39 | |
| *** catintheroof has quit IRC | 14:47 | |
| *** catintheroof has joined #openstack-performance | 14:48 | |
| bvanhav | Hello, are there plans to launch a new release of OSProfiler in the near future? A number things (like MongoDB support) were added recently, but are too new for the 1.3.0 release... | 14:49 |
|---|---|---|
| *** catintheroof has quit IRC | 14:53 | |
| *** catintheroof has joined #openstack-performance | 15:25 | |
| *** hieulq_ has quit IRC | 15:29 | |
| *** rmart04 has quit IRC | 15:32 | |
| *** rmart04 has joined #openstack-performance | 15:34 | |
| *** rmart04 has quit IRC | 15:34 | |
| *** victorhe has joined #openstack-performance | 15:48 | |
| *** pcaruana has quit IRC | 15:48 | |
| *** kristian__ has joined #openstack-performance | 15:51 | |
| *** rcherrueau has joined #openstack-performance | 15:55 | |
| *** hieulq_ has joined #openstack-performance | 15:57 | |
| *** gokrokve has joined #openstack-performance | 15:58 | |
| DinaBelova | #startmeeting Performance Team | 16:00 |
| openstack | Meeting started Tue Aug 9 16:00:02 2016 UTC and is due to finish in 60 minutes. The chair is DinaBelova. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| openstack | The meeting name has been set to 'performance_team' | 16:00 |
| DinaBelova | hey folks! | 16:00 |
| DinaBelova | bvanhav indeed :) | 16:00 |
| DinaBelova | going to push new release today | 16:00 |
| DinaBelova | ok, let's see who's around | 16:01 |
| gokrokve | o/ | 16:01 |
| DinaBelova | gokrokve o/ | 16:01 |
| *** rohanion has joined #openstack-performance | 16:01 | |
| rcherrueau | hi | 16:01 |
| rohanion | hi o/ | 16:01 |
| DinaBelova | rohanion, rcherrueau nice to see you :) | 16:02 |
| rohanion | same here :) | 16:02 |
| DinaBelova | okay, so let's get started with usual topics | 16:02 |
| DinaBelova | #topic Performance docs update | 16:03 |
| DinaBelova | lots of stuff happened during last week | 16:03 |
| DinaBelova | we were able to merge almost all changes on review | 16:03 |
| DinaBelova | and I cleaned up a bit our LP bugs list | 16:03 |
| DinaBelova | #link https://bugs.launchpad.net/performance-docs | 16:04 |
| DinaBelova | so right now if bug is assigned to me you can feel free to reassign it to yourself and start work on it | 16:04 |
| DinaBelova | :) | 16:04 |
| DinaBelova | #link https://review.openstack.org/#/c/334964/ | 16:04 |
| DinaBelova | this change is going to make our life less painful next merge party | 16:05 |
| rohanion | yaaay, merge party! | 16:05 |
| DinaBelova | :D | 16:05 |
| DinaBelova | so we won't need to rebase all changes after each merged commit | 16:05 |
| DinaBelova | rohanion please find Lenya and remind him about https://bugs.launchpad.net/performance-docs/+bug/1609924 :) | 16:06 |
| openstack | Launchpad bug 1609924 in OpenStack Performance Docs "500 nodes lab description is missing" [Medium,Triaged] - Assigned to Leontiy Istomin (listomin) | 16:06 |
| DinaBelova | it's a bit strange we're running our tests against 500 nodes lab and no lab description is yet published here | 16:06 |
| rohanion | yeah will do | 16:07 |
| DinaBelova | #info next week will be dedicated to minor fixes for the bugs from the list https://bugs.launchpad.net/performance-docs | 16:07 |
| * rohanion bookmarked | 16:08 | |
| DinaBelova | so I would say that's all regarding performance docs themselves | 16:08 |
| DinaBelova | #topic Current progress on the planned tests | 16:08 |
| DinaBelova | so as I mentioned some meetings ago, Q3 is a logical continuation of the work done in Q2 | 16:09 |
| DinaBelova | so today we've continued Neutron testing | 16:09 |
| DinaBelova | on 250 nodes lab | 16:09 |
| DinaBelova | so more results will be coming in next 2 weeks or so | 16:10 |
| DinaBelova | so that will be extended scalability tests, performance measurements under the load, etc. | 16:10 |
| DinaBelova | also in parallel we're working on baseline performance testing | 16:11 |
| DinaBelova | #link http://docs.openstack.org/developer/performance-docs/test_plans/control_plane/plan.html | 16:11 |
| DinaBelova | against 500 nodes | 16:11 |
| DinaBelova | and its comparison with 250 nodes runs we did already | 16:11 |
| DinaBelova | extended reliability testing is planned later :) | 16:12 |
| DinaBelova | #info working on extended Neutron testing and on baseline performance testing on 500 nodes. Reliability tests, 1000 nodes emulation and others are planned in 2 weeks or so | 16:13 |
| DinaBelova | any updates on current test runs from other folks? | 16:13 |
| DinaBelova | rcherrueau ? | 16:13 |
| rcherrueau | Yes | 16:14 |
| rcherrueau | We have a first solution that deploys a distributed OpenStack over g5k (a grid) using kolla. | 16:14 |
| rcherrueau | You can find the code on github | 16:14 |
| rcherrueau | #link https://github.com/BeyondTheClouds/kolla-g5k | 16:14 |
| DinaBelova | ack, bookmarked | 16:14 |
| kristian__ | DinaBelova, hi, how are you? good to interact again...just curious, is there an established ETA / date for osprofiler w/ Neutron and Swift? any future plans to integrate w/ Juniper Contrail? | 16:14 |
| rohanion | oh darn swift | 16:14 |
| rcherrueau | We actually test with a small number of controller and go with the `fake_node` option to multiply the number of compute node. | 16:14 |
| rcherrueau | 16:14 | |
| rohanion | we forgot about it T_T | 16:15 |
| rcherrueau | #link https://github.com/openstack/kolla/blob/master/doc/nova-fake-driver.rst | 16:15 |
| DinaBelova | kristian__ neutron is already supported :) | 16:15 |
| DinaBelova | and you're right, Swift is not yet supported | 16:15 |
| DinaBelova | I'd say it's up to rohanion to define the ETA | 16:15 |
| kristian__ | ...just confirming to make sure. and that is releasable for us, yes :) | 16:15 |
| rohanion | do we have to integrate osprofiler with swift or just patch it to work with the new driver architecture? | 16:16 |
| kristian__ | more interested in Juniper contrail integration these days... | 16:16 |
| DinaBelova | regarding the Contrail - the issue is that osprofiler is python library, so we'll be able to hack into its python client.. but not to the Juniper contrail itself using the profiler | 16:17 |
| kristian__ | yes so coding integration is blocked there, how about with juniper's REST API interface? | 16:18 |
| DinaBelova | so right now I'm not sure how we can include contrail specific tracing details to the osprofiler trace | 16:18 |
| DinaBelova | kristian__ well, I think (??) we need to check how osprofiler is working with the neurtron drivers | 16:18 |
| rohanion | DinaBelova: maybe we can use osprofiler_web with juniper's REST API | 16:18 |
| kristian__ | good points, guys | 16:19 |
| DinaBelova | rohanion - no need, we'll just wrap already wrapped stuff in the neutron drivers code - It looks like it'll be enough to check how drivers are wrapped in neutron | 16:19 |
| DinaBelova | kristian__ good point anyway, let us check | 16:20 |
| kristian__ | np, thank you | 16:20 |
| DinaBelova | #action DinaBelova rohanion check how can we estimate contrail profiling out of Neutron code using osprofiler | 16:20 |
| DinaBelova | ok, so let's jump to the osprofiler itself :) | 16:21 |
| rohanion | #info check if osprofiler is compatible with juniper's REST API | 16:21 |
| DinaBelova | #topic OSProfiler weekly update | 16:21 |
| DinaBelova | so hey! merge party finished today :) | 16:21 |
| rohanion | yeah we had a great merge party last week :) | 16:21 |
| DinaBelova | and I'm really glad to say I'm going to cut new release today :) | 16:21 |
| DinaBelova | #info new Osprofiler release coming today | 16:22 |
| DinaBelova | this will unblock lots of stuff | 16:22 |
| DinaBelova | bvanhav ^^ | 16:22 |
| DinaBelova | currently we basically have only one major change on review | 16:22 |
| rohanion | too bad I'm not in the US, we could have had a release party lol | 16:22 |
| DinaBelova | #link https://review.openstack.org/#/c/340936/ | 16:22 |
| DinaBelova | rohanion indeed :D | 16:23 |
| rohanion | Alex is away | 16:23 |
| DinaBelova | I'll post the link to the openstack/releases once it'll be ready to this chat | 16:23 |
| rohanion | Elasticsearch driver is almost ready | 16:23 |
| DinaBelova | rohanion true, but I don't want to delay release because of this | 16:23 |
| rohanion | he is going to cover it with tests and the change will be ready to merge | 16:24 |
| rohanion | yes, there's no need to delay | 16:24 |
| DinaBelova | I think it'll be more logical to cut one more with elasticsearch once it'll be landed | 16:24 |
| DinaBelova | rohanion thanks for the information | 16:24 |
| rohanion | elasticsearch does not change anything in the base architecture | 16:24 |
| rohanion | u r welcome | 16:24 |
| DinaBelova | indeed | 16:24 |
| rohanion | will it be v1.4.0? | 16:24 |
| DinaBelova | yep, I think so | 16:25 |
| DinaBelova | in fact we may cut 2.0.0 | 16:25 |
| rohanion | because our changes do not look like a minor update | 16:25 |
| rohanion | true | 16:25 |
| DinaBelova | keeping in mind that arch was changed much | 16:25 |
| DinaBelova | agree | 16:25 |
| rohanion | 2.0.0 sounds better | 16:25 |
| rohanion | so elastic will be in 2.0.1 | 16:25 |
| DinaBelova | #info so it'll be 2.0.0 release | 16:25 |
| DinaBelova | agree | 16:25 |
| * rohanion hopes so | 16:25 | |
| bvanhav | I'll be looking forward to it no matter which release number it ends up being | 16:26 |
| DinaBelova | ok, so it looks like that's all about osprofiler | 16:26 |
| DinaBelova | bvanhav true :) | 16:26 |
| DinaBelova | anything else to add here? | 16:26 |
| rcherrueau | I have a question about osprofiler. We are interesting by measuring which methods of the database api (nova/db/api.py) is called the most often. Do you think that osprofiler is a good condidate to do that? | 16:27 |
| rcherrueau | 16:27 | |
| rohanion | yes | 16:27 |
| rcherrueau | cool | 16:27 |
| DinaBelova | rcherrueau yes it is, but nova support was not yet merged | 16:27 |
| DinaBelova | this needs to be rebased https://review.openstack.org/#/c/254703/ | 16:27 |
| rcherrueau | ok when the merge will take place | 16:27 |
| rohanion | we already count the database and wsgi operations | 16:27 |
| rohanion | ah | 16:28 |
| rohanion | got your point | 16:28 |
| DinaBelova | rcherrueau not in Newton for sure - it's frozen for nova | 16:28 |
| rohanion | it's not the change in nova | 16:28 |
| rcherrueau | ok thanks | 16:29 |
| rohanion | it looks like a change in osprofiler's 'stats' field | 16:29 |
| DinaBelova | rohanion it is change for nova | 16:29 |
| DinaBelova | as we need to teach NOva how to use osprofiler | 16:29 |
| DinaBelova | exactly the same as it was for cinder and others | 16:30 |
| rohanion | ah that's what you're talking about | 16:30 |
| rohanion | yes, right | 16:30 |
| DinaBelova | rohanion :D | 16:30 |
| rohanion | I was talking about changing the stats | 16:30 |
| DinaBelova | :) I see. For Nova I was really glad to gain help from Mark | 16:30 |
| DinaBelova | but this did not change the fact that it's feature freeze for Nova will end of Newton | 16:31 |
| rohanion | but we can always switch to a dev branch :) | 16:31 |
| DinaBelova | ok, so it looks like we're done for osprofiler | 16:31 |
| rohanion | seems so | 16:31 |
| DinaBelova | rohanion hehe, we can, but the question is if rcherrueau can do it :) | 16:32 |
| rcherrueau | Why I cannot? | 16:32 |
| DinaBelova | rcherrueau btw, what OpenStack are you running? Mitaka? | 16:32 |
| rcherrueau | the dev branch is public no ? | 16:32 |
| DinaBelova | rcherrueau I'm asking :) | 16:32 |
| rcherrueau | yes mitake | 16:32 |
| rohanion | yes | 16:32 |
| rcherrueau | mitaka* | 16:32 |
| rohanion | it's somewhere in review.openstack.org/... | 16:32 |
| DinaBelova | rcherrueau - I can find in the history of the change the number of patch set | 16:32 |
| rcherrueau | is it a problem to run osprogiler over mitaka? | 16:33 |
| rcherrueau | osprofiler* | 16:33 |
| DinaBelova | that was workable against mitaka | 16:33 |
| rcherrueau | great | 16:33 |
| DinaBelova | rcherrueau nope, we just need to identify what patch set it was | 16:33 |
| DinaBelova | lemme find it and send it to you | 16:33 |
| rcherrueau | thank you DinaBelova | 16:33 |
| rohanion | it will work with mitaka but you won't be able to use any drivers but messaging/ceilometer | 16:33 |
| * notmyname reads scrollback and notices "swift" | 16:34 | |
| notmyname | let me know when/if you have swift questions | 16:34 |
| DinaBelova | #action DinaBelova find patch set # and link to the https://review.openstack.org/#/c/254703/<n> to rcherrueau with pure Mitaka support | 16:34 |
| *** hieulq_ has quit IRC | 16:34 | |
| rcherrueau | rohanion: thanks for the precision | 16:34 |
| DinaBelova | notmyname ack, no so far - as we did not start working on swift support yet :) | 16:34 |
| DinaBelova | #topic Open Discussion | 16:35 |
| DinaBelova | ok, so let's jump to open discussion | 16:35 |
| DinaBelova | anything else to share? | 16:35 |
| rcherrueau | I wanna talk about the 1000 node perf | 16:35 |
| rcherrueau | can I? | 16:35 |
| DinaBelova | sure | 16:35 |
| DinaBelova | :) | 16:35 |
| rcherrueau | So just like I said before, we go with kolla for the deployment of OpenStack | 16:36 |
| DinaBelova | so yeah, container-based installation | 16:36 |
| rcherrueau | We uses kolla to deploy some controller nodes and then we take the advantages of the `fake_driver` option to get many compute | 16:37 |
| DinaBelova | popular topic - at Mirantis we're working on CCP (containeraized control plane) as well | 16:37 |
| rcherrueau | Are any one of you familiar with this option | 16:37 |
| rcherrueau | #link https://github.com/openstack/kolla/blob/master/doc/nova-fake-driver.rst | 16:37 |
| DinaBelova | rcherrueau I used it some time ago | 16:37 |
| DinaBelova | any specific questions? | 16:37 |
| DinaBelova | if i won't be able to answer, I'll ask alwex to help | 16:38 |
| DinaBelova | as he did similar testing in q2 | 16:38 |
| rcherrueau | DinaBelova: we don't know how this option works exactly and we wanna be sure that doing some perf test with this option is equivalent to doing the same test with real node. | 16:38 |
| DinaBelova | rcherrueau well, it's not in fact | 16:38 |
| rcherrueau | arf ... | 16:38 |
| DinaBelova | fake driver emulates all operations to the hypervizor, but not doing them in fact | 16:39 |
| rcherrueau | it was a little bit expected in fact | 16:39 |
| DinaBelova | so fake nodes have no libvirt running, and no REAL VMs starting | 16:39 |
| DinaBelova | and this means that some specific network operations and setup is not going aas well | 16:40 |
| *** hieulq_ has joined #openstack-performance | 16:40 | |
| rcherrueau | ok thanks | 16:40 |
| rcherrueau | do you know if we can (easily) deploy many compute on the same physical node with kolla? | 16:40 |
| DinaBelova | rcherrueau this means we're emulating the "control plane" for the hypervisor and everything underlaying, but no real VMs will be created | 16:40 |
| DinaBelova | rcherrueau sadly not an expert here :( , I'm pretty sure there will be issues with locating several compute containers on the same node | 16:41 |
| DinaBelova | I think it'd be better to ask in kolla channel | 16:42 |
| rcherrueau | DinaBelova: ok thanks for your responses | 16:42 |
| rcherrueau | I will do so | 16:42 |
| DinaBelova | rcherrueau the only 100% workable way I know to place several computes on the same physical node is to place them in the VMs | 16:42 |
| DinaBelova | this is something we're going to do in next 2-3 weeks or so | 16:43 |
| DinaBelova | against 500 nodes lab | 16:43 |
| rcherrueau | ok | 16:43 |
| rcherrueau | ah ok | 16:43 |
| DinaBelova | so we'll place ~2 VMs per node and have real 1000 nodes cluster | 16:43 |
| rcherrueau | ok got it | 16:43 |
| DinaBelova | but i don't know how to manage this with docker | 16:43 |
| DinaBelova | rcherrueau sorry for not helping much | 16:43 |
| rcherrueau | no don't worry. I will ask on the mailing list | 16:44 |
| DinaBelova | ack, thanks | 16:44 |
| DinaBelova | ok, anything else? | 16:44 |
| rcherrueau | yep | 16:44 |
| DinaBelova | go ahead :) | 16:44 |
| rcherrueau | We run the performance test using rally. | 16:45 |
| rcherrueau | especially the `boot-and-delete` scenario | 16:45 |
| rcherrueau | Which test do you use in the 500 nodes lab? | 16:45 |
| DinaBelova | rcherrueau ack, can you share details regarding number of nodes, cloud topology and the results? | 16:45 |
| DinaBelova | rcherrueau we are using what we're calling baseline performance test suite - http://docs.openstack.org/developer/performance-docs/test_plans/control_plane/plan.html | 16:46 |
| rcherrueau | actually no, not now. We only run it over a small number of instances (less than 50). | 16:46 |
| DinaBelova | rcherrueau ack -so for nova we're stressing it with booting and deleting VMs with security groups | 16:47 |
| rcherrueau | ok thanks | 16:47 |
| DinaBelova | as secgroups is really something that can ruin all the stuff in specific cases | 16:47 |
| rcherrueau | ok | 16:47 |
| DinaBelova | we've published the plugin we used here https://github.com/openstack/performance-docs/tree/master/doc/source/test_plans/control_plane/plugins | 16:48 |
| DinaBelova | rcherrueau ^^ | 16:48 |
| rcherrueau | nice bookmarked ;) | 16:48 |
| DinaBelova | so don't be surprised seeing NovaPerformancePlugin.boot_attach_and_delete_server_with_secgroups scenario :) | 16:48 |
| DinaBelova | this is used from this plugin | 16:48 |
| rcherrueau | OK, I will take a look | 16:48 |
| DinaBelova | all other scenarios are standard | 16:49 |
| DinaBelova | cool :) | 16:49 |
| DinaBelova | rcherrueau anything else? | 16:49 |
| rcherrueau | no | 16:50 |
| DinaBelova | ok, so it looks like we're done for today :) | 16:50 |
| rohanion | guys please wait for a minute :) | 16:50 |
| DinaBelova | rohanion a-ha :D | 16:50 |
| rohanion | Alex is going to join | 16:50 |
| *** aignatyev has joined #openstack-performance | 16:50 | |
| DinaBelova | aignatyev hey :) | 16:50 |
| DinaBelova | nice to see you here :) | 16:50 |
| aignatyev | hi :) | 16:50 |
| rohanion | Welcome Alex! | 16:50 |
| rcherrueau | hello | 16:51 |
| rohanion | please meet another osprofiler monster :) | 16:51 |
| *** subscope has quit IRC | 16:51 | |
| aignatyev | nice to meet you all :) | 16:51 |
| DinaBelova | aignatyev can you please share info about what's left to be done for https://review.openstack.org/#/c/340936/ ? | 16:51 |
| DinaBelova | so we can understand the ETA | 16:51 |
| aignatyev | I need to write 1-2 tests for retrieving reports from es | 16:51 |
| aignatyev | there's already 1 test, but I need more | 16:51 |
| aignatyev | will be done tomorrow | 16:51 |
| DinaBelova | aignatyev ok, so I'll check the driver against my test lab when it'll be finalized | 16:52 |
| DinaBelova | probably tomorrow as well | 16:53 |
| DinaBelova | thanks | 16:53 |
| aignatyev | you are welcome | 16:53 |
| rohanion | I'll check it as well | 16:53 |
| DinaBelova | rohanion ack | 16:53 |
| DinaBelova | ok, so we had really nice discussion today | 16:53 |
| DinaBelova | thanks everyone for coming | 16:53 |
| DinaBelova | bye! | 16:53 |
| DinaBelova | #endmeeting | 16:53 |
| openstack | Meeting ended Tue Aug 9 16:53:51 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:53 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/performance_team/2016/performance_team.2016-08-09-16.00.html | 16:53 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/performance_team/2016/performance_team.2016-08-09-16.00.txt | 16:53 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/performance_team/2016/performance_team.2016-08-09-16.00.log.html | 16:53 |
| *** hieulq_ has quit IRC | 16:57 | |
| *** aignatyev has quit IRC | 17:00 | |
| *** rcherrueau has quit IRC | 17:00 | |
| *** gokrokve has quit IRC | 17:02 | |
| *** pcaruana has joined #openstack-performance | 17:44 | |
| *** rohanion has quit IRC | 17:54 | |
| *** victorhe has quit IRC | 18:05 | |
| DinaBelova | harlowja hey! please +1 :) https://review.openstack.org/#/c/353015/2 | 18:06 |
| harlowja | yup yup, right after i fix this cloud-init script :-P | 18:06 |
| openstackgerrit | Merged openstack/performance-docs: Make main docs page more readable https://review.openstack.org/334964 | 18:18 |
| *** rohanion has joined #openstack-performance | 18:31 | |
| *** rohanion has quit IRC | 18:40 | |
| *** catintheroof has quit IRC | 19:12 | |
| *** bvanhav has quit IRC | 19:29 | |
| *** kristian__ has quit IRC | 19:37 | |
| *** rfolco has quit IRC | 20:59 | |
| *** rfolco has joined #openstack-performance | 20:59 | |
| *** rfolco has quit IRC | 21:00 | |
| *** f13o has joined #openstack-performance | 21:16 | |
| *** Deng has quit IRC | 21:30 | |
| *** f13o has quit IRC | 21:57 | |
| *** dimtruck is now known as zz_dimtruck | 22:06 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!