| *** dschroeder has quit IRC | 00:49 | |
| *** marzif has joined #openstack-freezer | 08:12 | |
| *** reldan has joined #openstack-freezer | 08:14 | |
| *** marzif has quit IRC | 08:19 | |
| *** marzif has joined #openstack-freezer | 08:20 | |
| *** reldan has quit IRC | 08:26 | |
| *** reldan has joined #openstack-freezer | 09:02 | |
| *** marzif_ has joined #openstack-freezer | 09:04 | |
| *** marzif has quit IRC | 09:04 | |
| *** jonaspf has joined #openstack-freezer | 09:09 | |
| openstackgerrit | Memo Garcia proposed stackforge/freezer-web-ui: Fixed retrieval of freezer url from keystone catalog https://review.openstack.org/234844 | 09:12 |
|---|---|---|
| openstackgerrit | Merged stackforge/freezer: Test Coverage: Improving test coverage by creation fs_like storage https://review.openstack.org/234270 | 09:57 |
| *** reldan has quit IRC | 11:00 | |
| *** reldan has joined #openstack-freezer | 11:04 | |
| openstackgerrit | Eldar Nugaev proposed stackforge/freezer: Forgotten return in ssh https://review.openstack.org/235271 | 11:35 |
| *** marzif_ has quit IRC | 11:57 | |
| *** marzif_ has joined #openstack-freezer | 11:58 | |
| openstackgerrit | Merged stackforge/freezer: Forgotten return in ssh https://review.openstack.org/235271 | 12:18 |
| *** marzif_ has quit IRC | 12:28 | |
| *** marzif_ has joined #openstack-freezer | 12:28 | |
| *** reldan has quit IRC | 12:58 | |
| *** marzif_ has quit IRC | 13:02 | |
| *** marzif_ has joined #openstack-freezer | 13:03 | |
| openstackgerrit | Merged stackforge/freezer-web-ui: Fixed retrieval of freezer url from keystone catalog https://review.openstack.org/234844 | 13:06 |
| *** marzif_ has quit IRC | 13:08 | |
| openstackgerrit | Merged stackforge/freezer-web-ui: Fix: Job creation now sends client_id instead of uuid https://review.openstack.org/234234 | 13:08 |
| *** reldan has joined #openstack-freezer | 13:14 | |
| *** dschroeder has joined #openstack-freezer | 13:59 | |
| openstackgerrit | Fausto Marzi proposed stackforge/freezer: Swift from pytest to testr for unittests https://review.openstack.org/235387 | 14:29 |
| openstackgerrit | Fausto Marzi proposed stackforge/freezer: Swift from pytest to testr for unittests https://review.openstack.org/235387 | 14:30 |
| daemontool_ | all: this is very important, please review https://review.openstack.org/235387 | 14:30 |
| vannif | hi everyone | 15:10 |
| daemontool_ | Hi vannif | 15:10 |
| vannif | shall we start the meeting ? | 15:11 |
| daemontool_ | yep | 15:11 |
| vannif | #startmeeting 2015-10-15 | 15:12 |
| freezerBot` | Meeting started Thu Oct 15 15:12:27 2015 UTC and is due to finish in 60 minutes. The chair is vannif. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:12 |
| freezerBot` | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:12 |
| freezerBot` | The meeting name has been set to '2015_10_15' | 15:12 |
| openstack | Meeting started Thu Oct 15 15:12:27 2015 UTC and is due to finish in 60 minutes. The chair is vannif. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:12 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:12 |
| openstack | The meeting name has been set to '2015_10_15' | 15:12 |
| vannif | hi everyone | 15:12 |
| vannif | daemontool_ would you start ? | 15:13 |
| daemontool_ | ok | 15:13 |
| daemontool_ | I've been working mainly on testing | 15:13 |
| daemontool_ | helping vannif integrated testing with devstack | 15:14 |
| daemontool_ | and also now I'm working on moveing from pytest to testr | 15:14 |
| daemontool_ | to be consistent witht he other openstack proejcts | 15:14 |
| daemontool_ | the issue with this | 15:14 |
| daemontool_ | is that many of our unittests are now incompatible | 15:14 |
| daemontool_ | because of the use of monkeypatch from pytests | 15:15 |
| daemontool_ | the unittests needs to be changed asap after https://review.openstack.org/235387 lands in | 15:15 |
| daemontool_ | I'm planning this week also to swift the stackforge/freezer repo to PBR | 15:15 |
| daemontool_ | that's all from me | 15:15 |
| vannif | can you give a rough estimate of the effort to remove the dependency on pytest ? | 15:17 |
| daemontool_ | I think it will take ~2 weeks | 15:17 |
| daemontool_ | realistically | 15:17 |
| daemontool_ | these are the bp for pbr and testr https://blueprints.launchpad.net/freezer/+spec/switch-to-pbr https://blueprints.launchpad.net/freezer/+spec/swift-to-testr | 15:18 |
| daemontool_ | I'll be full on it to have it completed | 15:18 |
| daemontool_ | asap | 15:18 |
| daemontool_ | reldan, I think this task involves you too | 15:18 |
| reldan | Yes, sure. I have some pytest tests and can help with migration | 15:19 |
| reldan | I work with tests anyway, so can replace pytes to unittest step by step | 15:19 |
| daemontool_ | perfect | 15:20 |
| daemontool_ | afaik the probelm is only with monkeypatch | 15:20 |
| daemontool_ | s/probelm/problem/ | 15:20 |
| vannif | good. maybe everyone can help moving it's own part of code to unittest | 15:20 |
| daemontool_ | that's be fantastic | 15:21 |
| vannif | I've recently moved the tests for lvm to unittest. if you know the code, it's faster. even though to be really good the tests should be written by someone else ;) | 15:22 |
| vannif | ok. | 15:22 |
| vannif | reldan ? | 15:22 |
| reldan | Thank you | 15:23 |
| daemontool_ | ok | 15:23 |
| reldan | I was improving test coverage. So I wrote additional tests and for example improved ssh and local storeages in terms of code deduplication | 15:23 |
| reldan | Now ssh only has stuff related to work with ssh and local only os.blabla invokations | 15:24 |
| reldan | I have 62 code coverage overall. The biggest uncovered part is scheduler, so I’m trying to write some tests on it. At least invoke methods to catch syntax errors if any | 15:25 |
| reldan | Also the problem part is OpenStack Cinder/Nova backups | 15:25 |
| reldan | Current coverage is http://pastebin.com/GfAdZ3cE | 15:26 |
| reldan | Also ssh paramiko is very-very slow. Probably we should try some different libary after release. And I have an idea how to read only part of backups from ssh - probably it will improve speed of our integration tests with ssh | 15:27 |
| reldan | Because they get all backups many many times pre run | 15:27 |
| reldan | It is all from my side | 15:27 |
| vannif | thank you. remember not to use pytest ;) | 15:28 |
| reldan | Yes, sure :) Thank you | 15:28 |
| vannif | on my side | 15:28 |
| vannif | the freezer-api devstack plugin is ready. | 15:29 |
| vannif | well, almost. | 15:29 |
| vannif | I just need to add a few lines in the readme | 15:29 |
| vannif | and the plugin for the web-ui also. | 15:30 |
| vannif | so, all it takes to have freezer, freezer-api and freezer-web-ui working in devstack is a couple of lines in the local.conf | 15:30 |
| daemontool_ | vannif, sorry one sec | 15:30 |
| daemontool_ | reldan, why paramiko is slow, do you have an idea? | 15:30 |
| *** openstackgerrit has quit IRC | 15:31 | |
| *** openstackgerrit has joined #openstack-freezer | 15:32 | |
| reldan | I have read a lot of message threads on stackoverflow and in repo of paramiko. And I also saw a lot of branches where some guys are trying to resolve it. As far as I know Fabric has it’s own fork of paramiko | 15:32 |
| reldan | But why paramiko is so slow - id don’t know. https://www.google.ie/search?q=paramiko+is+slow&oq=paramiko+is+slow&aqs=chrome..69i57j69i60l2j0l3.2588j0j7&sourceid=chrome&es_sm=91&ie=UTF-8 | 15:33 |
| daemontool_ | ok... | 15:33 |
| reldan | Probably it has some problem with buffers | 15:33 |
| reldan | I read it reads information byte-by-byte | 15:34 |
| daemontool_ | ah ok | 15:34 |
| daemontool_ | there's some for in binary_data there.... | 15:34 |
| reldan | I’m not sure, but it seems to be a common problem | 15:34 |
| daemontool_ | ok | 15:34 |
| reldan | I have found it - but don’t try it yet https://github.com/wallix/pylibssh2 | 15:34 |
| reldan | and it is old | 15:35 |
| daemontool_ | let's have a meeting at some point about it | 15:35 |
| daemontool_ | but how much is slow | 15:35 |
| daemontool_ | how much MB/s can we transfer? | 15:35 |
| daemontool_ | do you have an idea? | 15:35 |
| reldan | Nope, but it is really slow - I work with hpcloud and even listdir takes minutes | 15:36 |
| daemontool_ | ah..... | 15:36 |
| daemontool_ | ok | 15:36 |
| vannif | that's really slow | 15:37 |
| daemontool_ | yes | 15:37 |
| reldan | I was trying to tune windows size - didn’t help | 15:38 |
| reldan | window size | 15:38 |
| daemontool_ | ok | 15:38 |
| vannif | window size of tcp connection ? | 15:38 |
| reldan | Yes | 15:38 |
| daemontool_ | what uses ansible under the hood? | 15:38 |
| vannif | hmm. that should be a separate issue. the OS should take care of that ... | 15:39 |
| daemontool_ | I'm not sure it's the tcp windows | 15:39 |
| vannif | I think paramiko | 15:39 |
| daemontool_ | anyway we need to find a solution about it | 15:40 |
| daemontool_ | but I have to say reldan with the tests from vanni | 15:40 |
| reldan | I know that Fabric uses paramiko | 15:40 |
| daemontool_ | using ssh | 15:40 |
| reldan | # Development version of Paramiko, just in case we're in one of those phases. | 15:40 |
| reldan | -e git+https://github.com/paramiko/paramiko#egg=paramiko | 15:40 |
| reldan | # Pull in actual "you already have local installed checkouts of Fabric + | 15:40 |
| daemontool_ | it wasn't that slow... | 15:40 |
| reldan | # Paramiko" dev deps. | 15:40 |
| reldan | -r dev-requirements.txt | 15:40 |
| daemontool_ | vannif, do you remember the tests we did also for the demo? | 15:40 |
| daemontool_ | it was acceptable | 15:40 |
| daemontool_ | we should try to transfer few GB | 15:41 |
| reldan | I suppose it is because your ssh machine was not in hpcloud | 15:41 |
| vannif | no, sorry, ansible uses paramiko as a fall back | 15:41 |
| daemontool_ | ah ok | 15:41 |
| vannif | it uses something else | 15:41 |
| daemontool_ | we where using a devstack instance | 15:41 |
| daemontool_ | we'll check this in the future | 15:42 |
| daemontool_ | s/future/near future/ | 15:42 |
| daemontool_ | :) | 15:42 |
| daemontool_ | now let's fix the tests things | 15:42 |
| vannif | yes, definitely. it wouldn't be bad to have a summary of the upload/download speed of the backups | 15:42 |
| reldan | Sure! | 15:42 |
| reldan | Agree | 15:42 |
| reldan | Probably we need some test installation | 15:43 |
| reldan | With distribute metrics | 15:43 |
| reldan | Like http://graphite.wikidot.com/ | 15:44 |
| vannif | you know that the temperature of the exhaust gas from an engine is an important parameter to understand if the engines are healthy and in good operating conditions ? so should be the effective transfer rate for freezer engines :) | 15:44 |
| reldan | Agree. It will be great to actually do some improvement and see - that after merging we actually have improvement ) | 15:45 |
| vannif | yes. I'll help you with that if you want. it's interesting | 15:46 |
| reldan | Let’s say we can have a machine that will be do every 2 hours backup of 1 GB of data to different storages and report transfer speed/errors/… | 15:46 |
| reldan | We also can use https://www.elastic.co/products/kibana for gathering errors from multiple instances | 15:47 |
| reldan | https://www.elastic.co/videos/kibana-logstash | 15:47 |
| daemontool_ | that's th centralized logging tool used in helion :) | 15:47 |
| vannif | yes, with elasticsearch it will be easy to gather statistics and trigger alarms upon threshold crossings | 15:48 |
| vannif | anyway ... back on track | 15:48 |
| reldan | So let’s say we will have several machines and they will check new code from repo, run big tests with actual data and show errors and performance | 15:48 |
| reldan | Ok | 15:49 |
| daemontool_ | reldan, can you try to disable compression in paramiko if enabled by default? | 15:49 |
| vannif | we need to invertigate the effective speed and possible alternatives to paramiko, right ? | 15:49 |
| daemontool_ | we are always sending compressed data with freezer | 15:49 |
| reldan | Yes, I have checked that - no improvement | 15:49 |
| daemontool_ | ok | 15:49 |
| reldan | It will be great. I know how to reduce count of invoking listdir significantly, but still have no idea how to improve transfer | 15:50 |
| vannif | which should play the biggest part in large backups, right ? | 15:51 |
| reldan | It depends on how many backups do we already have in our repo | 15:51 |
| reldan | Now I have one additional listdir per zero-level-backup | 15:52 |
| reldan | I actually can reduce number - if I need only last one for example - I should get number or zero-level and then get all incremental only for this backup | 15:52 |
| reldan | So instead of n+1 backups I can get 2 | 15:53 |
| daemontool_ | reldan, did you take a looka at this? http://asyncssh.readthedocs.org/en/latest/ | 15:53 |
| reldan | not backups - listdirs | 15:53 |
| daemontool_ | check under "Direct TCP connections" | 15:53 |
| vannif | so, you are getting listdirs which could be avoided ? | 15:54 |
| reldan | We can try | 15:54 |
| reldan | Yes, I can define only zero-level backup that I need and make only one additional checks for increments instead of doing it for all zero-level backups | 15:55 |
| daemontool_ | ok, I 've never used it, just found it now, so t might not fit the purpose | 15:55 |
| reldan | Will see | 15:55 |
| vannif | it seems it requires python 3.4 | 15:56 |
| daemontool_ | there's a port for python 2 | 15:56 |
| daemontool_ | http://trollius.readthedocs.org/ | 15:56 |
| daemontool_ | + https://github.com/ronf/asyncssh | 15:56 |
| vannif | damn back ports ... we'll never get rid of python2 ... nor IPv4 :) | 15:57 |
| daemontool_ | haha | 15:57 |
| reldan | ))) | 15:57 |
| daemontool_ | let's move forward | 15:59 |
| daemontool_ | :) | 15:59 |
| vannif | shall we move forward ? | 15:59 |
| vannif | ok | 15:59 |
| vannif | :) | 15:59 |
| vannif | so. as I was saying, the plugins for devstack are ready | 15:59 |
| vannif | in review | 16:00 |
| vannif | they "work on my machine" :) | 16:00 |
| vannif | but you're encouraged to test them on your VMs | 16:00 |
| vannif | aah, only thing, you need to adjust the git repo to one on your local machine where you have the correct patchset, so that devstack will clone and install that repo/branch | 16:01 |
| vannif | thanks daemontool_ for the support :) | 16:02 |
| vannif | I'm also peeking to Saad's from time to time, he's working on oslo.conf and oslo.log, but I haven't still taken a deep look at that. maybe it will be in revirew soon | 16:03 |
| vannif | I was also evaluating the simplification of agent tasks. this is not a high priority I think. just an idea running on | 16:04 |
| vannif | as emerged in recent meetings, the plan is to have a sophisticated freezer-scheduler to manage the jobs, while the agent should be relatively dumb. | 16:04 |
| vannif | ATM the agent does a somewhat articulated task which is: lock the tables, take the snapshot, release the table lock, upload the backup, release the snapshot | 16:05 |
| vannif | this should be broken into pieces and controlled by the scheduler. | 16:05 |
| vannif | as reldan was suggesting, maybe using manager objects with specific responsibilities: snapshots, db locking/unlocking | 16:06 |
| vannif | each one instructed to act in specific moments of the backup "workflow" | 16:07 |
| vannif | kind of "hooks" | 16:07 |
| reldan | Oh, it will be great. I was trying to refactor lvm/shadow - but it’s really hard | 16:07 |
| vannif | yes, we also need to provide a consistent interface for lvm and shadow | 16:08 |
| reldan | it would be great to have single responsability objects, otherwise we are stopping database in one place and starting it in shadow/lvm code - it is conterintuitive | 16:08 |
| vannif | reldan, I can work on some diagrams and then we can discus them | 16:09 |
| vannif | exactly | 16:09 |
| reldan | We can just seat together and try to understand it :) | 16:09 |
| vannif | code is not just nested, it's even intermixed | 16:09 |
| reldan | Agree! | 16:09 |
| vannif | I think I'll send for review the small update to lvm.py I have been working on, and then plan for some more deep refactoring | 16:11 |
| reldan | Deal | 16:11 |
| vannif | so, let's move on. that's all from me. | 16:11 |
| vannif | if you don't have any question ... | 16:11 |
| vannif | m3m0_ | 16:12 |
| m3m0_ | I've been fixing some minor bugs in the ui | 16:12 |
| m3m0_ | specially for the backup_id for local and ssh modes | 16:13 |
| m3m0_ | the id is a string with slashes | 16:13 |
| m3m0_ | so the urls using those slashes breaks the ui | 16:13 |
| m3m0_ | but that is fixed now | 16:13 |
| vannif | and also the retrieval of the freezer-api endpoint from the keystone catalog, right ? | 16:13 |
| m3m0_ | yes, that was another bug, the retrieval was incorrect | 16:14 |
| m3m0_ | at least the logic of the function | 16:14 |
| m3m0_ | but now is fixed and merged | 16:14 |
| m3m0_ | and I'm working in UX improvements | 16:14 |
| vannif | any plan fo future improvements of the web-ui ? | 16:15 |
| m3m0_ | but they are not going to get into master soon | 16:15 |
| m3m0_ | yes, first of all, an overhaul of the readme | 16:15 |
| m3m0_ | then a specific tab for actions | 16:15 |
| m3m0_ | and another for clients | 16:15 |
| m3m0_ | improve unittesting | 16:16 |
| m3m0_ | and ui testing with selenium | 16:16 |
| m3m0_ | improvement in the action and job modal window | 16:16 |
| m3m0_ | add more visual clues for the user | 16:16 |
| m3m0_ | and an overview page as well | 16:17 |
| m3m0_ | so there is a lot to do for the ui | 16:17 |
| vannif | yes, the ui hasn't been high in the priority scale. but in the end it's the think that the user sees and interacts with. it sometimes help also the developers to understand if all they have been working on makes sense :) | 16:17 |
| m3m0_ | but I have a very limited time in freezer now | 16:17 |
| vannif | s/think/thing | 16:17 |
| m3m0_ | agree with that for us, it seems simple for new people they don't know what even a job means | 16:18 |
| m3m0_ | well at least in the context | 16:18 |
| m3m0_ | of backup and restore | 16:18 |
| openstackgerrit | Memo Garcia proposed stackforge/freezer-web-ui: Fix minor bugs in freezer dashboard https://review.openstack.org/235478 | 16:19 |
| vannif | you can work on it on the upcoming long and boring winter sundays :) | 16:19 |
| m3m0_ | winter is coming | 16:19 |
| vannif | good. anythink else to say ? | 16:19 |
| m3m0_ | yes, windows | 16:20 |
| m3m0_ | I need to rebase the changes and make sure that I didn't break the code by adding the support of the new code | 16:20 |
| m3m0_ | and that's it | 16:20 |
| vannif | btw, when we rework the code for the snapshots we well need your support for the windows snapshots | 16:21 |
| vannif | s/well/will | 16:21 |
| m3m0_ | of course | 16:21 |
| m3m0_ | just let me know | 16:21 |
| vannif | definitely. we'll need you: days are getting shorter, and the nights are long and full of terrors | 16:22 |
| vannif | thanks m3m0 | 16:22 |
| m3m0_ | it's dangerous to go alone :) | 16:22 |
| m3m0_ | you're welcome | 16:22 |
| vannif | szaher | 16:23 |
| szaher | Thanks vannif | 16:23 |
| vannif | I know you've been involved in other tasks | 16:23 |
| szaher | I have been working on using oslo.log now it works fine with freezer api | 16:24 |
| vannif | but you had also taken a look at oslo log and conf right ? | 16:24 |
| vannif | is it in review ? | 16:24 |
| szaher | not yet | 16:25 |
| szaher | I am just trying to add all files then I will commit | 16:26 |
| szaher | I need to change all freezer-api files to use oslo.log | 16:26 |
| szaher | and I will do that in one commit | 16:27 |
| szaher | after that I will move to freezer scheduler then agent | 16:27 |
| vannif | if you want some early advice, you can send in for review and mark it as work in progress | 16:27 |
| vannif | just to start the discussion and allow others to see anch give hints | 16:27 |
| vannif | s/anch/and | 16:28 |
| szaher | Ok, will do that today | 16:28 |
| szaher | there is also problem with pylint | 16:28 |
| vannif | I'll checkout the patchset and have a look at it. so we'll see how to deal with it | 16:30 |
| vannif | thanks szaher | 16:30 |
| szaher | thanks vannif :) | 16:30 |
| vannif | I think that's all since slashme hasn't been involved on the public side | 16:31 |
| vannif | anythink else to say ? | 16:31 |
| vannif | anyone ? | 16:31 |
| vannif | ok. thanks all | 16:36 |
| vannif | #endmeeting | 16:36 |
| freezerBot` | Meeting ended Thu Oct 15 16:36:49 2015 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:36 |
| freezerBot` | Minutes: http://meetbot.stefanocanepa.it/freezer/openstack-freezer/2015/openstack-freezer.2015-10-15-15.12.html | 16:36 |
| freezerBot` | Minutes (text): http://meetbot.stefanocanepa.it/freezer/openstack-freezer/2015/openstack-freezer.2015-10-15-15.12.txt | 16:36 |
| freezerBot` | Log: http://meetbot.stefanocanepa.it/freezer/openstack-freezer/2015/openstack-freezer.2015-10-15-15.12.log.html | 16:36 |
| openstack | Meeting ended Thu Oct 15 16:36:49 2015 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:36 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/2015_10_15/2015/2015_10_15.2015-10-15-15.12.html | 16:36 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/2015_10_15/2015/2015_10_15.2015-10-15-15.12.txt | 16:36 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/2015_10_15/2015/2015_10_15.2015-10-15-15.12.log.html | 16:36 |
| *** reldan has quit IRC | 17:02 | |
| *** bitblt has joined #openstack-freezer | 17:13 | |
| *** jonaspf has quit IRC | 17:38 | |
| *** bitblt has quit IRC | 18:16 | |
| *** reldan has joined #openstack-freezer | 18:27 | |
| *** marzif has joined #openstack-freezer | 18:36 | |
| *** marzif has quit IRC | 18:36 | |
| *** marzif has joined #openstack-freezer | 18:36 | |
| *** jonaspf has joined #openstack-freezer | 18:40 | |
| *** subscope has joined #openstack-freezer | 18:56 | |
| *** jonaspf has quit IRC | 19:00 | |
| *** jonaspf has joined #openstack-freezer | 19:20 | |
| *** jonaspf has quit IRC | 19:47 | |
| *** marzif has quit IRC | 19:50 | |
| *** subscope has quit IRC | 20:25 | |
| *** jonaspf has joined #openstack-freezer | 20:33 | |
| *** jonaspf has quit IRC | 20:48 | |
| *** jonaspf has joined #openstack-freezer | 20:50 | |
| *** reldan has quit IRC | 21:08 | |
| *** jonaspf has quit IRC | 21:11 | |
| *** reldan has joined #openstack-freezer | 21:27 | |
| *** reldan has quit IRC | 21:32 | |
| *** reldan has joined #openstack-freezer | 21:34 | |
| *** jonaspf has joined #openstack-freezer | 21:37 | |
| *** jonaspf has quit IRC | 22:19 | |
| *** dschroeder has quit IRC | 23:26 | |
| *** vannif_ has joined #openstack-freezer | 23:27 | |
| *** vannif has quit IRC | 23:32 | |
| *** ChanServ has quit IRC | 23:54 | |
| *** nullvariable has quit IRC | 23:54 | |
| *** module000 has quit IRC | 23:54 | |
| *** sc has quit IRC | 23:54 | |
| *** smekel has quit IRC | 23:54 | |
| *** freezerBot` has quit IRC | 23:55 | |
| *** vannif_ has quit IRC | 23:55 | |
| *** openstackgerrit has quit IRC | 23:55 | |
| *** jokke_ has quit IRC | 23:55 | |
| *** daemontool_ has quit IRC | 23:55 | |
| *** samuelBartel_ has quit IRC | 23:55 | |
| *** m3m0_ has quit IRC | 23:55 | |
| *** szaher has quit IRC | 23:55 | |
| *** frescof__ has quit IRC | 23:55 | |
| *** epheo has quit IRC | 23:55 | |
| *** Slashme_ has quit IRC | 23:55 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!