*** harlowja has quit IRC | 00:01 | |
clarkb | fungi it timed out in puppet then hadno indexes? | 00:02 |
---|---|---|
clarkb | I guess this is why wetest with the dev server | 00:02 |
clarkb | imacdonn: slaweq was going to try and look at it | 00:02 |
clarkb | fungi I wonder if the puppet didnt do what we wanted or if the indexes really were gone | 00:05 |
imacdonn | clarkb: yeah, looks like he did, but has not yet gotten to the bottom of it (based on comment from earlier today) | 00:05 |
openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: Fix noop job toDict() https://review.openstack.org/630409 | 00:08 |
fungi | Reindexed 107659 documents in changes index in 1184.9s (90.9/s) | 00:11 |
fungi | and now i've started the service | 00:12 |
*** jcoufal has quit IRC | 00:13 | |
fungi | seems we've got the webui back now | 00:13 |
fungi | by Gerrit Code Review (2.13.12-8-g6c12de7) | 00:14 |
fungi | looks right to me | 00:14 |
*** rascasoft has quit IRC | 00:16 | |
*** gyee has quit IRC | 00:21 | |
corvus | infra-root, config-core: first thing next week, i'll work on factoring out the docker build/promote stuff into zuul-jobs roles and send emails to the lists telling folks about the pattern. | 00:22 |
*** rascasoft has joined #openstack-infra | 00:22 | |
fungi | it's already awesome | 00:23 |
corvus | fungi: review-dev looks like a ui to me! | 00:23 |
fungi | soon to be reconsumable awesome, all the better | 00:23 |
*** sreejithp has quit IRC | 00:24 | |
*** tosky has quit IRC | 00:24 | |
corvus | fungi, clarkb: so it's no clear to me what we should do in prod with review | 00:25 |
*** rascasoft has quit IRC | 00:31 | |
*** rascasoft has joined #openstack-infra | 00:38 | |
fungi | i think the puppet exec to trigger an offline reindex on upgrade (or even a restart on upgrade) is an issue since it can't complete within the exec timeout | 00:38 |
fungi | better to do the upgrade without the restart triggered, and then perform a manual restart during a maintenance window? | 00:38 |
*** chandankumar has quit IRC | 00:46 | |
*** chandankumar has joined #openstack-infra | 00:47 | |
*** rascasoft has quit IRC | 00:53 | |
*** chandankumar has quit IRC | 00:53 | |
*** chandankumar has joined #openstack-infra | 00:54 | |
*** armax has quit IRC | 01:05 | |
*** rascasoft has joined #openstack-infra | 01:08 | |
*** slaweq has joined #openstack-infra | 01:13 | |
*** armax has joined #openstack-infra | 01:15 | |
clarkb | fungi or extend the timeout? that is what 20 minutes? that seems long but ya | 01:19 |
*** chandankumar has quit IRC | 01:20 | |
*** armax has joined #openstack-infra | 01:21 | |
clarkb | what I dont getis we shouldnt need to do an offline reindex | 01:21 |
clarkb | its the same version if gerrit (mostly) | 01:21 |
clarkb | I wonderif puppet didnt detect that properly and did the wrong thing | 01:22 |
*** chandankumar has joined #openstack-infra | 01:22 | |
*** slaweq has quit IRC | 01:25 | |
*** armax has quit IRC | 01:25 | |
*** rascasoft has quit IRC | 01:27 | |
*** chandankumar has quit IRC | 01:30 | |
*** chandankumar has joined #openstack-infra | 01:32 | |
*** rascasoft has joined #openstack-infra | 01:35 | |
*** wolverineav has quit IRC | 01:46 | |
*** wolverineav has joined #openstack-infra | 01:46 | |
*** wolverineav has quit IRC | 01:54 | |
*** pabelanger has joined #openstack-infra | 01:56 | |
*** dave-mccowan has quit IRC | 02:05 | |
*** rfolco has quit IRC | 02:07 | |
*** rascasoft has quit IRC | 02:10 | |
*** jamesmcarthur has joined #openstack-infra | 02:10 | |
*** slaweq has joined #openstack-infra | 02:11 | |
*** rascasoft has joined #openstack-infra | 02:11 | |
*** jamesmcarthur has quit IRC | 02:14 | |
*** slaweq has quit IRC | 02:15 | |
*** diablo_rojo has quit IRC | 02:23 | |
*** rascasoft has quit IRC | 02:33 | |
*** pabelanger has quit IRC | 02:36 | |
*** rascasoft has joined #openstack-infra | 02:37 | |
*** rfolco has joined #openstack-infra | 02:39 | |
openstackgerrit | Andriy Shevchenko proposed openstack/gertty master: Update home-page https://review.openstack.org/630120 | 02:48 |
*** rascasoft has quit IRC | 02:48 | |
*** rascasoft has joined #openstack-infra | 02:53 | |
*** yamamoto has joined #openstack-infra | 03:03 | |
*** yamamoto_ has joined #openstack-infra | 03:07 | |
*** yamamoto_ has quit IRC | 03:08 | |
*** yamamoto_ has joined #openstack-infra | 03:09 | |
*** yamamoto has quit IRC | 03:10 | |
*** rascasoft has quit IRC | 03:10 | |
*** yamamoto_ has quit IRC | 03:16 | |
*** apetrich has quit IRC | 03:16 | |
*** anteaya has quit IRC | 03:16 | |
*** rascasoft has joined #openstack-infra | 03:19 | |
*** zaneb has joined #openstack-infra | 03:29 | |
*** rkukura has quit IRC | 03:41 | |
*** armax has joined #openstack-infra | 03:53 | |
openstackgerrit | James E. Blair proposed openstack-infra/system-config master: WIP: run a gerrit container https://review.openstack.org/630406 | 04:03 |
*** zaneb has quit IRC | 04:05 | |
*** slaweq has joined #openstack-infra | 04:11 | |
*** slaweq has quit IRC | 04:16 | |
*** hwoarang has quit IRC | 04:24 | |
*** hwoarang has joined #openstack-infra | 04:24 | |
*** rfolco has quit IRC | 04:27 | |
*** rascasoft has quit IRC | 04:34 | |
*** rascasoft has joined #openstack-infra | 04:40 | |
*** weshay has quit IRC | 05:19 | |
*** rascasoft has quit IRC | 05:30 | |
*** ykarel|away has joined #openstack-infra | 05:32 | |
*** rascasoft has joined #openstack-infra | 05:39 | |
*** martinkennelly has quit IRC | 05:41 | |
*** rascasoft has quit IRC | 05:45 | |
*** rascasoft has joined #openstack-infra | 05:51 | |
*** rascasoft has quit IRC | 05:56 | |
*** whoami-rajat has joined #openstack-infra | 06:02 | |
*** rascasoft has joined #openstack-infra | 06:03 | |
*** slaweq has joined #openstack-infra | 06:11 | |
*** slaweq has quit IRC | 06:16 | |
*** rascasoft has quit IRC | 06:18 | |
*** lpetrut has joined #openstack-infra | 06:47 | |
*** rascasoft has joined #openstack-infra | 07:06 | |
*** slaweq has joined #openstack-infra | 07:11 | |
*** slaweq has quit IRC | 07:16 | |
*** armax has quit IRC | 07:18 | |
*** slaweq has joined #openstack-infra | 07:50 | |
*** lpetrut has quit IRC | 08:00 | |
*** ykarel|away has quit IRC | 08:00 | |
*** slaweq has quit IRC | 08:05 | |
*** irdr has quit IRC | 08:08 | |
*** jamesmcarthur has joined #openstack-infra | 08:11 | |
*** jamesmcarthur has quit IRC | 08:16 | |
*** lpetrut has joined #openstack-infra | 08:18 | |
*** jbadiapa has quit IRC | 08:29 | |
*** slaweq has joined #openstack-infra | 08:29 | |
*** cmorpheus is now known as cmurphy | 08:56 | |
openstackgerrit | Merged openstack-infra/project-config master: Add merge-output-to-logs to base-test https://review.openstack.org/629584 | 08:59 |
*** lpetrut has quit IRC | 09:14 | |
*** rascasoft has quit IRC | 09:47 | |
*** rascasoft has joined #openstack-infra | 09:48 | |
*** e0ne has joined #openstack-infra | 09:50 | |
*** e0ne has quit IRC | 09:53 | |
*** ykarel|away has joined #openstack-infra | 10:08 | |
openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: WIP: Match tag items against containing branches https://review.openstack.org/578557 | 10:11 |
*** remi_ness has joined #openstack-infra | 10:41 | |
*** d0ugal has joined #openstack-infra | 10:41 | |
*** d0ugal has quit IRC | 10:59 | |
*** jtomasek has joined #openstack-infra | 11:01 | |
*** d0ugal has joined #openstack-infra | 11:02 | |
*** jaosorior has quit IRC | 11:07 | |
*** d0ugal has quit IRC | 11:12 | |
*** yamamoto has joined #openstack-infra | 11:14 | |
*** yamamoto has quit IRC | 11:18 | |
*** tosky has joined #openstack-infra | 11:38 | |
*** ykarel|away has quit IRC | 11:41 | |
*** dkehn has quit IRC | 12:06 | |
*** whoami-rajat has quit IRC | 12:11 | |
*** martinkennelly has joined #openstack-infra | 12:56 | |
*** dkehn has joined #openstack-infra | 13:24 | |
*** evrardjp has quit IRC | 13:34 | |
*** rfolco has joined #openstack-infra | 13:47 | |
fungi | clarkb: i *think* the puppet manifest just runs an offline reindex any time the warfile changes | 14:08 |
fungi | looking | 14:08 |
fungi | yeah, the gerrit-reindex exec has subscribe => File['/home/gerrit2/review_site/bin/gerrit.war'] | 14:10 |
fungi | oh, though it's wrapped in a conditional for $offline_reindex which defaults to false in the main gerrit class, so unless we're overriding that, it's an automatic behavior of gerrit init | 14:13 |
fungi | yeah, we don't override that, so i don't think we actually fired the gerrit-reindex exec | 14:17 |
fungi | Jan 11 22:58:18 review-dev01 puppet-user[30401]: (/Stage[main]/Gerrit/Exec[gerrit-init]) Failed to call refresh: /etc/init.d/gerrit stop; /usr/bin/java -jar /home/gerrit2/review_site/bin/gerrit.war init -d /home/gerrit2/review_site --batch --no-auto-start returned 128 instead of one of [0] | 14:20 |
fungi | oh, i bet that's because before that we have | 14:21 |
fungi | Jan 11 22:58:13 review-dev01 puppet-user[30401]: (/Stage[main]/Gerrit/Exec[gerrit-initial-init]) Triggered 'refresh' from 1 events | 14:21 |
fungi | though there's also | 14:22 |
fungi | Jan 11 23:03:18 review-dev01 puppet-user[30401]: (/Stage[main]/Gerrit/Exec[gerrit-initial-index]) Failed to call refresh: Command exceeded timeout | 14:22 |
fungi | why are we triggering gerrit-initial-index? | 14:22 |
fungi | rereading the source again | 14:22 |
fungi | triggered by subscribe => Exec['gerrit-initial-init'] | 14:24 |
fungi | which also has subscribe => File['/home/gerrit2/review_site/bin/gerrit.war'] | 14:25 |
fungi | so any time we replace /home/gerrit2/review_site/bin/gerrit.war we trigger gerrit-initial-init which in turn triggers gerrit-initial-index | 14:25 |
fungi | even though the comment for gerrit-initial-index says "We need to make the initial index for a fresh install." so presumably is not meant to happen on upgrade | 14:26 |
fungi | reusing the gerrit container and leaving this spaghetti behind is looking better and better | 14:27 |
*** roman_g has joined #openstack-infra | 14:32 | |
*** rfolco has quit IRC | 14:54 | |
*** jtomasek has quit IRC | 15:00 | |
*** kgiusti has left #openstack-infra | 15:10 | |
*** yamamoto has joined #openstack-infra | 15:17 | |
*** yamamoto has quit IRC | 15:22 | |
openstackgerrit | James E. Blair proposed openstack-infra/system-config master: WIP: run a gerrit container https://review.openstack.org/630406 | 15:31 |
openstackgerrit | Jeremy Stanley proposed openstack-infra/jeepyb master: Correct blueprint topic links for modern Gerrit https://review.openstack.org/630461 | 15:37 |
clarkb | fungi :( I was worried about that. I'll have to see if there is an easy fix on monday. I expect its mostly an issue because we had to reindex after every upgrade for a while andnow it is mostly online amd our puppet didnt catch up | 15:38 |
fungi | well, and i think we hacked triggers into the manifest in the past without noticing they resulted in reindexing more often than was strictly required | 15:39 |
*** rfolco has joined #openstack-infra | 15:49 | |
*** jamesmcarthur has joined #openstack-infra | 15:52 | |
openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config master: Add puppetforge publish and add to puppet release https://review.openstack.org/627573 | 15:53 |
fungi | tobias-urdin: ^ sorry for the delay | 15:53 |
*** e0ne has joined #openstack-infra | 15:56 | |
*** tosky has quit IRC | 15:58 | |
*** e0ne has quit IRC | 16:05 | |
corvus | clarkb, fungi: yeah, that sounds about right; i think that was required in the past | 16:18 |
mordred | fungi: yah - 'gerrit in container' is gonna be much cleaner I think | 16:18 |
mordred | corvus: in the docker-compose you have there - you just going with the upstream image rather than the locally built reason for POC expediency? | 16:20 |
mordred | (I like the patch!) | 16:20 |
corvus | mordred: yeah, i just wanted to start sketching out a framework | 16:21 |
mordred | cool. I think the shape of that sketch is fantastic | 16:21 |
openstackgerrit | James E. Blair proposed openstack-infra/system-config master: WIP: run a gerrit container https://review.openstack.org/630406 | 16:23 |
corvus | maybe someday zuul will agree :) | 16:23 |
mordred | corvus: bah | 16:23 |
mordred | corvus: I'm gonna brainstorm a little bit about a way to take the gerrit dockerfile patch I've got up but have that patch live in system-config (or somewhere else) and see if we can't just point it at the upstream gerrit git repo using multi-repo job in zuul rather than needing to maintain the fork / branch for each release | 16:25 |
corvus | mordred: oh you have a gerrit dockerfile patch? i didn't see that | 16:25 |
mordred | yeah - you even +2'd it :) | 16:25 |
mordred | https://review.openstack.org/#/c/610395 | 16:26 |
mordred | it's for 2.15 | 16:26 |
corvus | that was so last year! :) | 16:26 |
mordred | I should port in your new docker roles (althoughj I see in scrollback you're going to generalize them into zuul-jobs - so maybe I'll wait for that) | 16:26 |
corvus | ++ | 16:27 |
mordred | but maybe if we do a job that has required-project: https://gerrit.googlesource.com/gerrit - and a pre-playbook that just copies our dockerfile and does the modifications for the plugins | 16:27 |
mordred | something something | 16:28 |
mordred | corvus: in fact, if we went that route, we could get rid of the submodule section here: https://review.openstack.org/#/c/610395/4/.gitmodules | 16:31 |
*** martinkennelly has quit IRC | 16:31 | |
mordred | and also add its-base, its-storyboard and javamelody as required-projects | 16:31 |
corvus | and all supporting depends-on. i'm digging it. | 16:32 |
mordred | we'd just need to add https://gerrit-review.googlesource.com as a source :) | 16:33 |
corvus | seems a reasonable thing to do | 16:33 |
corvus | i don't think it has an ssh listener though | 16:33 |
mordred | no? poop. I mean - we could also just use git driver | 16:34 |
corvus | that'll at least get us the projects | 16:35 |
corvus | we have support for reporting over http; we can probably add querying to that, and make ssh optional, then we'd only lack events, but depends-on would work. | 16:35 |
mordred | do they have http events? | 16:35 |
corvus | afaik no; we'd have to do gertty-style queries | 16:36 |
mordred | nod | 16:36 |
corvus | (which means maintaining local state) | 16:36 |
mordred | well- gerrithub ci is doing ci on the gerrit gerrit - so maybe there is an ssh interface that they expose to some people? | 16:37 |
mordred | or maybe it's just also polling | 16:38 |
*** ykarel|away has joined #openstack-infra | 16:42 | |
*** jamesmcarthur has quit IRC | 16:43 | |
*** jonher has quit IRC | 16:44 | |
*** jamesmcarthur has joined #openstack-infra | 16:46 | |
*** jamesmcarthur has quit IRC | 16:47 | |
*** ykarel|away has quit IRC | 16:48 | |
*** ykarel|away has joined #openstack-infra | 16:48 | |
*** whoami-rajat has joined #openstack-infra | 16:49 | |
*** imacdonn has quit IRC | 16:51 | |
*** imacdonn has joined #openstack-infra | 16:52 | |
*** ykarel|away has quit IRC | 16:54 | |
corvus | mordred: success! https://review.openstack.org/630406 | 16:59 |
openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Build a bazel image https://review.openstack.org/630465 | 17:00 |
mordred | corvus: neat! | 17:00 |
mordred | corvus: testinfra produces a junit.xml? | 17:01 |
* mordred hadn't noticed that before | 17:02 | |
*** kopecmartin|off has quit IRC | 17:03 | |
*** kopecmartin has joined #openstack-infra | 17:04 | |
*** ykarel|away has joined #openstack-infra | 17:31 | |
*** yamamoto has joined #openstack-infra | 17:39 | |
*** yamamoto has quit IRC | 17:43 | |
*** yamamoto has joined #openstack-infra | 17:44 | |
*** armax has joined #openstack-infra | 17:50 | |
*** ykarel|away has quit IRC | 17:58 | |
*** rkukura has joined #openstack-infra | 18:00 | |
*** armax has quit IRC | 18:16 | |
openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Build a bazel image https://review.openstack.org/630465 | 18:22 |
openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Add support for passing path to docker build https://review.openstack.org/630467 | 18:22 |
openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Build a gerrit image https://review.openstack.org/630468 | 18:22 |
mordred | corvus: ^^ ok. there's the whole thing with dockerfiles in system-config and referencing gerrit source directly rather than in our fork | 18:22 |
mordred | corvus: I didn't do any required-projects things yet - and also we probably want to talk about how to handle major versions with the tag/promote workflow | 18:24 |
mordred | corvus: I could see us wanting an opendev/gerrit:2.15 and opendev/gerrit:2.16 for instance | 18:25 |
*** yamamoto has quit IRC | 18:32 | |
clarkb | add a tag triggered job that tags docoer image based on git tag? | 18:33 |
clarkb | runs alongside the commit by commit promotion pipeline | 18:34 |
clarkb | if you want to avoid building new images you could lookup the change that matches the tag and update the docoer tag on that ? | 18:34 |
*** lpetrut has joined #openstack-infra | 18:36 | |
*** armax has joined #openstack-infra | 18:37 | |
openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Build a gerrit image https://review.openstack.org/630468 | 18:45 |
openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Build a gerrit image https://review.openstack.org/630468 | 18:47 |
mordred | clarkb: turns out corvus had already thought of the thign I was thinking about ... and that last patch uses it (you can give a list of tags to promote to) | 18:48 |
mordred | clarkb: I mostly wanted to make sure that once we're running 2.15 and we go to start building 2.16 we don't wind up overwriting the 2.15 images | 18:48 |
mordred | but I think that latest patch will do what we want - and we should be able to happily have builds for 2.15 and 2.16 coexist without borking ourselves | 18:49 |
openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Build a gerrit image https://review.openstack.org/630468 | 18:50 |
*** e0ne has joined #openstack-infra | 18:50 | |
mordred | corvus, clarkb: also - I'd like to say how pleasing it is that https://review.openstack.org/#/c/630468/ is much smaller than https://review.openstack.org/#/c/610395 | 18:51 |
*** whoami-rajat has quit IRC | 18:58 | |
*** whoami-rajat has joined #openstack-infra | 19:03 | |
*** yamamoto has joined #openstack-infra | 19:05 | |
*** roman_g has quit IRC | 19:07 | |
*** yamamoto has quit IRC | 19:11 | |
*** jonher has joined #openstack-infra | 19:11 | |
*** lpetrut has quit IRC | 19:14 | |
*** efried has quit IRC | 19:28 | |
*** efried1 has joined #openstack-infra | 19:28 | |
*** efried1 is now known as efried | 19:29 | |
*** yamamoto has joined #openstack-infra | 19:41 | |
*** e0ne has quit IRC | 19:45 | |
openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Revert "docs: Added missing -i on docker-compose up command" https://review.openstack.org/630470 | 19:45 |
*** yamamoto has quit IRC | 19:46 | |
openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Gracefully close db connections in tests https://review.openstack.org/630471 | 19:51 |
openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Enforce sql connections for scheduler and web https://review.openstack.org/630472 | 19:51 |
openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Build a gerrit image https://review.openstack.org/630468 | 20:05 |
openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Put bazel image into gerrit Dockerfile https://review.openstack.org/630473 | 20:05 |
mordred | corvus, clarkb: ^^ that last patch of the stack is a sake-of-argument - if we prefer that version I can squash it down | 20:07 |
*** jonher has quit IRC | 20:16 | |
*** jonher_ has joined #openstack-infra | 20:16 | |
*** jonher_ is now known as jonher | 20:16 | |
*** yamamoto has joined #openstack-infra | 20:35 | |
*** yamamoto has quit IRC | 20:40 | |
*** eernst has joined #openstack-infra | 20:48 | |
openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Fix documented invocation of docker-compose https://review.openstack.org/630474 | 20:53 |
*** hongbin has joined #openstack-infra | 20:54 | |
*** yamamoto has joined #openstack-infra | 21:08 | |
*** whoami-rajat has quit IRC | 21:08 | |
corvus | mordred: yeah, i vote drop the bazel image | 21:12 |
*** yamamoto has quit IRC | 21:12 | |
corvus | mordred: when you squash, tabs need fixing in 630468 | 21:13 |
*** remi_ness has quit IRC | 21:18 | |
corvus | mordred: all reviewed | 21:19 |
mordred | corvus: silly tabs | 21:25 |
*** kopecmartin has quit IRC | 21:27 | |
openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Add support for passing path to docker build https://review.openstack.org/630467 | 21:35 |
openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Build a gerrit image https://review.openstack.org/630468 | 21:35 |
mordred | corvus: ^^ fixed tabs and docs - thanks. also, squashed | 21:36 |
*** yamamoto has joined #openstack-infra | 21:42 | |
*** yamamoto has quit IRC | 21:47 | |
openstackgerrit | Merged openstack-infra/zuul master: Fix documented invocation of docker-compose https://review.openstack.org/630474 | 21:51 |
*** vultaire has joined #openstack-infra | 21:52 | |
*** yamamoto has joined #openstack-infra | 21:59 | |
*** eernst has quit IRC | 22:04 | |
*** jonher_ has joined #openstack-infra | 22:05 | |
*** jonher has quit IRC | 22:08 | |
*** jonher_ is now known as jonher | 22:08 | |
*** slaweq has quit IRC | 22:14 | |
*** jtomasek has joined #openstack-infra | 22:41 | |
*** rascasoft has quit IRC | 23:13 | |
*** rkukura_ has joined #openstack-infra | 23:14 | |
*** rkukura has quit IRC | 23:17 | |
*** rkukura_ is now known as rkukura | 23:17 | |
*** quiquell|off has quit IRC | 23:20 | |
*** quiquell has joined #openstack-infra | 23:21 | |
*** quiquell has quit IRC | 23:22 | |
*** quiquell has joined #openstack-infra | 23:22 | |
*** quiquell has quit IRC | 23:23 | |
*** quiquell has joined #openstack-infra | 23:23 | |
*** mtreinish has quit IRC | 23:36 | |
*** rascasoft has joined #openstack-infra | 23:43 | |
*** rascasoft has quit IRC | 23:49 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!