| *** sgw has quit IRC | 00:09 | |
| *** Defolos has quit IRC | 00:13 | |
| *** toabctl has quit IRC | 00:19 | |
| *** toabctl has joined #zuul | 00:22 | |
| *** sgw has joined #zuul | 00:28 | |
| *** mattw4 has quit IRC | 00:29 | |
| *** igordc has quit IRC | 00:33 | |
| *** igordc has joined #zuul | 00:53 | |
| openstackgerrit | Tristan Cacqueray proposed zuul/zuul master: Implement zookeeper-auth https://review.opendev.org/619156 | 01:31 |
|---|---|---|
| *** nhicher has quit IRC | 01:38 | |
| *** nhicher has joined #zuul | 01:38 | |
| *** igordc has quit IRC | 02:07 | |
| *** Goneri has quit IRC | 02:24 | |
| *** sanjayu_ has joined #zuul | 02:42 | |
| *** bhavikdbavishi has joined #zuul | 02:51 | |
| *** sanjayu_ has quit IRC | 02:53 | |
| *** raukadah is now known as chandankumar | 03:02 | |
| *** rlandy|bbl is now known as rlandy | 04:29 | |
| *** rlandy has quit IRC | 04:53 | |
| *** zxiiro has joined #zuul | 05:00 | |
| *** swest has joined #zuul | 05:28 | |
| *** evrardjp has quit IRC | 05:35 | |
| *** evrardjp has joined #zuul | 05:35 | |
| *** reiterative has quit IRC | 05:40 | |
| *** reiterative has joined #zuul | 05:40 | |
| *** marvs has joined #zuul | 06:06 | |
| *** saneax has joined #zuul | 06:36 | |
| *** pabelanger has quit IRC | 06:41 | |
| *** threestrands has quit IRC | 06:48 | |
| *** jcapitao_off has joined #zuul | 07:20 | |
| *** jcapitao_off is now known as jcapitao | 07:22 | |
| *** migi has quit IRC | 07:40 | |
| openstackgerrit | Benedikt Löffler proposed zuul/zuul master: Fix override variables in zuul_return https://review.opendev.org/711002 | 07:43 |
| *** Defolos has joined #zuul | 07:50 | |
| AJaeger | zuul-maint, FYI, zuul-ci.org is down currently. If you add "23.253.245.150 zuul-ci.org" to /etc/hosts, you can reach it. Further discussion on #openstack-infra | 08:22 |
| *** jpena|off is now known as jpena | 08:26 | |
| *** avass has joined #zuul | 08:28 | |
| *** saneax has quit IRC | 08:29 | |
| *** saneax has joined #zuul | 08:30 | |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Allow check runs to be configured as required status in pipeline config https://review.opendev.org/711241 | 08:34 |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Allow check runs to be configured as required status in pipeline config https://review.opendev.org/711241 | 08:45 |
| *** hashar has joined #zuul | 08:49 | |
| *** hashar_ has joined #zuul | 08:50 | |
| *** hashar__ has joined #zuul | 08:53 | |
| *** hashar has quit IRC | 08:54 | |
| *** hashar_ has quit IRC | 08:55 | |
| openstackgerrit | Ian Wienand proposed opendev/zone-zuul-ci.org master: Use static.opendev.org https://review.opendev.org/711403 | 08:56 |
| openstackgerrit | Merged opendev/zone-zuul-ci.org master: git.zuul-ci.org : point to static.opendev.org https://review.opendev.org/710142 | 09:02 |
| *** hashar__ is now known as hashar | 09:03 | |
| openstackgerrit | Merged opendev/zone-zuul-ci.org master: Use static.opendev.org https://review.opendev.org/711403 | 09:06 |
| *** zxiiro has quit IRC | 09:07 | |
| *** Shrews has quit IRC | 09:07 | |
| *** irclogbot_2 has quit IRC | 09:08 | |
| *** portdirect has quit IRC | 09:08 | |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Don't rely on report-build-page when building the buildset result url https://review.opendev.org/711406 | 09:19 |
| *** sugaar has joined #zuul | 09:20 | |
| *** tosky has joined #zuul | 09:25 | |
| AJaeger | zuul-maint, zuul-ci.org should be up again. | 09:31 |
| *** bhavikdbavishi has quit IRC | 09:35 | |
| *** sshnaidm|afk is now known as sshnaidm | 09:37 | |
| *** zxiiro has joined #zuul | 09:43 | |
| *** Shrews has joined #zuul | 09:43 | |
| *** irclogbot_2 has joined #zuul | 09:43 | |
| *** portdirect has joined #zuul | 09:43 | |
| *** openstackstatus has quit IRC | 09:45 | |
| *** jcapitao has quit IRC | 10:20 | |
| *** jcapitao has joined #zuul | 10:21 | |
| *** armstrongs has joined #zuul | 10:26 | |
| *** jcapitao has quit IRC | 10:32 | |
| *** jcapitao has joined #zuul | 10:32 | |
| *** hashar has quit IRC | 10:38 | |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Tests bindep role on all-platforms https://review.opendev.org/708704 | 11:00 |
| openstackgerrit | Sorin Sbarnea proposed zuul/zuul-jobs master: Improve ensure-tox role https://review.opendev.org/708642 | 11:13 |
| *** pabelanger has joined #zuul | 11:30 | |
| *** zxiiro has quit IRC | 11:48 | |
| *** jcapitao is now known as jcapitao_lunch | 11:54 | |
| *** rlandy has joined #zuul | 12:02 | |
| openstackgerrit | Benedikt Löffler proposed zuul/zuul master: Fix override variables in zuul_return https://review.opendev.org/711002 | 12:07 |
| *** jpena is now known as jpena|lunch | 12:09 | |
| *** dpawlik has quit IRC | 12:14 | |
| *** dpawlik has joined #zuul | 12:15 | |
| *** jcapitao_lunch has quit IRC | 12:19 | |
| *** jcapitao_lunch has joined #zuul | 12:21 | |
| *** dpawlik has quit IRC | 12:22 | |
| *** dpawlik has joined #zuul | 12:36 | |
| *** armstrongs has quit IRC | 12:37 | |
| openstackgerrit | Benedikt Löffler proposed zuul/zuul master: Fix override variables in zuul_return https://review.opendev.org/711002 | 12:51 |
| openstackgerrit | Benedikt Löffler proposed zuul/zuul master: Fix override variables in zuul_return https://review.opendev.org/711002 | 12:56 |
| *** jcapitao_lunch is now known as jcapitao | 13:15 | |
| *** jamesmcarthur has joined #zuul | 13:20 | |
| *** hashar has joined #zuul | 13:22 | |
| *** jamesmcarthur has quit IRC | 13:38 | |
| *** jamesmcarthur has joined #zuul | 13:40 | |
| *** jamesmcarthur has quit IRC | 13:45 | |
| *** avass has quit IRC | 13:47 | |
| *** jamesmcarthur has joined #zuul | 14:09 | |
| *** jamesmcarthur has quit IRC | 14:15 | |
| *** flaper87 has joined #zuul | 14:27 | |
| *** jpena|lunch is now known as jpena | 14:29 | |
| *** Goneri has joined #zuul | 14:31 | |
| *** hashar has quit IRC | 14:32 | |
| *** hashar has joined #zuul | 14:32 | |
| *** sgw has quit IRC | 14:33 | |
| *** jamesmcarthur has joined #zuul | 14:35 | |
| *** flaper87 has quit IRC | 14:38 | |
| *** flaper87 has joined #zuul | 14:38 | |
| *** jamesmcarthur has quit IRC | 14:41 | |
| *** jcapitao has quit IRC | 14:42 | |
| *** jamesmcarthur has joined #zuul | 14:42 | |
| *** jcapitao has joined #zuul | 14:44 | |
| *** sgw has joined #zuul | 14:49 | |
| *** jamesmcarthur has quit IRC | 14:58 | |
| *** jamesmcarthur has joined #zuul | 14:58 | |
| *** hashar has quit IRC | 15:16 | |
| *** jamesmcarthur has quit IRC | 15:23 | |
| *** jamesmcarthur has joined #zuul | 15:27 | |
| *** jamesmcarthur has quit IRC | 15:36 | |
| *** jamesmcarthur has joined #zuul | 15:45 | |
| *** jamesmcarthur has joined #zuul | 15:45 | |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Provide some documentation for the checks API implementation https://review.opendev.org/711493 | 15:55 |
| *** jcapitao is now known as jcapitao_afk | 15:56 | |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Make github file annotation levels configurable via zuul return https://review.opendev.org/711179 | 15:58 |
| mordred | corvus: ^^ that seems like a thing we should point out to our gerrit friends as a potential improvement to the robot_comments feature | 16:02 |
| mordred | paladox: ^^ | 16:02 |
| *** jcapitao_afk is now known as jcapitao | 16:10 | |
| *** mattw4 has joined #zuul | 16:11 | |
| *** jcapitao has quit IRC | 16:12 | |
| openstackgerrit | Felix Edel proposed zuul/zuul master: Dequeue changes via github checks API https://review.opendev.org/709135 | 16:16 |
| *** jcapitao has joined #zuul | 16:18 | |
| *** felixedel has joined #zuul | 16:19 | |
| *** bhavikdbavishi has joined #zuul | 16:25 | |
| felixedel | mnaser, pabelanger, corvus, mordred: Tobias told me about the discussion you had two days ago about some aspects of the current checks API implementation: I've tried to cover them in the documentation https://review.opendev.org/#/c/711493/. I'm not sure if that's the right place to put it, but I think it wouldn't be bad to have it somewhere :-) The other topic was about whether or not a check can be used as pipeline.require - currently not | 16:26 |
| felixedel | , but with that it would be possible https://review.opendev.org/#/c/711241/ | 16:26 |
| zenkuro | hi, is there a way to chech zuul config(config project)? to debug errors | 16:27 |
| corvus | felixedel: thanks! | 16:29 |
| AJaeger | The infra manual says "Visit the `OpenStack Zuul App <https://github.com/apps/openstack-zuul>`_ page on GitHub" - but that URL is not working for me. What is the current URL? | 16:33 |
| paladox | mordred: I think they’ve done improvements in that area. | 16:34 |
| *** felixedel has quit IRC | 16:37 | |
| mordred | paladox: cool. so we might be able to follow up with that and add support for gerrit too | 16:38 |
| mordred | AJaeger: https://github.com/apps/opendev-zuul | 16:39 |
| *** zxiiro has joined #zuul | 16:40 | |
| paladox | Yup | 16:40 |
| AJaeger | thx, mordred | 16:41 |
| corvus | zenkuro: zuul itself should tell you if a proposed change to a config-project has errors | 16:46 |
| corvus | zenkuro: here's an example of what that looks like from just this morning: https://review.opendev.org/711474 | 16:47 |
| *** Defolos has quit IRC | 16:53 | |
| corvus | pabelanger, tobiash: can you take a look at my comment on https://review.opendev.org/711241 ? | 17:01 |
| *** jamesmcarthur_ has joined #zuul | 17:21 | |
| *** igordc has joined #zuul | 17:22 | |
| tobiash | corvus: you're right | 17:24 |
| *** jamesmcarthur has quit IRC | 17:24 | |
| *** evrardjp has quit IRC | 17:35 | |
| *** evrardjp has joined #zuul | 17:35 | |
| openstackgerrit | Merged zuul/nodepool master: Use explicit provides/requires for container jobs https://review.opendev.org/710115 | 17:46 |
| clarkb | corvus: tristanC left some thoughts on https://review.opendev.org/#/c/619155/32 I think my biggest concern is that we don't appear to be testing with ssl? but expect people to use ssl in production? | 18:02 |
| Shrews | tristanC: corvus: the nodepool side of zk-auth lgtm except for the script name referenced in the release notes | 18:04 |
| corvus | clarkb: good points. i think we should figure out the tls story. do we need explicit support in zuul/nodepool for that? adding it to tests would help us confirm. | 18:04 |
| corvus | clarkb: also your point on zk.py line 965 relates to my point on configuration.rst line 146 | 18:05 |
| clarkb | corvus: ya whether it should be structured or not. I can go either way, it just felt ewird to be converting back and forth in several places | 18:05 |
| corvus | yep. | 18:05 |
| corvus | Shrews: i walked back my +2 a little bit after seeing the zuul change -- i think there are a couple of details we should hammer out first | 18:06 |
| corvus | but yeah, i think we're just about there | 18:07 |
| *** jcapitao is now known as jcapitao_off | 18:10 | |
| *** jamesmcarthur_ has quit IRC | 18:10 | |
| Shrews | corvus: does zuul create any zuul-only znodes currently? i can't recall | 18:12 |
| corvus | Shrews: autohold? | 18:12 |
| Shrews | ooh, yeah. so only running the update script from nodepool (as suggested in the release notes) is not enough | 18:13 |
| Shrews | we need a separate one | 18:13 |
| *** jcapitao_off has quit IRC | 18:15 | |
| Shrews | left comments on the zuul change | 18:15 |
| *** chandankumar is now known as raukadah | 18:18 | |
| fungi | i've been asked to provide an overview of zuul at the next cd foundation interoperability sig meeting on march 19. i'm planning to give a little history and talk about what specific features of zuul were designed with interoperability in mind. will circulate my draft slide deck next week to get input from the community, so be thinking about whether there's anything in particular you want me to make sure to | 18:21 |
| fungi | cover | 18:21 |
| mordred | fungi: cool | 18:22 |
| *** Goneri has quit IRC | 18:23 | |
| fungi | they've so far been getting presentations from tekton, spinnaker, et cetera. i want to be sure i explain at a low level how zuul is different | 18:24 |
| mordred | fungi: terms/buzzwords: multi-repo jobs, multi-node jobs, multi-source jobs, multi-tenant service, scalable-service, gating, speculative-execution, multi-zuul ecosystem, | 18:25 |
| mordred | fungi: and I can't buzzword it - but "we're not opinonated about how you write your app, we're good if you use baremetal, VMs or containers" | 18:25 |
| corvus | cross-repo-dependencies, cross-source-dependencies? | 18:26 |
| *** jpena is now known as jpena|off | 18:26 | |
| fungi | yeah, in case it's not obvious, this is what drove me to start fleshing out our glossary doc | 18:26 |
| mordred | fungi: :) | 18:26 |
| mordred | fungi: oh - there'sa. REALLY important difference | 18:27 |
| *** Goneri has joined #zuul | 18:27 | |
| fungi | but yeah, i like the point about flexibility on where workloads run. a bunch of newer ci systems assume all your jobs are containers and you have a kubernetes | 18:28 |
| mordred | in zuul, overall workflows are global and shared - we have a per-tenant description of a few discreet ways that all changes react to stimuli and report back | 18:28 |
| mordred | this is vastly different to the model they're all used to where triggers and reporters are attached to the individual definition of a job | 18:28 |
| fungi | that's a nice systemic description, i like that. thanks! | 18:29 |
| *** bhavikdbavishi has quit IRC | 18:29 | |
| mordred | it's possible one of the largest semantic divergences | 18:29 |
| fungi | right, define jobs anywhere, instantiate them where you need them | 18:29 |
| mordred | yeah - but not just that ... because tekton has job libraries | 18:30 |
| fungi | like .h vs .c files | 18:30 |
| clarkb | fungi: mordred: another major difference after poking around at these things to possibly writea thing that I got side tracked from is multiple code system inputs. Many will only talk to github | 18:30 |
| clarkb | speculative execution support as a top level feature (and from that scalable proper gating) is another huge divergence | 18:30 |
| mordred | but that the system overall defines "as a group of peope we want to always respond to pull requests when they get a lgtm vote and we always want to then merge them" | 18:30 |
| clarkb | all the other platforms basically force you to figure that out on your own | 18:30 |
| fungi | mordred: ahh, i get what you're saying. triggers being tied to the pipeline definitions | 18:30 |
| mordred | yes | 18:30 |
| clarkb | (and prow punts and just does batching) | 18:31 |
| mordred | it's a HUGE semantic difference | 18:31 |
| fungi | this is helpful as i have very limited user experience with ci systems other than zuul | 18:31 |
| *** mattw4 has quit IRC | 18:32 | |
| mordred | fungi: the other huge thing is speculative job definitions including which jobs should be run by the input trigger ... because that's one of the incompatibilities between our world view and the tekton worldview - where they manage a job as a k8s resource - but that means that the job, since it defines the trigger and reporter as part of its contents, MUST pre-exist and be managed outside of the patches | 18:32 |
| *** mattw4 has joined #zuul | 18:32 | |
| clarkb | (note it is possible that a batching dependent pipeline manager would be desireable in zuul but no one has asked for it yet) | 18:32 |
| fungi | and i definitely want to be sure to explain these concepts in ways to which users of other ci systems can relate | 18:33 |
| mordred | yeah - this is one of the few times when talking about zuul in the terms of the other systems is desirable and valuable | 18:33 |
| clarkb | mordred: in the tekton world view you'd have a job that made new jobs aiui | 18:33 |
| clarkb | mordred: so again its doable there, but it isn't a top level feature so users are left to figure it out themselges | 18:34 |
| mordred | clarkb: yes - possibly so | 18:34 |
| fungi | clarkb: the desire for circular dep resolution in gating sort of qualifies as asking fr batching | 18:34 |
| clarkb | fungi: ya I guess that is a particular variant of it | 18:34 |
| mordred | oh - the other thing that's different - we're VERY focused on being a system that is driven by and operates on git changes | 18:34 |
| fungi | right, that's one point i'm planning to put at the forefront | 18:35 |
| mordred | many of the other systems are systems that are built to handle arbitrary triggers and perform arbitrary automation and git triggers are simply one of the set | 18:35 |
| clarkb | mordred: fungi ya they seem to be of two extremes: either only github triggers or all the things (and often humans clicking buttons) | 18:35 |
| *** mattw4 has quit IRC | 18:37 | |
| *** mattw4 has joined #zuul | 18:38 | |
| *** hashar has joined #zuul | 18:46 | |
| *** erbarr has joined #zuul | 19:11 | |
| *** Defolos has joined #zuul | 19:12 | |
| *** saneax has quit IRC | 19:42 | |
| *** sgw has quit IRC | 20:23 | |
| openstackgerrit | Merged zuul/zuul master: Don't rely on report-build-page when building the buildset result url https://review.opendev.org/711406 | 20:28 |
| openstackgerrit | Merged zuul/zuul master: Use explicit provides/requires for container jobs https://review.opendev.org/710116 | 20:28 |
| *** sgw has joined #zuul | 20:39 | |
| *** hashar has quit IRC | 20:48 | |
| *** sshnaidm is now known as sshnaidm|afk | 20:48 | |
| *** jamesmcarthur has joined #zuul | 20:59 | |
| *** jamesmcarthur has quit IRC | 21:16 | |
| *** jamesmcarthur has joined #zuul | 21:18 | |
| *** dpawlik has quit IRC | 21:18 | |
| *** michael-beaver has joined #zuul | 21:42 | |
| corvus | Shrews: i haven't been able to get zk-shell to work with sasl | 21:54 |
| corvus | the only thing i can see to try is "add_auth sasl super:adminsecret" and that isn't working so well | 21:55 |
| Shrews | hmm. so it's *supposed* to work with sasl? | 21:55 |
| Shrews | i've never tried (or even thought about it, tbh) | 21:55 |
| corvus | i don't know; i get the feeling sasl with passwords is an unusual use case for zk | 21:56 |
| Shrews | maybe we should try the client that comes with zookeeper | 21:57 |
| corvus | Shrews: what's that? | 21:57 |
| Shrews | zkCli.sh i think | 21:57 |
| Shrews | i actually haven't used it, but i'm hoping it's similar | 21:58 |
| corvus | Shrews, clarkb: https://cwiki.apache.org/confluence/display/ZOOKEEPER/ZooKeeper+SSL+User+Guide that says "There is currently no support for SSL for the communication between ZooKeeper servers." | 21:58 |
| corvus | that seems like something that may be be concerning? | 21:59 |
| corvus | https://zookeeper.apache.org/doc/r3.5.5/zookeeperAdmin.html seems to contradict that with references to "quorum tls" | 22:01 |
| Shrews | i'm not sure what the security impact of that might be. Is our goal to just keep unwanted users from connecting, or keeping the data from being read? We don't store anything of importance, really | 22:01 |
| corvus | hopefully the docs are more accurate than the wiki :) | 22:01 |
| corvus | Shrews: we're about to store everything of importance in zk | 22:01 |
| mordred | corvus: why would ... like ... why would something choose to not have ssl support on intra-server communication? | 22:02 |
| mordred | I can't even | 22:02 |
| Shrews | oh, for the scheduler changes. well then... yeah. that's a potential problem | 22:03 |
| Shrews | etcd it is! | 22:03 |
| Shrews | lol | 22:03 |
| corvus | well, i just pointed out docs that say there is support for server-to-server ssl | 22:03 |
| mordred | corvus: looking at that doc - that also says "for quorum and leader election" | 22:03 |
| corvus | mordred: which one? | 22:04 |
| *** mattw4 has quit IRC | 22:04 | |
| *** mattw4 has joined #zuul | 22:04 | |
| mordred | the doc not the wiki | 22:04 |
| mordred | "Please note that Quorum TLS encapsulates securing both leader election and quorum communication protocols." | 22:04 |
| corvus | right. that's the sort of thing that makes me think that it is supported. | 22:04 |
| mordred | is that just a way of saying "server to server communication"? then | 22:06 |
| clarkb | corvus: https://github.com/apache/zookeeper/pull/826 | 22:06 |
| mordred | corvus: but yeah - it seems to be a fundamental feature of netty -so as long as you tell zk to use netty instead of nio it seems doable | 22:06 |
| corvus | mordred: i think so. even the wiki put that statement under the heading "quorum" | 22:06 |
| clarkb | corvus: I believe that pull request says "yes this is supported and we updated logging to make it easier for you to confirm it" | 22:06 |
| corvus | clarkb: yeah, that change added the section to the doc i'm reading now which is an ssl howto | 22:07 |
| corvus | "Quorum TLS" "New in 3.5.5" | 22:07 |
| corvus | sorry can't deeplink | 22:07 |
| mordred | so - yeah - I think those docs describe what we want | 22:09 |
| Shrews | seems like it | 22:09 |
| corvus | cool, i'll see about getting that going | 22:10 |
| Shrews | i love this line in that doc: "The disk is death to ZooKeeper." | 22:10 |
| mordred | corvus: and it looks like "secure" is on a different port, so it should be decently easy to disallow insecure connection | 22:10 |
| clarkb | https://github.com/apache/zookeeper/pull/184 added the actual feature | 22:11 |
| mordred | (for client/server) | 22:11 |
| clarkb | this PR also equates quorum with server - server | 22:11 |
| mordred | cool | 22:12 |
| mordred | clarkb, Shrews, corvus: while on this topic, ianw has some patches up to move towards ansible+containers for opendev's nodepool-builder - https://review.opendev.org/#/c/710908/ - and the first is setting up zk for testing for system-config ... should we put a pause on that and just wait until we know what this story is going to shake out to be? | 22:13 |
| mordred | or go ahead with that since non-ssl to ssl transition is going to be a transition anywa? | 22:13 |
| corvus | i believe our most recent thoughts on the subject were to store job secrets and project keys in zk, but still store them encrypted with a shared key, and make sure zuul components have that key installed out of band; so it shouldn't be super critical, but honestly, tls just seems like a good idea. | 22:14 |
| clarkb | mordred: maybe? we're likely to do a staged transition where we go from non ssl to ssl, then add auth, and I think we can probably start with non ssl in that testing | 22:14 |
| corvus | mordred: i don't feel that needs to be blocked on this | 22:14 |
| corvus | (also, the auth is using digest, so technically should be okay in the clear, but meh) | 22:15 |
| clarkb | and for opendev that might me, redeploy all zk with containers on bionic, add ssl, remove not ssl, add auth | 22:15 |
| clarkb | *might be | 22:15 |
| mordred | kk. just wanted to check | 22:16 |
| ianw | thanks; i wouldn't want to block too much on it either -- it's really only there so the container can start and we see it running in testinfra ... i don't think we want to do full image builds in system-config testing, just validate the daemon starts | 22:19 |
| *** mattw4 has quit IRC | 23:04 | |
| *** mattw4 has joined #zuul | 23:05 | |
| *** jamesmcarthur has quit IRC | 23:14 | |
| *** mattw4 has quit IRC | 23:18 | |
| *** mattw4 has joined #zuul | 23:19 | |
| *** jamesmcarthur has joined #zuul | 23:20 | |
| *** jamesmcarthur has quit IRC | 23:32 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!