*** javeriak has joined #openstack-ansible | 00:00 | |
*** Mudpuppy has joined #openstack-ansible | 00:05 | |
*** Mudpuppy has quit IRC | 00:11 | |
*** Mudpuppy has joined #openstack-ansible | 00:11 | |
*** stevemar has joined #openstack-ansible | 00:18 | |
*** javeriak has quit IRC | 00:40 | |
*** Mudpuppy is now known as Cloudpuppy | 00:55 | |
*** britthou_ has joined #openstack-ansible | 00:58 | |
*** britthouser has quit IRC | 01:01 | |
*** Cloudpuppy has quit IRC | 01:02 | |
*** dkalleg has quit IRC | 01:05 | |
*** abitha has quit IRC | 01:08 | |
*** sigmavirus24 is now known as sigmavirus24_awa | 01:20 | |
*** Mudpuppy has joined #openstack-ansible | 01:28 | |
*** stevemar has quit IRC | 03:06 | |
*** tlian has joined #openstack-ansible | 03:14 | |
*** sura8257 has quit IRC | 03:17 | |
*** mfisch has quit IRC | 03:32 | |
*** sdake has quit IRC | 03:32 | |
*** mfisch has joined #openstack-ansible | 03:36 | |
*** mfisch is now known as Guest68660 | 03:36 | |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Move swift_allow_all_users to role defaults https://review.openstack.org/190988 | 03:41 |
---|---|---|
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Set permissions on user_secrets.yml to 0600 https://review.openstack.org/191851 | 03:44 |
*** abitha has joined #openstack-ansible | 03:49 | |
*** fangfenghua has joined #openstack-ansible | 03:55 | |
*** abitha has quit IRC | 04:12 | |
*** radek_ has joined #openstack-ansible | 04:14 | |
*** radek__ has quit IRC | 04:15 | |
*** sdake has joined #openstack-ansible | 04:17 | |
*** meshok0 has joined #openstack-ansible | 04:21 | |
openstackgerrit | Merged stackforge/os-ansible-deployment: Configurable lxc cache and rootfs path https://review.openstack.org/190048 | 04:28 |
openstackgerrit | Merged stackforge/os-ansible-deployment: Fix spelling of runlevel in init scripts https://review.openstack.org/191495 | 04:49 |
*** meshok0 has quit IRC | 05:11 | |
openstackgerrit | Merged stackforge/os-ansible-deployment: Configure DB addresses for each service https://review.openstack.org/190271 | 05:23 |
*** javeriak has joined #openstack-ansible | 05:24 | |
*** tlian has quit IRC | 05:28 | |
*** stevemar has joined #openstack-ansible | 05:30 | |
*** meshok0 has joined #openstack-ansible | 06:09 | |
*** sdake has quit IRC | 06:14 | |
*** fawadkhaliq has joined #openstack-ansible | 06:17 | |
*** fangfenghua has quit IRC | 06:28 | |
openstackgerrit | Javeria Khan proposed stackforge/os-ansible-deployment: Adding PLUMgrid plugin option to neutron https://review.openstack.org/188964 | 06:54 |
*** fawadkhaliq has quit IRC | 07:05 | |
*** jmccrory has quit IRC | 07:08 | |
*** jmccrory has joined #openstack-ansible | 07:12 | |
*** JRobinson__ has quit IRC | 07:16 | |
*** meshok0 has quit IRC | 07:19 | |
*** fangfenghua has joined #openstack-ansible | 07:23 | |
*** meshok0 has joined #openstack-ansible | 07:57 | |
*** meshok0 has quit IRC | 08:09 | |
*** meshok0 has joined #openstack-ansible | 08:09 | |
*** javeriak has quit IRC | 08:26 | |
openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Split environment file into component parts https://review.openstack.org/192102 | 08:29 |
*** stevemar has quit IRC | 08:52 | |
*** fawadkhaliq has joined #openstack-ansible | 08:52 | |
openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Set default pass in aio scripts to be random https://review.openstack.org/190266 | 08:54 |
openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Move swift_allow_all_users to role defaults https://review.openstack.org/190988 | 08:54 |
openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Allow protocol to be set per endpoint-type https://review.openstack.org/190736 | 08:57 |
*** meshok0 has quit IRC | 08:58 | |
openstackgerrit | Merged stackforge/os-ansible-deployment: Set permissions on user_secrets.yml to 0600 https://review.openstack.org/191851 | 09:44 |
openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Added apt update tasks to everything using apt https://review.openstack.org/191528 | 10:15 |
openstackgerrit | Merged stackforge/os-ansible-deployment: Allow protocol to be set per endpoint-type https://review.openstack.org/190736 | 10:34 |
openstackgerrit | Merged stackforge/os-ansible-deployment: Set default pass in aio scripts to be random https://review.openstack.org/190266 | 10:39 |
openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Use default service_region var for all roles https://review.openstack.org/189664 | 10:45 |
openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Added apt update tasks to everything using apt https://review.openstack.org/191528 | 10:45 |
openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Added apt update tasks to everything using apt https://review.openstack.org/191528 | 10:48 |
openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Implement service_region user_group_var https://review.openstack.org/189664 | 10:51 |
openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Added apt update tasks to everything using apt https://review.openstack.org/191528 | 11:08 |
openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Allow protocol to be set per endpoint-type https://review.openstack.org/192148 | 11:15 |
openstackgerrit | Darren Birkett proposed stackforge/os-ansible-deployment: Set default pass in aio scripts to be random https://review.openstack.org/192162 | 11:51 |
openstackgerrit | Jesse Pretorius proposed stackforge/os-ansible-deployment: Configurable lxc cache and rootfs path https://review.openstack.org/192165 | 11:52 |
mancdaz | https://review.openstack.org/192162 cores this is a straight cherry-pick of a simple patch from master-kilo. Appreciate if you could do the needful | 11:53 |
*** fangfenghua has quit IRC | 11:58 | |
*** fawadkhaliq has quit IRC | 12:10 | |
*** tlian has joined #openstack-ansible | 12:54 | |
openstackgerrit | Merged stackforge/os-ansible-deployment: Move swift_allow_all_users to role defaults https://review.openstack.org/190988 | 12:56 |
*** sdake has joined #openstack-ansible | 13:32 | |
*** sdake has quit IRC | 13:32 | |
*** sdake has joined #openstack-ansible | 13:33 | |
*** Bjoern__ has joined #openstack-ansible | 13:41 | |
*** rrrobbb has joined #openstack-ansible | 13:59 | |
*** Mudpuppy has joined #openstack-ansible | 14:04 | |
*** Bjoern__ is now known as BjoernT | 14:08 | |
*** rromans has quit IRC | 14:11 | |
sigmavirus24_awa | Cool story: the problem I'm having with the fernet review is fixed by cryptography 1.0 (which is unreleased) thanks to a cffi 1.0 release | 14:12 |
palendae | Better go help them release | 14:12 |
sigmavirus24_awa | So here's the question, do we point to master of cryptography for a while and wait for them to release 1.0? | 14:12 |
sigmavirus24_awa | palendae: okay | 14:12 |
sigmavirus24_awa | will do | 14:12 |
Sam-I-Am | sigmavirus24_awa: become core? | 14:12 |
odyssey4me | sigmavirus24_awa what's the issue exactly? | 14:13 |
sigmavirus24_awa | odyssey4me: even with single-threaded apache, cffi reloads the bindings for openssl which causes some rather nasty exceptions that are pretty much https://github.com/pyca/cryptography/issues/1776 | 14:13 |
sigmavirus24_awa | We're using a version with the commit (that fixed that issue) but it still happens anyway | 14:14 |
*** rromans has joined #openstack-ansible | 14:14 | |
*** jwagner_away is now known as jwagner | 14:14 | |
sigmavirus24_awa | At least on my AIOs and it prevents me from completeing a full AIO build using fernet tokens | 14:14 |
odyssey4me | sigmavirus24_awa so what are our options? do we drop the apache front-end? have you tested any updates beyond to see if it is fixed by the next proposed release? | 14:14 |
odyssey4me | sigmavirus24_awa also, it would seem that the openstack-ci build worked? | 14:15 |
sigmavirus24_awa | odyssey4me: openstack CI build isn't using fernet | 14:15 |
*** jwagner is now known as jwagner_ceph_tra | 14:15 | |
odyssey4me | sigmavirus24_awa ah, of course - it wasn't something you enabled for the AIO so it would build successfully... fair enough | 14:16 |
* odyssey4me needs coffee or something | 14:16 | |
sigmavirus24_awa | odyssey4me: or something | 14:16 |
*** sigmavirus24_awa is now known as sigmavirus24 | 14:16 | |
palendae | odyssey4me: Come on, you should be wide awake by now :) | 14:17 |
*** Mudpuppy has quit IRC | 14:17 | |
odyssey4me | palendae the afternoon slump | 14:17 |
palendae | Ah, yes | 14:17 |
palendae | Go do your 10k steps, re-energize ;) | 14:17 |
*** Mudpuppy has joined #openstack-ansible | 14:18 | |
*** yaya has joined #openstack-ansible | 14:19 | |
odyssey4me | sigmavirus24 bah, it seems that was the last commit to the 0.8.x branch too | 14:23 |
openstackgerrit | Hugh Saunders proposed stackforge/os-ansible-deployment: Generate consistent length passwords in AIO https://review.openstack.org/192220 | 14:23 |
sigmavirus24 | odyssey4me: yep | 14:23 |
*** Guest68660 is now known as mfisch | 14:30 | |
*** mfisch is now known as Guest81226 | 14:31 | |
sigmavirus24 | so odyssey4me thoughts on this problem? | 14:32 |
sigmavirus24 | odyssey4me: fwiw, I tested out cryptography 1.0 from master in the keystone containers and it works perfectly | 14:32 |
sigmavirus24 | Didn't fail once. I'm even thinking of turning up the thread values to see if it continues to work perfectly | 14:32 |
odyssey4me | sigmavirus24 give that a bash, for interest's sake... | 14:33 |
odyssey4me | I'm curious about what's different between 0.8.2 and 1.0 then | 14:33 |
sigmavirus24 | Should I use Mjolnar? | 14:33 |
sigmavirus24 | odyssey4me: cffi pre-1.0 and cffi 1.0 | 14:33 |
sigmavirus24 | cffi 1.0 was a huge effort partially funded by the Python Software Foundation | 14:33 |
sigmavirus24 | It has a huge list of improvements | 14:33 |
sigmavirus24 | cffi 1.0 required some breaking changes though which are addressed in cryptography master | 14:34 |
* sigmavirus24 hangs out with the cryptography developers which is the only reason he knows this | 14:34 | |
sigmavirus24 | *"hangs out" | 14:34 |
sigmavirus24 | (a.k.a, lurks) | 14:34 |
odyssey4me | sigmavirus24 https://github.com/pyca/cryptography/issues/1868 | 14:35 |
odyssey4me | sigmavirus24 is there some sort of timeline for the 1.0 release? | 14:36 |
sigmavirus24 | I jumped off IRC last night and no one's around this morning. I think half the team is on vacation at the moment | 14:36 |
sigmavirus24 | I've also offered to help get cryptography 1.0 out the door however possible | 14:37 |
sigmavirus24 | Which of course means staying up past 1AM instead of staying up til 1AM to keep my other stuff moving too | 14:37 |
*** Guest81226 has quit IRC | 14:38 | |
odyssey4me | sigmavirus24 did you try 0.91 ? | 14:46 |
odyssey4me | *0.9.1 | 14:46 |
*** rrrobbb has quit IRC | 14:46 | |
sigmavirus24 | odyssey4me: that's what's installed by default | 14:46 |
sigmavirus24 | That's the version I was seeing the issue with initially | 14:46 |
sigmavirus24 | sorry | 14:46 |
odyssey4me | bah :/ | 14:47 |
sigmavirus24 | odyssey4me: I was working on this until 1900 local time last night | 14:47 |
sigmavirus24 | or there-abous | 14:47 |
sigmavirus24 | *abouts | 14:47 |
odyssey4me | maybe a solution then is to fix to a current sha in master? | 14:47 |
odyssey4me | once 1.0 releases we can revise | 14:47 |
*** stevemar has joined #openstack-ansible | 14:50 | |
*** galstrom_zzz is now known as galstrom | 14:50 | |
odyssey4me | sigmavirus24 yeah, I'd suggest doing that - just explain why you're doing it in the review | 14:51 |
sigmavirus24 | Yep | 14:51 |
sigmavirus24 | And fwiw, with the default thread count, it works still | 14:51 |
sigmavirus24 | This is definitely a cffi problem | 14:51 |
sigmavirus24 | What I've learned from tailing the logs, though, is that we need to make sure the key_repository directory is not world readable | 14:52 |
sigmavirus24 | keystone issues warnings when it sees that | 14:52 |
odyssey4me | cool - perhaps also make a note in the two issues indicating that it's resolved in the current master | 14:52 |
sigmavirus24 | Which two issues? | 14:53 |
odyssey4me | sigmavirus24 https://github.com/pyca/cryptography/issues/1776 and https://github.com/pyca/cryptography/issues/1868 | 14:55 |
sigmavirus24 | Oh on cryptography's issues, yeah sure | 14:56 |
sigmavirus24 | There appears to be two features left before cryptography wanted to release 1.0 | 14:56 |
*** mfisch has joined #openstack-ansible | 14:56 | |
sigmavirus24 | So I'm going to see how I can help accelerate those | 14:56 |
*** mfisch is now known as Guest25676 | 14:56 | |
odyssey4me | sigmavirus24 sweet, that'll be helpful I'm sure :) | 14:59 |
sigmavirus24 | Paul's been trying to trick me into working on Cryptography for a while now | 15:00 |
sigmavirus24 | So | 15:00 |
sigmavirus24 | He's welcome | 15:00 |
palendae | Just make a bug that affects you and boom | 15:01 |
palendae | Done | 15:01 |
*** Guest25676 is now known as mfisch | 15:01 | |
*** mfisch has quit IRC | 15:02 | |
*** mfisch has joined #openstack-ansible | 15:02 | |
sigmavirus24 | I'm a scourge on projects | 15:03 |
sigmavirus24 | You don't want me | 15:03 |
palendae | I know I don't | 15:03 |
palendae | I've seen you contributions | 15:03 |
sigmavirus24 | Urban Dictionary definition of "sigmavirus24": Brings death and pestilence to all he touches | 15:03 |
*** Mudpuppy has quit IRC | 15:03 | |
palendae | sigmavirus24: Please go work on io and node then | 15:04 |
*** Mudpuppy has joined #openstack-ansible | 15:04 | |
sigmavirus24 | I'm working on rack right now | 15:04 |
sigmavirus24 | palendae: https://github.com/rack/rack/issues/726 https://github.com/rack/rack/pull/892 | 15:05 |
palendae | Well that one's already dead | 15:05 |
palendae | Also not openstack, stop. | 15:05 |
palendae | Though you're helping tenderlove, so carry on | 15:05 |
sigmavirus24 | Also https://github.com/kennethreitz/requests/issues/2117 | 15:06 |
*** jaypipes has quit IRC | 15:06 | |
*** yaya has quit IRC | 15:07 | |
*** jwagner_ceph_tra is now known as jwagner_away | 15:24 | |
*** jwagner_away is now known as jwagner | 15:28 | |
openstackgerrit | Merged stackforge/os-ansible-deployment: Set default pass in aio scripts to be random https://review.openstack.org/192162 | 15:55 |
*** yaya has joined #openstack-ansible | 16:03 | |
Sam-I-Am | mmmmeeting? | 16:04 |
*** gtt116_ has joined #openstack-ansible | 16:05 | |
odyssey4me | o/ yeah, who else is around? | 16:07 |
palendae | <- | 16:08 |
palendae | b3rnard0: ? | 16:08 |
*** gtt116__ has quit IRC | 16:08 | |
odyssey4me | bug list: https://bugs.launchpad.net/openstack-ansible/+bugs?search=Search&field.status=New | 16:08 |
*** javeriak has joined #openstack-ansible | 16:09 | |
odyssey4me | https://bugs.launchpad.net/openstack-ansible/+bug/1463956 | 16:09 |
openstack | Launchpad bug 1463956 in openstack-ansible "slow_query_log should be off by default and relocated" [Undecided,New] | 16:09 |
b3rnard0 | did cloudnull assign a backup? | 16:09 |
Sam-I-Am | b3rnard0: looks like you volunteered | 16:09 |
odyssey4me | b3rnard0 he's just asked me to cover for him today and thu | 16:10 |
b3rnard0 | k | 16:10 |
Sam-I-Am | b3rnard0: denied | 16:10 |
odyssey4me | this info looks wrong for master/kilo - the slow log is located in /var/log/mysql_logs/ for master/kilo as I recall, so it would appear that this bug info is for juno/icehouse | 16:11 |
odyssey4me | BjoernT are you around? | 16:11 |
BjoernT | yes | 16:11 |
*** daneyon has joined #openstack-ansible | 16:11 | |
odyssey4me | ok, confirmed - /var/log/mysql/mysql-slow.log is for juno; /var/log/mysql_logs/mysql-slow.log is for kilo/master | 16:13 |
*** javeriak has quit IRC | 16:13 | |
odyssey4me | as I recall log rotation is enabled for master/kilo too | 16:13 |
*** daneyon_ has joined #openstack-ansible | 16:14 | |
odyssey4me | ah - no log rotation for anything | 16:16 |
BjoernT | yeah ii should not go into the container at all | 16:16 |
BjoernT | because our log rotation is still incomplete, or mostly missing | 16:16 |
*** daneyon has quit IRC | 16:17 | |
odyssey4me | BjoernT not sure I agree with that, the slow log is meant to be a useful tool for diagnosis - so I'd prefer to ensure that log rotation is set, but the slow log is left enabled... but that would be up to an implementer I guess | 16:17 |
BjoernT | well as I said no one is looking into it so why even logging in the first place unless you actually need it | 16:18 |
odyssey4me | we should allow that to be enabled/disabled though instead of hard setting it | 16:18 |
*** vdo has quit IRC | 16:18 | |
BjoernT | yeah thats fine too | 16:18 |
*** javeriak has joined #openstack-ansible | 16:20 | |
odyssey4me | any thoughts about importance? | 16:20 |
BjoernT | normal | 16:21 |
BjoernT | nothing high | 16:21 |
odyssey4me | alright, set to low | 16:22 |
odyssey4me | next: https://bugs.launchpad.net/openstack-ansible/+bug/1464372 | 16:23 |
openstack | Launchpad bug 1464372 in openstack-ansible "Environment variable setup does not work in container creation" [Undecided,New] | 16:23 |
odyssey4me | for now that looks pertinent to kilo/master - any thoughts on whether this matters for juno? | 16:25 |
*** daneyon_ has quit IRC | 16:25 | |
*** daneyon has joined #openstack-ansible | 16:25 | |
palendae | I would say leave juno out unless we have an urgent need | 16:26 |
odyssey4me | palendae agreed, any thoughts on importance here? | 16:27 |
palendae | Low, I'd say. Also sounds like we may have a commit incoming from the reporter | 16:27 |
odyssey4me | I suppose for the proxy stuff this is pretty important? | 16:27 |
palendae | Hrm, yeah...I haven't worked with them enough to know overall impact though | 16:28 |
palendae | Not sure how many deployments are behind them | 16:28 |
openstackgerrit | Ian Cordasco proposed stackforge/os-ansible-deployment: Add support for deploying Keystone with Fernet https://review.openstack.org/189998 | 16:28 |
odyssey4me | I'm surprised that it wasn't resolved as part of the proxy work: https://review.openstack.org/#/c/181603/ | 16:28 |
odyssey4me | maybe this issue is actually resolved for master/kilo and only is an issue for juno? | 16:29 |
sigmavirus24 | Did juno get that included? I don't recall if I backported that patch | 16:29 |
*** rrrobbb has joined #openstack-ansible | 16:30 | |
odyssey4me | sigmavirus24 only master/kilo here: https://review.openstack.org/#/q/I2a397abad6557a48bc737dcd9cca787fc5afa728,n,z | 16:31 |
sigmavirus24 | palendae: you're on bugs now right? | 16:31 |
palendae | I suppose | 16:31 |
sigmavirus24 | (not on the multi-region stuff) | 16:31 |
palendae | Yeah | 16:31 |
sigmavirus24 | Mind taking over that backport, you'll probably need https://gist.github.com/cloudnull/3015a4cff9573866ce55 | 16:31 |
palendae | Sure | 16:32 |
sigmavirus24 | juno/icehouse were going to be a pain compared to master/kilo | 16:32 |
palendae | Re-reading that bug | 16:33 |
palendae | It sounds like this might actually be juno | 16:33 |
odyssey4me | palendae if you can confirm that'd be great - shall I assign it to you? | 16:33 |
palendae | Sure | 16:33 |
palendae | hrm, no, that role doesn't exist in Juno. Still something that'll need to be done though | 16:34 |
odyssey4me | thanks palendae | 16:34 |
odyssey4me | next: https://bugs.launchpad.net/openstack-ansible/+bug/1465723 | 16:34 |
openstack | Launchpad bug 1465723 in openstack-ansible "Please add bash completion for the openstack clients" [Undecided,New] | 16:34 |
odyssey4me | this seems like something that would be useful for us to do | 16:35 |
sigmavirus24 | BjoernT: how do you propose we add bash completion for the clients? Do all of them have bash completion for their individual packages? | 16:35 |
odyssey4me | it would seem that https://github.com/openstack/python-novaclient/blob/master/tools/nova.bash_completion has something to do with it? | 16:35 |
BjoernT | yes | 16:36 |
odyssey4me | (for nova) | 16:36 |
* sigmavirus24 is pretty sure nova's the only one he's seen with bash completion | 16:36 | |
BjoernT | each project has the own config, at least nova and neutron does | 16:36 |
odyssey4me | keystone: https://github.com/openstack/python-keystoneclient/blob/master/tools/keystone.bash_completion | 16:36 |
odyssey4me | we'll have to figure out how that's implemented | 16:37 |
odyssey4me | I think this is a good candidate for master/kilo | 16:37 |
sigmavirus24 | Anyway, as long as it's installed properly (which may not be the case anymore, depending on how pbr implements this) we should be able to automate that | 16:37 |
BjoernT | yeah | 16:37 |
* sigmavirus24 agrees with odyssey4me | 16:37 | |
* sigmavirus24 has a concern with pbr + pip 7.1 doing things the right way | 16:38 | |
sigmavirus24 | but I'll have to experiment with that before I comment further =P | 16:38 |
odyssey4me | is that a volunteer I hear sigmavirus24 ? | 16:38 |
sigmavirus24 | not for the next few weeks =P | 16:38 |
odyssey4me | ok, I think this is a wishlist item personally - any thoughts? | 16:39 |
sigmavirus24 | My concern is only with how pbr tells setuptools to consider those files | 16:39 |
sigmavirus24 | odyssey4me: I agree | 16:39 |
sigmavirus24 | there's a bug with setuptools + wheel + data_files and if pbr is pointing at those files calling them data_files then they're already broken and there's going to be a lot of fun getting those bash completions in the right place for use | 16:39 |
sigmavirus24 | (or even determining where they're installed) | 16:40 |
palendae | Oops - we removed all the environment variable stuff: https://github.com/stackforge/os-ansible-deployment/commit/86b6e8e6409f76c46551becdac783a76bb853980#diff-9724ba1a4e7c37915c868ad54dd46552 | 16:41 |
sigmavirus24 | palendae: https://github.com/stackforge/os-ansible-deployment/commit/86b6e8e6409f76c46551becdac783a76bb853980#diff-9724ba1a4e7c37915c868ad54dd46552R204 | 16:42 |
sigmavirus24 | palendae: split out (it seems) to a newtask | 16:42 |
palendae | Ah, you're right | 16:43 |
odyssey4me | yep | 16:43 |
palendae | though that's a poor name... | 16:43 |
palendae | Our case was proxies | 16:43 |
* odyssey4me agrees | 16:43 | |
palendae | But it's doing all env vars | 16:43 |
palendae | w/e | 16:43 |
sigmavirus24 | palendae: patches welcome | 16:43 |
palendae | Thanks | 16:43 |
sigmavirus24 | You're welcome | 16:43 |
sigmavirus24 | palendae: EVERYONE IS WELCOME | 16:43 |
palendae | That was thanks for the actual help, not the snark | 16:44 |
* sigmavirus24 really should have slept in this morning | 16:44 | |
odyssey4me | ok, new bug triage done - any specific bugs that anyone wants to look through or get help with? | 16:44 |
* Apsu should have slept in this week | 16:44 | |
sigmavirus24 | I'd like to file a bug about my caffeine leves | 16:45 |
sigmavirus24 | *levels | 16:45 |
Apsu | Drove my caffeine to the levee but the levee was dry.. | 16:45 |
sigmavirus24 | A+ reference Apsu | 16:45 |
sigmavirus24 | mmmmm pie | 16:46 |
Apsu | 'merica | 16:46 |
odyssey4me | lol, alright guys - thanks for all the fish :) | 16:46 |
sigmavirus24 | fish is an okay shell | 16:46 |
* sigmavirus24 prefers zsh though | 16:47 | |
odyssey4me | go to sleep sigmavirus24 and Apsu | 16:47 |
sigmavirus24 | nou | 16:47 |
Apsu | kk | 16:47 |
Apsu | Fish is a hilarious shell | 16:47 |
Apsu | Though, I think sea creatures with shells are by definition not fish. | 16:47 |
*** javeriak has quit IRC | 16:48 | |
sigmavirus24 | they're rustaceans, no? | 16:49 |
sigmavirus24 | WE SHOULD WRITE A SHELL IN RUST! | 16:50 |
sigmavirus24 | anyone else seeing build failures due to a ppa problem with rsyslog? | 16:51 |
sigmavirus24 | (http://paste.openstack.org/show/5MaLRETdnp4GWCNyNAdu/) | 16:52 |
*** yaya has quit IRC | 16:58 | |
odyssey4me | sigmavirus24 that has been seen to happen from time to time | 16:59 |
sigmavirus24 | gate's seeing a different error | 16:59 |
sigmavirus24 | maybe it's something transient on my AIO | 16:59 |
odyssey4me | we should actually just log a bug for it and remove it - the ppa isn't actually needed for anything we're doing in master/kilo | 16:59 |
*** Mudpuppy has quit IRC | 17:00 | |
sigmavirus24 | So yaprt doesn't seem to like https://review.openstack.org/#/c/189998/14/playbooks/defaults/repo_packages/openstack_other.yml it seems | 17:00 |
sigmavirus24 | That | 17:00 |
sigmavirus24 | Or ansible auto-fails when something's printed to stderr | 17:01 |
sigmavirus24 | http://logs.openstack.org/98/189998/14/check/os-ansible-deployment-dsvm-check-commit/fd54977/console.html#_2015-06-16_16_52_27_962 | 17:01 |
odyssey4me | sigmavirus24 check the full ansible log: http://logs.openstack.org/98/189998/14/check/os-ansible-deployment-dsvm-check-commit/fd54977/logs/ansible-logging/ansible.log | 17:05 |
sigmavirus24 | I did | 17:05 |
sigmavirus24 | it doesn't give much more info | 17:05 |
sigmavirus24 | the error is at the very bottm | 17:05 |
*** yaya has joined #openstack-ansible | 17:05 | |
*** javeriak has joined #openstack-ansible | 17:05 | |
*** abitha has joined #openstack-ansible | 17:11 | |
*** rrrobbb has quit IRC | 17:12 | |
*** javeriak has quit IRC | 17:16 | |
*** javeriak has joined #openstack-ansible | 17:16 | |
*** rrrobbb has joined #openstack-ansible | 17:16 | |
*** gtt116_ has quit IRC | 17:16 | |
*** gtt116_ has joined #openstack-ansible | 17:17 | |
*** yaya has quit IRC | 17:19 | |
*** sdake has quit IRC | 17:19 | |
*** sdake_ has joined #openstack-ansible | 17:19 | |
odyssey4me | sigmavirus24 sorry, was talking with mancdaz - that should work, dammit | 17:24 |
sigmavirus24 | lol | 17:25 |
*** dkalleg has joined #openstack-ansible | 17:25 | |
sigmavirus24 | I scrapped my AIO for a new one | 17:25 |
sigmavirus24 | because it wasn't doin gthe build packages step and I'm running stuff on a fresh AIO to repro | 17:25 |
odyssey4me | sigmavirus24 I'm out - if it's still an issue tomorrow then I'll build an AIO to see if I can figure it out. | 17:29 |
sigmavirus24 | No problem | 17:29 |
sigmavirus24 | I'm working on it | 17:29 |
odyssey4me | palendae if you could peek at https://review.openstack.org/191528 it'd be great - this patch will help overcome some of the gating failures we've been seeing, specifically hpcloud apt failures. | 17:35 |
odyssey4me | cloudnull I did a revision of https://review.openstack.org/189664 which you may prefer. Please review. | 17:36 |
*** jaypipes has joined #openstack-ansible | 17:37 | |
*** yaya has joined #openstack-ansible | 17:56 | |
*** daneyon_ has joined #openstack-ansible | 17:57 | |
*** yaya has quit IRC | 17:59 | |
*** daneyon has quit IRC | 18:00 | |
palendae | odyssey4me: Reviewed, think I found a very small problem | 18:00 |
*** fawadkhaliq has joined #openstack-ansible | 18:03 | |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Implement service_region user_group_var https://review.openstack.org/189664 | 18:08 |
*** Mudpuppy has joined #openstack-ansible | 18:11 | |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Added apt update tasks to everything using apt https://review.openstack.org/191528 | 18:14 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Implement service_region user_group_var https://review.openstack.org/189664 | 18:14 |
*** BjoernT has left #openstack-ansible | 18:18 | |
*** Mudpuppy has quit IRC | 18:21 | |
*** fawadkhaliq has quit IRC | 18:22 | |
*** jwagner is now known as jwagner_away | 18:36 | |
*** daneyon has joined #openstack-ansible | 18:42 | |
*** daneyon_ has quit IRC | 18:45 | |
openstackgerrit | Miguel Grinberg proposed stackforge/os-ansible-deployment: Implement service_region user_group_var https://review.openstack.org/189664 | 18:52 |
openstackgerrit | Miguel Grinberg proposed stackforge/os-ansible-deployment: Support an externally hosted keystone https://review.openstack.org/192015 | 18:52 |
openstackgerrit | Miguel Grinberg proposed stackforge/os-ansible-deployment: Support an externally hosted keystone https://review.openstack.org/192015 | 19:06 |
*** dkalleg has quit IRC | 19:06 | |
*** sdake_ is now known as sdake | 19:13 | |
*** sdake has quit IRC | 19:15 | |
*** sdake has joined #openstack-ansible | 19:15 | |
openstackgerrit | Miguel Grinberg proposed stackforge/os-ansible-deployment: Support an externally hosted keystone https://review.openstack.org/192015 | 19:16 |
*** yaya has joined #openstack-ansible | 19:20 | |
*** gtt116 has joined #openstack-ansible | 19:29 | |
* sigmavirus24 is very uncertain why yaprt is barfing on cryptography | 19:32 | |
*** gtt116_ has quit IRC | 19:32 | |
Sam-I-Am | sigmavirus24: cryptic errors? | 19:32 |
cloudnull | sigmavirus24: maybe i can help :) | 19:33 |
sigmavirus24 | cloudnull: out of training already? | 19:33 |
cloudnull | we were sent home , because of storm . | 19:33 |
Sam-I-Am | lol storm | 19:34 |
Sam-I-Am | this aint no storm | 19:34 |
sigmavirus24 | lol | 19:34 |
sigmavirus24 | cloudnull: so http://logs.openstack.org/98/189998/14/check/os-ansible-deployment-dsvm-check-commit/fd54977/console.html#_2015-06-16_16_52_27_962 is from https://review.openstack.org/#/c/189998/14/playbooks/defaults/repo_packages/openstack_other.yml | 19:34 |
cloudnull | idk, trainer bounced thus here i am | 19:34 |
sigmavirus24 | which is necessary for the fernet work | 19:34 |
sigmavirus24 | so my report generated on my AIO is showing no requirements for cryptography | 19:36 |
sigmavirus24 | which I'm pretty sure it has | 19:36 |
palendae | Ceph can't stand up to storms | 19:36 |
stevelle | so don't put your datacenter in Oklahoma? | 19:38 |
cloudnull | sigmavirus24: looking now. | 19:39 |
sigmavirus24 | cloudnull: I can give you access to this AIO boxen too if you want | 19:39 |
*** rrrobbb has quit IRC | 19:41 | |
openstackgerrit | Merged stackforge/os-ansible-deployment: Adding PLUMgrid plugin option to neutron https://review.openstack.org/188964 | 19:55 |
vincent_vdk | I suddenly understand Dutch... | 19:55 |
svg | vincent_vdk: ? | 19:56 |
sigmavirus24 | vincent_vdk: do you know kung fu too? | 19:56 |
vincent_vdk | svg: just went through the history :) | 19:56 |
vincent_vdk | sigmavirus24: not yet :) | 19:57 |
*** jwagner_away is now known as jwagner | 19:57 | |
sigmavirus24 | cloudnull: ideas? | 19:58 |
*** dkalleg has joined #openstack-ansible | 19:58 | |
cloudnull | i believe its the "-e ., -e vectors" because the dep processor is looking at the items in the requirement files as literal requirements. | 19:59 |
cloudnull | im testing something with that now. | 19:59 |
sigmavirus24 | ah | 19:59 |
sigmavirus24 | that explains that error message | 20:00 |
openstackgerrit | Ken Johnston proposed stackforge/os-ansible-deployment-specs: Spec for multiregion ansible deployment https://review.openstack.org/192421 | 20:02 |
cloudnull | yup thats the issue. | 20:06 |
* cloudnull going to go fix | 20:06 | |
sigmavirus24 | cloudnull: can probably just s/-e //g | 20:07 |
*** rrrobbb has joined #openstack-ansible | 20:08 | |
*** rrrobbb has quit IRC | 20:08 | |
cloudnull | i thought about that. however in the case of cryptography they have a dev requirement of vector which is a local sub package. | 20:08 |
*** rrrobbb has joined #openstack-ansible | 20:08 | |
sigmavirus24 | cloudnull: the other thing is that you're picking up the dev-requirements.txt and there is a slightly different set of requirements in https://review.openstack.org/#/dashboard/?foreach=%28project%3Aopenstack%2Fglance+OR+project%3Aopenstack%2Fpython%252Dglanceclient+OR+project%3Aopenstack%2Fglance_store+OR+project%3Aopenstack%2Fglance%252Dspecs%29+status%3Aopen+NOT+label%3ACode%252DReview%3C%3D%252D2&title=Glance+Review+Dashboar | 20:09 |
sigmavirus24 | d&With+Negative+Feedback=NOT+label%3AWorkflow%3C%3D%252D1+NOT+label%3AVerified%3C%3D%252D1%252cjenkins+NOT+owner%3Aself+label%3ACode%252DReview%252D1+limit%3A20+branch%3Amaster&Needs+Reverify=label%3AVerified%3C%3D%252D1%252cjenkins+branch%3Amaster+NOT+label%3ACode%252DReview%3C%3D%252D1+NOT+label%3AWorkflow%3C%3D%252D1&Work+In+Progress=NOT+label%3AWorkflow%3E%3D1+NOT+owner%3Aself+label%3AWorkflow%3C%3D%252D1+limit%3A20+branch% | 20:09 |
sigmavirus24 | 3Amaster&Needs+Approval=NOT+label%3AWorkflow%3E%3D1+NOT+label%3AWorkflow%3C%3D%252D1+label%3AVerified%3E%3D1%252cjenkins+NOT+owner%3Aself+label%3ACode%252DReview%3E%3D2+NOT+label%3ACode%252DReview%252D1+branch%3Amaster&Without+Negative+Feedback=NOT+label%3AWorkflow%3E%3D1+NOT+label%3AWorkflow%3C%3D%252D1+label%3AVerified%3E%3D1%252cjenkins+NOT+owner%3Aself+NOT+label%3ACode%252DReview%3C%3D%252D1+NOT+label%3ACode%252DReview%3E%3 | 20:09 |
sigmavirus24 | D2+limit%3A50+branch%3Amaster&5+Days+Without+Feedback=NOT+label%3AWorkflow%3E%3D1+NOT+label%3AWorkflow%3C%3D%252D1+label%3AVerified%3E%3D1%252cjenkins+NOT+owner%3Aself+NOT+label%3ACode%252DReview%3C%3D2+age%3A5d+branch%3Amaster&Own+Patches=owner%3Aself+status%3Aopen | 20:09 |
sigmavirus24 | ugh not that | 20:09 |
sigmavirus24 | https://github.com/pyca/cryptography/blob/master/setup.py#L34 | 20:10 |
cloudnull | its the processing of all the requirements which includes dev | 20:13 |
*** sdake_ has joined #openstack-ansible | 20:16 | |
*** sdake has quit IRC | 20:20 | |
openstackgerrit | Merged stackforge/os-ansible-deployment: Added apt update tasks to everything using apt https://review.openstack.org/191528 | 20:28 |
*** radek_ has quit IRC | 20:30 | |
*** yaya has left #openstack-ansible | 20:51 | |
sigmavirus24 | https://review.openstack.org/#/c/153057/ was finally approved | 20:52 |
sigmavirus24 | cloudnull: I see you pushed some stuff to github | 21:02 |
sigmavirus24 | should I test that out? | 21:02 |
cloudnull | https://github.com/cloudnull/yaprt/commit/c0a91e38842d71aa22a5b83c5f53056d5acfbb2d | 21:02 |
cloudnull | that makes it all happy | 21:02 |
sigmavirus24 | is that up on github already? | 21:03 |
sigmavirus24 | *pypi | 21:03 |
cloudnull | no | 21:03 |
sigmavirus24 | (or does it not need to be?) | 21:03 |
cloudnull | it would need to be uploaded for osad to use it | 21:03 |
cloudnull | im building now. | 21:03 |
cloudnull | to see if it goes. and then ill push | 21:04 |
cloudnull | if you wouldnt mind testing too it would be much appreciated. | 21:04 |
cloudnull | but it looks like its going | 21:04 |
sigmavirus24 | cool | 21:05 |
* sigmavirus24 goes to test | 21:05 | |
cloudnull | http://104.130.7.7:8181/reports/release-master-report.json http://104.130.7.7:8181/pools/cryptography-vectors/ http://104.130.7.7:8181/pools/cryptography/ | 21:06 |
* sigmavirus24 is running | 21:10 | |
sigmavirus24 | cloudnull: looks good from that | 21:10 |
*** tlian2 has joined #openstack-ansible | 21:11 | |
*** Mudpuppy has joined #openstack-ansible | 21:14 | |
*** tlian has quit IRC | 21:14 | |
*** yaya has joined #openstack-ansible | 21:17 | |
cloudnull | sigmavirus24: https://jenkins05.openstack.org/job/os-ansible-deployment-dsvm-check-commit/489/console is running now | 21:18 |
* cloudnull fingers crossed :) | 21:19 | |
sigmavirus24 | yeah mine's building everything and hasn't croaked yet | 21:19 |
*** persia has quit IRC | 21:23 | |
*** persia_ has joined #openstack-ansible | 21:24 | |
*** persia_ has quit IRC | 21:24 | |
*** javeriak has quit IRC | 21:24 | |
*** persia has joined #openstack-ansible | 21:25 | |
*** Mudpuppy has quit IRC | 21:27 | |
*** Mudpuppy has joined #openstack-ansible | 21:28 | |
*** javeriak_ has joined #openstack-ansible | 21:28 | |
*** javeriak_ has quit IRC | 21:39 | |
*** javeriak has joined #openstack-ansible | 21:41 | |
*** yaya has quit IRC | 21:47 | |
*** fawadkhaliq has joined #openstack-ansible | 21:48 | |
cloudnull | sigmavirus24: is everything right with the world ? | 21:50 |
cloudnull | gate wise its past keystone so LGTM https://jenkins05.openstack.org/job/os-ansible-deployment-dsvm-check-commit/489/console | 21:51 |
*** fawadkhaliq has quit IRC | 21:53 | |
sigmavirus24 | cloudnull: so it fails on galera_containers | 21:57 |
sigmavirus24 | when it tries to install python-memcached it can't find a version of six that satisfies memcached's requirements (which has no freaking bounds) | 21:58 |
cloudnull | it seems to be working in gate commit | 21:58 |
sigmavirus24 | hm | 21:59 |
sigmavirus24 | this seems to be a name normalization problem | 21:59 |
sigmavirus24 | maybe time to scrap this aio | 22:01 |
*** JRobinson__ has joined #openstack-ansible | 22:01 | |
*** sdake_ is now known as sdake | 22:02 | |
* sigmavirus24 shrugs | 22:02 | |
cloudnull | me too . | 22:02 |
cloudnull | i just work here... | 22:02 |
cloudnull | :) | 22:02 |
sigmavirus24 | well | 22:07 |
sigmavirus24 | time to switch over to working on cryptography so they can ship 1.0 | 22:07 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: [WIP] Backported tempest roles from Master https://review.openstack.org/192445 | 22:10 |
cloudnull | +1 for upstream work | 22:11 |
*** persia has quit IRC | 22:23 | |
*** Mudpuppy has quit IRC | 22:24 | |
*** 21WACGOA9 has joined #openstack-ansible | 22:25 | |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Allow Horizon to access multiple regions https://review.openstack.org/190302 | 22:25 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Generate consistent length passwords in AIO https://review.openstack.org/192220 | 22:25 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Implement service_region user_group_var https://review.openstack.org/189664 | 22:26 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Add read/write_affinity settings for Swift https://review.openstack.org/191023 | 22:27 |
javeriak | hello ansible people | 22:27 |
javeriak | was wondering if the this BP needs to get approved, as the code and spec are both merged now.. https://blueprints.launchpad.net/openstack-ansible/+spec/add-plumgrid-neutron-plugin | 22:28 |
*** 21WACGOA9 is now known as persia | 22:30 | |
*** persia has quit IRC | 22:31 | |
*** persia has joined #openstack-ansible | 22:31 | |
*** galstrom is now known as galstrom_zzz | 22:34 | |
*** openstackgerrit has quit IRC | 22:38 | |
*** openstackgerrit has joined #openstack-ansible | 22:39 | |
cloudnull | javeriak: done. | 22:40 |
cloudnull | looks like a clerical miss, from the spec approval. | 22:40 |
javeriak | cloudnull, thanks, also question, if wanted to add a new host afterwards during a play, how would i go about that | 22:40 |
cloudnull | new host? | 22:41 |
javeriak | saw the add_host module, it can add hosts to a new group | 22:41 |
cloudnull | is it a host that will always exist? if so you should add it in the *_user_config.yml file | 22:41 |
cloudnull | if its something that you will add in dynamically then the add_host module should be used. | 22:42 |
javeriak | say ive got another type of host_group that doesnt lie in rpc_user_config | 22:42 |
cloudnull | i'd say add_host is what you'd want to go with | 22:42 |
javeriak | it will always exist, so i should then add anew host_group to *_user_config.yml with the new hostnames/ips, the dynamic inventroy will pick them up right | 22:43 |
cloudnull | yes it will | 22:43 |
javeriak | oh okay, thanks | 22:43 |
cloudnull | if its something that you want osad to build containers for you can create an environment entry for the same thing | 22:44 |
cloudnull | but if a host group is all you need and its percistant then adding to user config is the simplest route. | 22:45 |
*** tlian2 has quit IRC | 22:45 | |
javeriak | no containers, but i want to my external playbooks to be able to run on that host with host: new_host | 22:45 |
cloudnull | sn entry in the user_config should do it | 22:45 |
*** Mudpuppy has joined #openstack-ansible | 22:45 | |
javeriak | yes user config sounds right | 22:45 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: [WIP] Backported tempest roles from Master https://review.openstack.org/192445 | 22:53 |
*** rrrobbb has quit IRC | 23:21 | |
stevelle | Just to be sure I understand the thinking, why do we not install our ansible plugins globally? | 23:35 |
cloudnull | stevelle: the libraries ? | 23:35 |
stevelle | yes | 23:35 |
cloudnull | https://github.com/stackforge/os-ansible-deployment/tree/master/playbooks/plugins/lookups and https://github.com/stackforge/os-ansible-deployment/tree/master/playbooks/library ? | 23:36 |
stevelle | particularly playbooks/plugins | 23:36 |
cloudnull | i guess you could. but im not sure why you would want to as it would dirty the global ansible namespace. | 23:37 |
cloudnull | it would be better to pr the plugin upstream if it needed to be global | 23:38 |
cloudnull | but in our case that lookup plugin is so specific to us that i dont think anisble would accept it | 23:38 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: [WIP] Backported tempest roles from Master https://review.openstack.org/192445 | 23:39 |
stevelle | I wrote a plugin filter last night that I need for haproxy. no this isn't a great use case anyway | 23:40 |
stevelle | trying to decide what to do with it | 23:40 |
cloudnull | is it general enough to move into osad ? | 23:41 |
*** sigmavirus24 is now known as sigmavirus24_awa | 23:42 | |
stevelle | allows you to filter a host group based on an individual var (from hostvars) matching a fixed value | 23:42 |
cloudnull | i mean does it improve haproxy and how we do haproxy deployments? it sounds like it might? if so i'd say spec it and pr it into master. | 23:43 |
stevelle | could be tweaked to do regex instead of isequal or w/e | 23:43 |
*** stevemar has quit IRC | 23:44 | |
*** stevemar has joined #openstack-ansible | 23:44 | |
stevelle | for a cross-regional identity cloud I have a template to config an haproxy-per-region and use this filter to only get the members of groups['keystone_all'] which belong to a keystone_service_region -- which I'm setting with container_vars. Is it generally useful? only for a multi-node dev env. | 23:45 |
stevelle | (and even then one that build a multi-regional keystone) | 23:45 |
stevelle | unless we think of another reason to filter hosts by vars | 23:45 |
cloudnull | hum... | 23:47 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: [WIP] Backported tempest roles from Master https://review.openstack.org/192445 | 23:48 |
stevelle | but here, don't use this in production ;) | 23:48 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Updated Juno for new dev work https://review.openstack.org/188975 | 23:50 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Backported tempest roles from Master https://review.openstack.org/192445 | 23:51 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Updated Juno for new dev work https://review.openstack.org/188975 | 23:51 |
cloudnull | stevelle: been pondering, but idk . . . i mean it couldn't be used with anything but haproxy right ? | 23:52 |
stevelle | well, it could be used anywhere you want a subset of groups['something'] because that is what it gives you | 23:52 |
stevelle | specifically a subset based on variation in vars | 23:53 |
stevelle | maybe it makes container_vars more useful when the values vary by host but otherwise you shouldn't have much variation there | 23:54 |
stevelle | and hostvars don't include facts do they? | 23:54 |
stevelle | doesn't seem to at least | 23:54 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Backported tempest roles from Master https://review.openstack.org/192445 | 23:55 |
openstackgerrit | Kevin Carter proposed stackforge/os-ansible-deployment: Updated Juno for new dev work https://review.openstack.org/188975 | 23:55 |
cloudnull | hostvars are set from inventory. | 23:56 |
cloudnull | facts are generated. | 23:56 |
cloudnull | but you access facts using the hostvars entry when accessing them from another host. | 23:57 |
cloudnull | but id say through it up as a wip and let some folks review it | 23:58 |
stevelle | ok, accessing facts using the hostvars entry might be a useful tool | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!