*** suro-patz has quit IRC | 00:09 | |
*** neilus has joined #kolla | 00:15 | |
*** neilus has quit IRC | 00:20 | |
*** suro-patz has joined #kolla | 01:05 | |
*** neilus has joined #kolla | 01:08 | |
openstackgerrit | Merged openstack/kolla: Adjust CMD for data and kolla-ansible https://review.openstack.org/261578 | 01:13 |
---|---|---|
*** cemason has joined #kolla | 01:14 | |
*** Jeffrey4l has joined #kolla | 01:17 | |
*** cemason has quit IRC | 01:25 | |
*** cemason has joined #kolla | 01:26 | |
*** cemason has quit IRC | 01:26 | |
*** jasonsb has quit IRC | 01:29 | |
*** inc0 has quit IRC | 01:35 | |
*** Jeffrey4l has quit IRC | 01:42 | |
*** rhallisey has quit IRC | 01:49 | |
*** Jeffrey4l has joined #kolla | 01:55 | |
openstackgerrit | Merged openstack/kolla: Fix file permissions https://review.openstack.org/261765 | 02:01 |
*** neilus has joined #kolla | 02:02 | |
*** neilus has quit IRC | 02:07 | |
*** Jeffrey4l has quit IRC | 02:12 | |
*** tzn has joined #kolla | 02:14 | |
*** Jeffrey4l has joined #kolla | 02:14 | |
*** tzn has quit IRC | 02:20 | |
openstackgerrit | Kuo-tung Kao proposed openstack/kolla: change libvirt sock group to nova https://review.openstack.org/260352 | 02:20 |
openstackgerrit | Merged openstack/kolla: Add missing contraints https://review.openstack.org/261781 | 02:37 |
*** suro-patz has quit IRC | 02:38 | |
*** unicell has quit IRC | 02:42 | |
openstackgerrit | Merged openstack/kolla: Fix pip install settings https://review.openstack.org/261784 | 02:44 |
openstackgerrit | Merged openstack/kolla: Fix version in master https://review.openstack.org/261847 | 02:49 |
*** tfukushima has joined #kolla | 02:53 | |
*** neilus has joined #kolla | 02:58 | |
*** neilus has quit IRC | 03:03 | |
*** Jeffrey4l has quit IRC | 03:10 | |
*** mark-casey1 has left #kolla | 03:24 | |
*** jasonsb has joined #kolla | 03:25 | |
*** suro-patz has joined #kolla | 03:26 | |
*** Jeffrey4l has joined #kolla | 03:28 | |
*** cemason has joined #kolla | 03:38 | |
*** Jeffrey4l has quit IRC | 03:38 | |
*** cemason has quit IRC | 03:42 | |
*** Jeffrey4l has joined #kolla | 03:51 | |
openstackgerrit | Merged openstack/kolla: Create ironic-pxe container with ironic-base https://review.openstack.org/262052 | 03:55 |
*** dave-mccowan has quit IRC | 03:57 | |
*** tfukushima has quit IRC | 03:58 | |
*** dims has quit IRC | 04:25 | |
*** dims has joined #kolla | 04:34 | |
*** suro-patz has quit IRC | 04:41 | |
*** dims has quit IRC | 04:44 | |
*** neilus has joined #kolla | 04:44 | |
*** neilus has quit IRC | 04:50 | |
*** dims has joined #kolla | 04:52 | |
*** unicell has joined #kolla | 04:58 | |
*** unicell has quit IRC | 05:04 | |
*** coolsvap|away is now known as coolsvap | 05:06 | |
*** unicell has joined #kolla | 05:06 | |
*** dims has quit IRC | 05:13 | |
*** sdake has quit IRC | 05:14 | |
*** dims has joined #kolla | 05:23 | |
*** coolsvap is now known as coolsvap|away | 05:25 | |
*** neilus has joined #kolla | 05:39 | |
*** neilus has quit IRC | 05:44 | |
*** tzn has joined #kolla | 05:50 | |
*** tzn has quit IRC | 05:57 | |
*** jasonsb has quit IRC | 06:18 | |
*** jasonsb has joined #kolla | 06:19 | |
*** coolsvap|away is now known as coolsvap | 06:19 | |
*** coolsvap is now known as coolsvap|away | 06:34 | |
*** cemason has joined #kolla | 06:38 | |
openstackgerrit | Wanlong Gao proposed openstack/kolla: Python 3 deprecated the logger.warn method in favor of warning https://review.openstack.org/262136 | 06:40 |
*** cemason has quit IRC | 06:43 | |
openstackgerrit | Artur Zarzycki proposed openstack/kolla: Base and openstack-base ubuntu binary containers. https://review.openstack.org/261957 | 06:58 |
*** salv-orl_ has joined #kolla | 07:23 | |
*** kjelly has quit IRC | 07:23 | |
openstackgerrit | Artur Zarzycki proposed openstack/kolla: Base and openstack-base ubuntu binary containers. https://review.openstack.org/261957 | 07:24 |
*** neilus has joined #kolla | 07:27 | |
openstackgerrit | Jeffrey Zhang proposed openstack/kolla: Make the logging more safe https://review.openstack.org/262146 | 07:28 |
*** neilus has quit IRC | 07:32 | |
openstackgerrit | Artur Zarzycki proposed openstack/kolla: Base and openstack-base ubuntu binary containers. https://review.openstack.org/261957 | 07:35 |
*** diga has joined #kolla | 07:58 | |
openstackgerrit | Michal Rostecki proposed openstack/kolla: Spec: Deploy Kolla images using Kubernetes https://review.openstack.org/255450 | 08:00 |
openstackgerrit | Michal Rostecki proposed openstack/kolla: Spec: Deploy Kolla images using Kubernetes https://review.openstack.org/255450 | 08:04 |
*** kjelly has joined #kolla | 08:05 | |
*** salv-orl_ has quit IRC | 08:17 | |
*** neilus has joined #kolla | 08:22 | |
*** salv-orlando has joined #kolla | 08:23 | |
openstackgerrit | Hui Kang proposed openstack/kolla: Enable nova fake driver in nova containers https://review.openstack.org/257442 | 08:24 |
*** Jeffrey4l has quit IRC | 08:25 | |
*** neilus has quit IRC | 08:26 | |
openstackgerrit | Sam Yaple proposed openstack/kolla: Convert magnum to kolla_docker https://review.openstack.org/261594 | 08:41 |
openstackgerrit | Sam Yaple proposed openstack/kolla: Convert Murano to kolla_docker https://review.openstack.org/261595 | 08:41 |
openstackgerrit | Sam Yaple proposed openstack/kolla: Convert heat to kolla_docker https://review.openstack.org/261592 | 08:41 |
openstackgerrit | Sam Yaple proposed openstack/kolla: Convert horizon to kolla_docker https://review.openstack.org/261593 | 08:41 |
openstackgerrit | Sam Yaple proposed openstack/kolla: Convert Swift to kolla_docker https://review.openstack.org/261598 | 08:41 |
openstackgerrit | Sam Yaple proposed openstack/kolla: Convert Ironic to kolla_docker https://review.openstack.org/261599 | 08:41 |
openstackgerrit | Sam Yaple proposed openstack/kolla: Convert neutron to kolla_docker https://review.openstack.org/261596 | 08:41 |
openstackgerrit | Sam Yaple proposed openstack/kolla: Convert Nova to kolla_docker https://review.openstack.org/261597 | 08:41 |
openstackgerrit | Sam Yaple proposed openstack/kolla: Convert rabbitmq to kolla_docker https://review.openstack.org/261584 | 08:41 |
openstackgerrit | Sam Yaple proposed openstack/kolla: Convert cinder to kolla_docker https://review.openstack.org/261591 | 08:41 |
openstackgerrit | Sam Yaple proposed openstack/kolla: Convert common role to kolla_docker https://review.openstack.org/261579 | 08:41 |
openstackgerrit | Sam Yaple proposed openstack/kolla: Convert mariadb to kolla_docker https://review.openstack.org/261583 | 08:41 |
openstackgerrit | Sam Yaple proposed openstack/kolla: Convert memcache to kolla_docker https://review.openstack.org/261582 | 08:41 |
openstackgerrit | Sam Yaple proposed openstack/kolla: Convert haproxy to kolla_docker https://review.openstack.org/261581 | 08:41 |
openstackgerrit | Sam Yaple proposed openstack/kolla: Convert Keystone to kolla_docker https://review.openstack.org/261580 | 08:41 |
openstackgerrit | Sam Yaple proposed openstack/kolla: Unblock Docker pinning in gate https://review.openstack.org/261601 | 08:41 |
openstackgerrit | Sam Yaple proposed openstack/kolla: Convert Ceph to kolla_docker https://review.openstack.org/261600 | 08:41 |
openstackgerrit | Sam Yaple proposed openstack/kolla: Add docker module in Kolla https://review.openstack.org/248812 | 08:41 |
openstackgerrit | Sam Yaple proposed openstack/kolla: Convert to named volumes instead of data container https://review.openstack.org/261843 | 08:51 |
*** Jeffrey4l has joined #kolla | 08:56 | |
*** salv-orl_ has joined #kolla | 09:01 | |
*** salv-orlando has quit IRC | 09:04 | |
*** cemason has joined #kolla | 09:05 | |
*** cemason has quit IRC | 09:09 | |
*** neilus has joined #kolla | 09:16 | |
*** neilus has quit IRC | 09:21 | |
*** tzn has joined #kolla | 09:26 | |
openstackgerrit | Hui Kang proposed openstack/kolla: Enable nova fake driver in nova containers https://review.openstack.org/257442 | 09:30 |
*** tzn has quit IRC | 09:32 | |
*** pmisiak has joined #kolla | 09:42 | |
pmisiak | hi guys | 09:42 |
pmisiak | I'm trying to deploy OpenStack using Kolla and I encoutered another issue | 09:44 |
pmisiak | it's related to kolla-ansible container | 09:44 |
pmisiak | when kolla run ansible inside the container there is error with lock because it can't write to /dev/shm directory | 09:45 |
nihilifer | hi pmisiak | 09:46 |
nihilifer | do you have some trace from that? | 09:46 |
nihilifer | i ask, because kolla-ansible runs in many plaves | 09:47 |
nihilifer | and we must be sure on which ster it failed for you | 09:47 |
pmisiak | http://pastebin.com/6Y1yRM3z | 09:47 |
nihilifer | thx | 09:47 |
pmisiak | on one of the controll nodes | 09:48 |
pmisiak | there are correct permissions to the /dev/shm directory | 09:50 |
pmisiak | but I can't write to it when i enter to the container | 09:50 |
nihilifer | what distro you're running on this controller node? | 09:51 |
*** tzn has joined #kolla | 09:53 | |
nihilifer | and maybe please check lxc and docker version | 09:57 |
nihilifer | but the strange fact, is that all issues with /dev/shm in docker were theoretically fixed in some 0.x release | 09:57 |
pmisiak | i'm running centos | 10:03 |
pmisiak | it's the one container running from ansible user | 10:05 |
pmisiak | other containers like rsyslogd doesn't have this issue | 10:05 |
pmisiak | i can write to the /dev/shm without any problems | 10:05 |
nihilifer | you checked the rsyslogd container by doing "docker exec" on the running one? | 10:07 |
pmisiak | yes | 10:08 |
nihilifer | well, rsyslogd runs in a privileged mode | 10:08 |
pmisiak | [root@control01 ~]# docker -v | 10:09 |
pmisiak | Docker version 1.8.2-el7.centos, build a01dc02/1.8.2 | 10:09 |
nihilifer | we probably have an answer https://github.com/docker/docker/issues/6758 | 10:10 |
*** neilus has joined #kolla | 10:10 | |
nihilifer | it seems that to access /dev/shm in docker | 10:10 |
nihilifer | there are two ways | 10:10 |
nihilifer | 1) privileged | 10:10 |
nihilifer | 2) --device option | 10:11 |
nihilifer | but the question now is why didn't anyone experienced this error :/ or why kolla-ansible for most of people probably doesn't touch semaphores | 10:12 |
nihilifer | well, a quick fix you may try to do | 10:12 |
nihilifer | is adding | 10:12 |
nihilifer | privileged: True | 10:12 |
nihilifer | to all kolla-ansible containers | 10:12 |
nihilifer | in playbooks | 10:12 |
pmisiak | for example memcached container: | 10:13 |
nihilifer | but probably we as kolla devs should consider checking --device option | 10:13 |
pmisiak | bash-4.2$ cd /dev/shm/ | 10:13 |
pmisiak | bash-4.2$ ls | 10:13 |
pmisiak | bash-4.2$ touch testfile | 10:13 |
pmisiak | bash-4.2$ ls | 10:13 |
pmisiak | testfile | 10:13 |
pmisiak | and it is not in privileged mode | 10:13 |
*** neilus has quit IRC | 10:15 | |
pmisiak | heh, I can't even see ansible version: http://pastebin.com/mH38u38X | 10:20 |
pmisiak | ansible version is 2.0.0 | 10:24 |
nihilifer | hmmm... | 10:26 |
nihilifer | maybe check whether base image has no problem with /dev/shm | 10:26 |
nihilifer | so smth like | 10:26 |
nihilifer | docker run -ti kollaglue/centos-binary-base:1.0.0 /bin/bash | 10:27 |
nihilifer | and touching some file in /dev/shm like you did before | 10:27 |
nihilifer | and then maybe check whether fresly runned kolla-ansible container can do this | 10:27 |
nihilifer | so i.e. | 10:27 |
nihilifer | docker run -ti kollaglue/centos-binary-kolla-ansible:1.0.0 /bin/bash | 10:28 |
pmisiak | hmm i have different namespace | 10:53 |
pmisiak | lokolla insted of kollaglue | 10:53 |
*** salv-orl_ has quit IRC | 10:54 | |
*** salv-orlando has joined #kolla | 10:55 | |
*** salv-orlando has quit IRC | 10:56 | |
pmisiak | http://pastebin.com/44k6znep | 10:56 |
*** salv-orlando has joined #kolla | 10:56 | |
pmisiak | in both fresh containers I can write to /dev/shm | 10:57 |
*** salv-orlando has quit IRC | 10:57 | |
*** salv-orlando has joined #kolla | 10:57 | |
pmisiak | docker inspect for problematic container: http://pastebin.com/iQPBR8DD | 11:01 |
pmisiak | and docker inspect for a fresh kolla-ansible container in which i can write to /dev/shm: http://pastebin.com/n8UEapWu | 11:03 |
pmisiak | fresh container does not have any mounts | 11:03 |
pmisiak | problematic container have /dev mount | 11:04 |
nihilifer | awww, yes, i see not that kolla-ansible mounts /dev | 11:04 |
*** neilus has joined #kolla | 11:04 | |
nihilifer | are you using selinux? | 11:04 |
nihilifer | if yes, that may be a problem | 11:05 |
pmisiak | yes, it's enabled on control01 host VM | 11:07 |
*** neilus has quit IRC | 11:09 | |
*** _tzn has joined #kolla | 11:11 | |
*** tzn has quit IRC | 11:12 | |
*** Jeffrey4l has quit IRC | 11:31 | |
pmisiak | after selinux disabling I can write to /dev/shm :) | 11:32 |
*** cemason has joined #kolla | 11:32 | |
pmisiak | so I think vagrant scripts need to disable selinux on host VMs or reconfigure it | 11:33 |
*** cemason has quit IRC | 11:37 | |
pmisiak | here you have selinux logs when touching /dev/shm/sdf file inside kolla-ansible container: http://pastebin.com/XjK4DP5f | 11:38 |
openstackgerrit | Hui Kang proposed openstack/kolla: Enable nova fake driver in nova containers https://review.openstack.org/257442 | 11:42 |
*** Jeffrey4l has joined #kolla | 11:45 | |
*** diga has quit IRC | 11:48 | |
nihilifer | pmisiak: yep. that's because selinux doesn't allow to use volumes which don't have svirt_sandbox_file_t flag | 11:53 |
nihilifer | ok, thanks for info, looks like bootstrap script for vagrant doesn't work well | 11:54 |
*** _tzn has quit IRC | 12:38 | |
*** rhallisey has joined #kolla | 12:44 | |
openstackgerrit | Andrey Shestakov proposed openstack/kolla: Add RadosGW container https://review.openstack.org/261104 | 12:49 |
openstackgerrit | Andrey Shestakov proposed openstack/kolla: Update Ceph doc for RadosGW https://review.openstack.org/261284 | 12:50 |
*** neilus has joined #kolla | 12:52 | |
*** neilus has quit IRC | 12:57 | |
ajafo | does kolla build is broken? I get http://paste.openstack.org/show/482807/ | 13:02 |
*** sdake has joined #kolla | 13:06 | |
ajafo | ok it's sthg wrong in my env | 13:08 |
nihilifer | ajafo: gate looks fine | 13:11 |
nihilifer | maybe reinstall pip package | 13:11 |
rhallisey | nihilifer, thanks for updating the kube spec. I meant to get back to that sooner | 13:12 |
nihilifer | rhallisey: np | 13:13 |
ajafo | nihilifer: thx it was sthg wrong with my env | 13:14 |
*** cemason has joined #kolla | 13:27 | |
*** cemason has quit IRC | 13:32 | |
*** jtriley has joined #kolla | 13:36 | |
*** tzn has joined #kolla | 14:10 | |
*** ashestakov has joined #kolla | 14:17 | |
*** jtriley_ has joined #kolla | 14:26 | |
*** jtriley has quit IRC | 14:26 | |
*** iNeilus has joined #kolla | 14:41 | |
*** inc0 has joined #kolla | 14:41 | |
inc0 | hey, good morning | 14:41 |
ashestakov | morning | 14:42 |
nihilifer | hi inc0 | 14:42 |
rhallisey | inc0, hey | 14:44 |
*** dave-mccowan has joined #kolla | 14:49 | |
*** iNeilus has quit IRC | 14:50 | |
ajafo | inc0: hi | 14:55 |
ajafo | success after few hours get back my env | 14:56 |
ajafo | have you got the same issue that sometimes vagrant hang up and when you cancel it or stop vm it reset env and after up vagran is provisioning it from scratch? | 14:58 |
*** salv-orl_ has joined #kolla | 15:02 | |
*** jtriley_ has quit IRC | 15:03 | |
*** unicell1 has joined #kolla | 15:03 | |
*** salv-orlando has quit IRC | 15:05 | |
*** inc0 has quit IRC | 15:05 | |
*** inc0 has joined #kolla | 15:06 | |
openstackgerrit | Kuo-tung Kao proposed openstack/kolla: change libvirt sock group to nova https://review.openstack.org/260352 | 15:06 |
*** unicell has quit IRC | 15:06 | |
*** ashestakov has quit IRC | 15:09 | |
*** kelkoobenoitr has quit IRC | 15:19 | |
openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: [WIP] Vagrant multinode deployment https://review.openstack.org/262242 | 15:23 |
openstackgerrit | Proskurin Kirill proposed openstack/kolla-mesos: Add multiple deployment support https://review.openstack.org/262247 | 15:43 |
*** salv-orl_ has quit IRC | 15:45 | |
*** salv-orlando has joined #kolla | 15:46 | |
*** Jeffrey4l has quit IRC | 15:50 | |
*** sdake has quit IRC | 15:56 | |
*** Jeffrey4l has joined #kolla | 16:07 | |
*** blahRus has joined #kolla | 16:13 | |
*** tzn has quit IRC | 16:13 | |
*** tzn has joined #kolla | 16:30 | |
*** pmisiak has quit IRC | 16:33 | |
*** inc0 has quit IRC | 16:36 | |
*** salv-orlando has quit IRC | 16:45 | |
*** salv-orlando has joined #kolla | 16:45 | |
*** diga has joined #kolla | 16:53 | |
*** diga has quit IRC | 17:00 | |
*** diga has joined #kolla | 17:00 | |
*** kproskurin has quit IRC | 17:15 | |
*** v1k0d3n has quit IRC | 17:20 | |
*** unicell1 has quit IRC | 17:22 | |
*** neilus has joined #kolla | 17:28 | |
openstackgerrit | Michal Rostecki proposed openstack/kolla-mesos: [WIP] Vagrant multinode deployment https://review.openstack.org/262242 | 17:29 |
*** neilus has quit IRC | 17:33 | |
*** bmace has quit IRC | 17:34 | |
*** bmace has joined #kolla | 17:34 | |
*** sdake has joined #kolla | 17:37 | |
SamYaple | hello guys. can I get some more reviews on the new kolla_docker module? Testing would also be nice :) https://review.openstack.org/#/c/248812/ | 18:02 |
*** unicell has joined #kolla | 18:13 | |
*** sdake has quit IRC | 18:21 | |
*** sdake has joined #kolla | 18:22 | |
*** neilus has joined #kolla | 18:22 | |
*** neilus has quit IRC | 18:27 | |
*** suro-patz has joined #kolla | 18:30 | |
*** jpeeler has quit IRC | 18:32 | |
SamYaple | dave-mccowan: re: ssl | 18:44 |
SamYaple | we talked about this a while back, but I am about to start on doing all the ssl things unless you are in the middle of that (didnt see a patch in the queue) | 18:45 |
*** neilus has joined #kolla | 19:17 | |
*** neilus has quit IRC | 19:22 | |
*** sdake has quit IRC | 19:25 | |
*** salv-orlando has quit IRC | 19:33 | |
*** diga has quit IRC | 19:34 | |
*** salv-orlando has joined #kolla | 19:50 | |
*** salv-orlando has quit IRC | 20:03 | |
*** salv-orlando has joined #kolla | 20:03 | |
*** kragniz has left #kolla | 20:32 | |
*** mandre has quit IRC | 20:39 | |
*** mandre has joined #kolla | 20:40 | |
*** Jeffrey4l has quit IRC | 20:54 | |
*** Jeffrey4l has joined #kolla | 21:00 | |
*** neilus has joined #kolla | 21:05 | |
*** neilus has quit IRC | 21:10 | |
*** sdake has joined #kolla | 21:15 | |
*** openstackgerrit has quit IRC | 21:17 | |
*** openstackgerrit has joined #kolla | 21:17 | |
*** jasonsb has quit IRC | 21:20 | |
*** dave-mccowan has quit IRC | 21:44 | |
*** v1k0d3n has joined #kolla | 21:45 | |
*** cemason has joined #kolla | 21:56 | |
*** blahRus has quit IRC | 22:05 | |
*** mark-casey1 has joined #kolla | 22:06 | |
*** tzn has quit IRC | 22:18 | |
*** tzn has joined #kolla | 22:19 | |
*** cemason has quit IRC | 22:21 | |
mark-casey1 | hello all. I've found this howto (http://blog.oddbit.com/2014/05/28/multiple-external-networks-wit/) on adding additional external networks to an existing OpenStack deployment and I'm wondering if there is anything about the way that the Neutron service has been dockerized in Kolla will prevent me from doing this. | 22:22 |
mark-casey1 | I've no problem figuring my way through I'd hate to spend a bunch of time on it if there's any obvious "can't get there from here" issue | 22:23 |
*** tzn has quit IRC | 22:24 | |
*** salv-orl_ has joined #kolla | 22:34 | |
*** salv-orlando has quit IRC | 22:37 | |
SamYaple | mark-casey1: you can absolutely use multiple external networks with kolla | 22:45 |
SamYaple | it is not setup in an automated manner however | 22:45 |
SamYaple | in the case of openvswitch you would have to manually add another bridge (and add that interface to that bridge) and update teh ml2_conf.ini | 22:46 |
*** neilus has joined #kolla | 22:53 | |
*** mark-casey has joined #kolla | 22:55 | |
mark-casey | SamYaple: awesome. thanks. I'll mess around with it. I'm gonna be brave and switch from Fuel at the same time so I may be back | 22:55 |
*** mark-casey1 has quit IRC | 22:56 | |
SamYaple | mark-casey: yay | 22:57 |
*** neilus has quit IRC | 22:58 | |
*** tzn has joined #kolla | 23:06 | |
*** mark-casey1 has joined #kolla | 23:07 | |
*** mark-casey has quit IRC | 23:08 | |
openstackgerrit | Artur Zarzycki proposed openstack/kolla: Base and openstack-base ubuntu binary containers. https://review.openstack.org/261957 | 23:12 |
*** mark-casey has joined #kolla | 23:20 | |
openstackgerrit | Artur Zarzycki proposed openstack/kolla: Heat ubuntu binary container https://review.openstack.org/262035 | 23:22 |
*** tzn has quit IRC | 23:23 | |
*** mark-casey1 has quit IRC | 23:24 | |
*** sdake has quit IRC | 23:40 | |
*** neilus has joined #kolla | 23:47 | |
openstackgerrit | Artur Zarzycki proposed openstack/kolla: Glance ubuntu binary container https://review.openstack.org/262004 | 23:52 |
*** neilus has quit IRC | 23:52 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!