*** HagunKim has joined #openstack-helm | 01:14 | |
*** gkadam has joined #openstack-helm | 01:16 | |
*** nishantkr has quit IRC | 01:39 | |
*** gkadam has quit IRC | 01:40 | |
*** michael-beaver has quit IRC | 02:00 | |
*** ianychoi has joined #openstack-helm | 02:00 | |
*** cfriesen has quit IRC | 03:13 | |
*** gkadam has joined #openstack-helm | 03:23 | |
*** gkadam has quit IRC | 03:23 | |
*** ianychoi has quit IRC | 03:49 | |
*** ianychoi has joined #openstack-helm | 03:49 | |
*** ianychoi has quit IRC | 03:51 | |
*** ianychoi has joined #openstack-helm | 03:52 | |
openstackgerrit | Prabhjot Singh Sethi proposed openstack/openstack-helm-infra master: [DONOT MERGE] test review https://review.opendev.org/659715 | 03:57 |
---|---|---|
*** sgrasley has quit IRC | 05:00 | |
openstackgerrit | Rahul Khiyani proposed openstack/openstack-helm-infra master: Ingress: Fix read-only-fs https://review.opendev.org/659719 | 05:23 |
*** pgaxatte has joined #openstack-helm | 06:09 | |
openstackgerrit | Rahul Khiyani proposed openstack/openstack-helm-infra master: Ingress: Fix read-only-fs https://review.opendev.org/659719 | 06:19 |
*** witek has joined #openstack-helm | 06:44 | |
HagunKim | I get error about k8s, when I try to install OSH using latest openstack-helm, openstack-helm-infra. | 06:50 |
HagunKim | kubernetes-entrypoint docker can't get service dependency. | 06:50 |
HagunKim | "Entrypoint WARNING: 2019/05/17 06:35:05 entrypoint.go:71: Resolving dependency Service rabbitmq in namespace openstack failed: Get https://10.96.0.1:443/api/v1/namespaces/openstack/endpoints/rabbitmq: dial tcp 10.96.0.1:443: i/o timeout ." | 06:50 |
HagunKim | I used my openstack VM(ubuntu 16.04) and OSH document(https://docs.openstack.org/openstack-helm/latest/install/developer/index.html) | 06:50 |
HagunKim | Does anyone know about this? | 06:50 |
*** pma78 has joined #openstack-helm | 07:00 | |
*** rezroo has joined #openstack-helm | 07:17 | |
*** jsuchome has joined #openstack-helm | 07:18 | |
openstackgerrit | John Haan proposed openstack/openstack-helm master: fix wrong mount key for scheduler and consoleauth https://review.opendev.org/659732 | 07:35 |
*** mikkos_ has joined #openstack-helm | 07:42 | |
*** mikkos_ is now known as mikkos | 07:45 | |
itxaka | HagunKim: is your rabbit container up? whats the output of kubectl -n openstack get pods? | 08:08 |
itxaka | I just deployed yesterday with those steps and found no issues | 08:09 |
openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-helm-infra master: Wait for pods for openstack-support scenario https://review.opendev.org/659284 | 08:12 |
openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-helm-infra master: Adapt rabbitmq test for py2 and py3 support https://review.opendev.org/659301 | 08:13 |
openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-helm-infra master: Wait for pods for openstack-support scenario https://review.opendev.org/659284 | 08:15 |
*** nam228 has joined #openstack-helm | 08:16 | |
openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-helm-infra master: Point to OSH-images images https://review.opendev.org/658766 | 08:17 |
openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-helm-infra master: Wait for pods for openstack-support scenario https://review.opendev.org/659284 | 08:18 |
openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-helm-infra master: Wait for pods for openstack-support scenario https://review.opendev.org/659284 | 08:18 |
nam228 | hello | 08:23 |
itxaka | o/ | 08:25 |
openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-helm-infra master: Adapt rabbitmq test for py2 and py3 support https://review.opendev.org/659301 | 08:46 |
openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-helm-infra master: Adapt rabbitmq test for py2 and py3 support https://review.opendev.org/659301 | 08:47 |
openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-helm-infra master: Adapt rabbitmq test for py2 and py3 support https://review.opendev.org/659301 | 08:47 |
evrardjp | itxaka: this should clarify things for you ^ | 08:48 |
*** roman_g has joined #openstack-helm | 08:51 | |
evrardjp | itxaka: if you check the latest comment, it should clarify the intent for you | 08:53 |
evrardjp | itxaka: you worked on a mariadb issue, right? | 09:00 |
evrardjp | because I see gates failing today with that | 09:01 |
openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-helm-infra master: Adapt rabbitmq test for py2 and py3 support https://review.opendev.org/659301 | 09:27 |
openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-helm-infra master: Adapt rabbitmq test for py2 and py3 support https://review.opendev.org/659301 | 09:28 |
*** rezroo has quit IRC | 09:31 | |
nick_kar | evrardjp: hi | 09:32 |
nick_kar | evrardjp: I'm having problem with networking of an osh installation over a k8s cluster installed by OPNFV XCI | 09:33 |
evrardjp | not sure I can help, but shoot :) | 09:34 |
nick_kar | evrardjp: the k8s cluster provides a bridge named 'br-vlan' but I can't get access to the br-ex | 09:34 |
nick_kar | evrardjp: and therefore no external access | 09:35 |
evrardjp | what you are saying makes sense... but as I said, I am not sure I can help you... maybe itxaka ? | 09:36 |
nick_kar | evrardjp: thanks | 09:36 |
evrardjp | Do you have logs we can use? | 09:36 |
nick_kar | evrardjp: give a minute | 09:37 |
evrardjp | and like ConfigMaps/secrets of the neutron config would help | 09:37 |
nick_kar | evrardjp: the scripts 140-170 and 900 : http://paste.openstack.org/show/751504/ | 09:41 |
nick_kar | evrardjp: ConfigMap of neutron-bin: http://paste.openstack.org/show/751506/ | 09:49 |
nick_kar | evrardjp: neutron-etc secrets: http://paste.openstack.org/show/751507/ | 09:56 |
itxaka | evrardjp: thanks! I commented too slow in that rabbitmq test stuff, sorry about that, I will re-review shortly | 10:00 |
itxaka | and indeed, Im working on the mysql issue chrashloopbackoff | 10:01 |
evrardjp | itxaka: cool | 10:02 |
nick_kar | evrardjp: itxaka: I forgot to mention that I've changed br-ex to br-vlan in https://github.com/openstack/openstack-helm/blob/4550ea9376026d55a583dd650e4ffd070e968b07/tools/deployment/developer/ceph/160-compute-kit.sh#L64 and L67 otherwise I get error in https://github.com/openstack/openstack-helm/blob/aca516f3a2483e0f56c4c5cb7c92c782cfd6bba1/tools/deployment/developer/common/170-setup-gateway.sh#L21 | 10:02 |
itxaka | nick_kar: sorry, network is not my forte, what is the exact error that you see? | 10:02 |
jsuchome | itxaka: what's mysql issue? do you already have a patch? | 10:03 |
evrardjp | I do not. | 10:03 |
evrardjp | oh sorry that was for itxaka:) | 10:03 |
*** JangwonLee_ has quit IRC | 10:04 | |
nick_kar | itxaka: eventually "Could not connect to 172.24.4.5 port 22 in time" in 900-use-it.sh | 10:04 |
nick_kar | itxaka: that's the floating IP of the test VM | 10:04 |
itxaka | ummm, I have not used the developer scripts to deploy lik ever, only the /tools/deployment/component/* scripts | 10:04 |
itxaka | jsuchome: https://storyboard.openstack.org/#!/story/2004702 no patch yet but able to reproduce it so a patch would come shortly-ish? I hope :P | 10:05 |
jsuchome | itxaka: "IndexError: list index out of range" looks like poorly written start script or something | 10:09 |
itxaka | indeed, seems to be an oversigth when restarting a mariadb pod, which knows that it was running before and tries to open the file directly, which doesnt exists yet | 10:12 |
openstackgerrit | Itxaka Serrano Garcia proposed openstack/openstack-helm-infra master: [DNM] mysql: fix restart on single node mariadb cluster https://review.opendev.org/659777 | 10:59 |
nam228 | Hi | 11:17 |
nam228 | I'm installing openstack-helm on a Centos 7 VM | 11:18 |
nam228 | I follow the instruction in Openstack website: https://docs.openstack.org/openstack-helm/latest/install/developer/kubernetes-and-common-setup.html#install-openstack-helm | 11:19 |
nam228 | and getting stuck when running the script to install Kubenetes | 11:19 |
nam228 | ./tools/deployment/developer/common/010-deploy-k8s.sh | 11:20 |
nam228 | Here is the error messages: https://pastebin.com/93g84D0U | 11:20 |
nam228 | and the whole log is in here: https://pastebin.com/J2s49Zu9 | 11:21 |
nam228 | the version of my Centos VM : Linux version 3.10.0-957.12.1.el7.x86_64 (mockbuild@kbuilder.bsys.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-36) (GCC) ) #1 SMP Mon Apr 29 14:59:59 UTC 2019 | 11:21 |
nam228 | It seems that there is a problem when using helm to install Calico in master node. | 11:22 |
nam228 | Anyone has any ideas about this error ? | 11:23 |
itxaka | nam228: check the output of kubectl -n kube-system get pods and kubectl -n kube-system logs $CALICO_POD_NAME | 11:38 |
itxaka | substitute the $CALICO_POD_NAME for the name of the pod obtained with the first command | 11:39 |
nam228 | kubectl has not been installed yet in my VM | 11:42 |
nam228 | https://www.irccloud.com/pastebin/owqAhg02/ | 11:42 |
*** mbuil has joined #openstack-helm | 11:42 | |
nam228 | Do you mean I should access the mater node and run kubectl.... ? | 11:42 |
nam228 | but at the moment my VM has kubelet | 11:46 |
openstackgerrit | Itxaka Serrano Garcia proposed openstack/openstack-helm-infra master: mysql: enable local PersistentVolume on single pod cluster https://review.opendev.org/659782 | 11:55 |
itxaka | nam228: yeah you should, it seems that it timed out while deploying so there is probably an issue in the pods, and thats the only way of getting those logs | 11:56 |
nam228 | ok I see, thanks for your suggestion | 12:05 |
openstackgerrit | Doug Aaser proposed openstack/openstack-helm-infra master: Patroni inclusion work for HA Postgres https://review.opendev.org/644388 | 12:22 |
jsuchome | itxaka: looks like a good idea ... but now my usual habit of nitpicking in commit message starts... | 12:32 |
itxaka | :D | 12:33 |
portdirect | itxaka: i need to look into this, but cant reproduce locally | 12:33 |
portdirect | but for gating/singe node, why not just use a bind mount from the host? | 12:33 |
jsuchome | itxaka: also, one question - how do I make sure the data _are_ deleted when I want to? | 12:35 |
itxaka | portdirect: re: bind mount...I did not know about that, probably its easier lmao | 12:37 |
itxaka | also I can reproduce it 100% on a single pod cluster under ubuntu like on our gates | 12:37 |
portdirect | will try today | 12:37 |
portdirect | re the bind mount: turn this false: https://github.com/openstack/openstack-helm-infra/blob/master/mariadb/values.yaml#L233 | 12:38 |
portdirect | and it will use this: https://github.com/openstack/openstack-helm-infra/blob/master/mariadb/templates/statefulset.yaml#L240-L243 | 12:38 |
portdirect | we could update that to support specifying a dir to bind in | 12:39 |
itxaka | with that on false I do get: " mysql-data: | 12:39 |
itxaka | Type: EmptyDir (a temporary directory that shares a pod's lifetime) | 12:39 |
itxaka | Medium: " | 12:39 |
itxaka | which I guess its an empty dir inside the container but not reflected outside, so the same issue arises of missing the data on pod restart/kill | 12:40 |
portdirect | yup | 12:40 |
portdirect | so we just need to update that to support specifying a dir to bind in | 12:40 |
itxaka | ok, that sounds good, but do we want to actually support that? my patch mkaes mention and makes it so you need to go out of your way to enable that, becuase it wont be supported due to the issues that can arise from there, i.e. if its scheduled in a different woorker, then it will also crashloopbackoff | 12:41 |
portdirect | thats a good question | 12:42 |
portdirect | i think its one of the reasons we used emptydir in the 1st place | 12:43 |
portdirect | so that it was expliclty unsupported, but allowed you to use a storageclass of your choosing | 12:43 |
portdirect | that could provide a lighterweight soultion than say ceph | 12:43 |
openstackgerrit | Pete Birley proposed openstack/openstack-helm-infra master: Rabbit: Ensure node has joined cluster on initial startup https://review.opendev.org/659671 | 12:46 |
openstackgerrit | Pete Birley proposed openstack/openstack-helm-infra master: Rabbit: Ensure node has joined cluster on initial startup https://review.opendev.org/659671 | 12:48 |
openstackgerrit | Itxaka Serrano Garcia proposed openstack/openstack-helm-infra master: mysql: set hostpath for mysql-data https://review.opendev.org/659787 | 12:52 |
*** aaronsheffield has joined #openstack-helm | 12:52 | |
itxaka | that one is simpler :D | 12:53 |
*** mikkos has quit IRC | 12:56 | |
openstackgerrit | Itxaka Serrano Garcia proposed openstack/openstack-helm-infra master: mysql: set hostpath for mysql-data https://review.opendev.org/659787 | 12:59 |
*** kranthikirang has joined #openstack-helm | 13:04 | |
jsuchome | that is much simpler itxaka :-) | 13:09 |
portdirect | Also I'm sorry im so behind on admin work atm | 13:11 |
portdirect | but I just noticed that I sent this from the wrong addr, so it had not actually made it to the mailing list | 13:11 |
portdirect | http://lists.openstack.org/pipermail/openstack-discuss/2019-May/006382.html | 13:12 |
portdirect | cores, please vote ^ | 13:12 |
portdirect | alanmeadows / lamt / srwilkers / mattmceuen | 13:12 |
evrardjp | itxaka: I commented on the commit message of https://review.opendev.org/#/c/659787/2 ? I am not sure to understand the last paragraph, could you clarify? | 13:15 |
srwilkers | Will do | 13:17 |
*** rezroo has joined #openstack-helm | 13:18 | |
*** howell has joined #openstack-helm | 13:21 | |
pgaxatte | hello | 13:24 |
pgaxatte | is there a way to deploy a new service (let's say glance) with a different endpoint URL and a different service name? | 13:24 |
pgaxatte | the service name seems hardcoded in the templates/job-ks-service.yaml | 13:25 |
pgaxatte | the endpoint URL on the other end seems more dynamic, right? | 13:26 |
*** portdirect has quit IRC | 13:32 | |
*** portdirect has joined #openstack-helm | 13:33 | |
*** ChanServ sets mode: +o portdirect | 13:33 | |
openstackgerrit | Merged openstack/openstack-helm master: Fix a typo in a comment for neutron values https://review.opendev.org/659154 | 13:55 |
openstackgerrit | Cliff Parsons proposed openstack/openstack-helm-infra master: Postgresql backup/restore enhancements https://review.opendev.org/655231 | 13:59 |
*** nishantkr has joined #openstack-helm | 14:00 | |
nick_kar | portdirect: hi | 14:03 |
nick_kar | portdirect: I asked about a networking problem earlier but it seems that you are the expert | 14:04 |
nick_kar | I'm having problem with networking of an osh installation over a k8s cluster installed by OPNFV XCI | 14:05 |
itxaka | pgaxatte: seems so :( | 14:05 |
nick_kar | the k8s cluster provides a bridge named 'br-vlan' but I can't get access to the br-ex of openstack | 14:05 |
itxaka | basically you would need to duplicate that job-ks-service.yaml to modify it :( | 14:06 |
*** dpawlik has quit IRC | 14:24 | |
*** roman_g has quit IRC | 14:28 | |
openstackgerrit | Sandeep Reddy Thumma proposed openstack/openstack-helm-addons master: Ranger: Add pod/container security context https://review.opendev.org/656405 | 14:31 |
openstackgerrit | Sandeep Reddy Thumma proposed openstack/openstack-helm-addons master: Ranger-Agent: Add pod/container security context https://review.opendev.org/656942 | 14:31 |
openstackgerrit | PRATEEK REDDY DODDA proposed openstack/openstack-helm master: Implement Security Context for Neutron https://review.opendev.org/657854 | 14:34 |
openstackgerrit | PRATEEK REDDY DODDA proposed openstack/openstack-helm master: Implement Security Context for Neutron https://review.opendev.org/657854 | 14:36 |
pgaxatte | itxaka: i don't know how exactly but apparently the service name can be changed | 14:37 |
pgaxatte | i managed to do that by modifying the endoint | 14:37 |
pgaxatte | endpoint* | 14:37 |
pgaxatte | if you change the name in the endpoint of the service, it creates a different service | 14:38 |
*** nick_kar has quit IRC | 14:48 | |
*** pgaxatte has quit IRC | 14:58 | |
*** cfriesen has joined #openstack-helm | 15:04 | |
openstackgerrit | Jean-Philippe Evrard proposed openstack/openstack-helm-infra master: Adapt rabbitmq test for py2 and py3 support https://review.opendev.org/659301 | 15:13 |
openstackgerrit | PRATEEK REDDY DODDA proposed openstack/openstack-helm master: Implement Security Context for Neutron https://review.opendev.org/657854 | 15:14 |
*** roman_g has joined #openstack-helm | 15:14 | |
*** witek has quit IRC | 15:45 | |
*** nam228 has quit IRC | 15:47 | |
openstackgerrit | Gage Hugo proposed openstack/openstack-helm-infra master: Create upgrade statefulset tpl for helm-toolkit https://review.opendev.org/659830 | 16:12 |
*** jsuchome has quit IRC | 16:16 | |
*** sreejithp has joined #openstack-helm | 16:18 | |
*** roman_g has quit IRC | 16:42 | |
openstackgerrit | chinasubbareddy mallavarapu proposed openstack/openstack-helm-infra master: Ceph-Client: Update, Enable and Cleanup helm tests https://review.opendev.org/647596 | 16:42 |
georgk | portdirect, alanmeadows: it would be nice if you could merge this one: https://review.opendev.org/#/c/659581/ Thank you! | 16:49 |
openstackgerrit | Matthew Heler proposed openstack/openstack-helm-infra master: [Ceph Enhancement] Move ceph-defragosds cron job to ceph-client chart https://review.opendev.org/641198 | 16:58 |
openstackgerrit | Doug Aaser proposed openstack/openstack-helm-infra master: Patroni inclusion work for HA Postgres https://review.opendev.org/644388 | 17:01 |
*** roman_g has joined #openstack-helm | 17:01 | |
openstackgerrit | Gage Hugo proposed openstack/openstack-helm-infra master: Create upgrade statefulset tpl for helm-toolkit https://review.opendev.org/659830 | 17:15 |
openstackgerrit | Gage Hugo proposed openstack/openstack-helm-infra master: Create upgrade statefulset tpl for helm-toolkit https://review.opendev.org/659830 | 17:18 |
*** nicolasbock has joined #openstack-helm | 17:22 | |
openstackgerrit | chinasubbareddy mallavarapu proposed openstack/openstack-helm-infra master: Ceph: Decrease ceph mon data store space to 20% https://review.opendev.org/659847 | 17:23 |
openstackgerrit | Stephen Taylor proposed openstack/openstack-helm-infra master: Do not use blkid to create partition symlinks https://review.opendev.org/659849 | 17:24 |
openstackgerrit | chinasubbareddy mallavarapu proposed openstack/openstack-helm-infra master: Ceph: Decrease ceph mon data store space to 20% https://review.opendev.org/659847 | 17:29 |
openstackgerrit | Rahul Khiyani proposed openstack/openstack-helm-infra master: Rabbitmq: Fix read-only-fs https://review.opendev.org/659852 | 17:33 |
openstackgerrit | Krishna Venkata proposed openstack/openstack-helm-infra master: [WIP] Implementing and parameterize Unavailable value for Ceph components https://review.opendev.org/658492 | 17:34 |
openstackgerrit | chinasubbareddy mallavarapu proposed openstack/openstack-helm-infra master: Ceph-Client: Update, Enable and Cleanup helm tests https://review.opendev.org/647596 | 17:55 |
openstackgerrit | Doug Aaser proposed openstack/openstack-helm-infra master: Patroni inclusion work for HA Postgres https://review.opendev.org/644388 | 18:08 |
*** roman_g has quit IRC | 18:16 | |
openstackgerrit | Rahul Khiyani proposed openstack/openstack-helm-infra master: Rabbitmq: Fix read-only-fs https://review.opendev.org/659852 | 18:22 |
*** rezroo has quit IRC | 18:22 | |
*** itlinux has joined #openstack-helm | 18:38 | |
*** mbuil has quit IRC | 18:50 | |
*** kranthikirang has quit IRC | 18:52 | |
openstackgerrit | chinasubbareddy mallavarapu proposed openstack/openstack-helm-infra master: Ceph-Client: Update, Enable and Cleanup helm tests https://review.opendev.org/647596 | 19:37 |
openstackgerrit | chinasubbareddy mallavarapu proposed openstack/openstack-helm-infra master: Ceph: Decrease ceph mon data store space to 15% https://review.opendev.org/659847 | 19:40 |
*** kranthikirang has joined #openstack-helm | 19:46 | |
*** itlinux has quit IRC | 20:06 | |
openstackgerrit | Merged openstack/openstack-helm master: Expose Anti-Affinity Weight Setting. https://review.opendev.org/658518 | 20:10 |
openstackgerrit | Krishna Venkata proposed openstack/openstack-helm-infra master: [WIP] Implementing and parameterize Unavailable value for Ceph components https://review.opendev.org/658492 | 20:17 |
*** unicell has joined #openstack-helm | 20:34 | |
openstackgerrit | Gage Hugo proposed openstack/openstack-helm master: Change cred-cleanup to not restart on failure https://review.opendev.org/656804 | 20:39 |
openstackgerrit | Merged openstack/openstack-helm-infra master: Do not use blkid to create partition symlinks https://review.opendev.org/659849 | 20:51 |
*** howell has quit IRC | 21:00 | |
*** KeithMnemonic has quit IRC | 21:08 | |
*** kranthikirang has quit IRC | 21:20 | |
*** sreejithp has quit IRC | 21:32 | |
*** rezroo has joined #openstack-helm | 21:33 | |
*** rezroo has quit IRC | 21:41 | |
*** itlinux has joined #openstack-helm | 21:57 | |
openstackgerrit | Krishna Venkata proposed openstack/openstack-helm-infra master: Parameterize updateStrategy values for Ceph components. Making it customizable for future. https://review.opendev.org/658492 | 22:01 |
*** rchurch_ has joined #openstack-helm | 22:01 | |
*** rchurch has quit IRC | 22:02 | |
georgk | portdirect: hi! | 22:33 |
openstackgerrit | Krishna Venkata proposed openstack/openstack-helm-infra master: Parameterize updateStrategy values for Ceph components. Making it customizable for future. https://review.opendev.org/658492 | 22:33 |
*** unicell has left #openstack-helm | 22:36 | |
openstackgerrit | Georg Kunz proposed openstack/openstack-helm master: Extending Neutron chart with support for OVS DPDK https://review.opendev.org/643284 | 22:39 |
openstackgerrit | Georg Kunz proposed openstack/openstack-helm-infra master: Extended OVS chart with support for DPDK https://review.opendev.org/626894 | 22:40 |
openstackgerrit | Pete Birley proposed openstack/openstack-helm-infra master: Rabbit: Ensure node has joined cluster on initial startup https://review.opendev.org/659671 | 22:47 |
georgk | mattmceuen, alanmeadows, srwilkers: are you guys still around? | 22:49 |
*** itlinux_ has joined #openstack-helm | 22:51 | |
*** itlinux has quit IRC | 22:52 | |
srwilkers | georgk: I’m still out on holiday | 23:01 |
georgk | srwilkers: oh, ok, then never mind and have a nice holiday / weekend | 23:02 |
*** aaronsheffield has quit IRC | 23:19 | |
*** rezroo has joined #openstack-helm | 23:26 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!