Monday, 2014-05-26

*** ramishra has joined #tripleo00:12
*** nati_ueno has joined #tripleo00:14
*** ramishra has quit IRC00:17
*** xuhaiwei has joined #tripleo00:21
xuhaiweimorning00:21
*** rcarrillocruz has quit IRC00:36
*** nati_ueno has quit IRC00:40
*** shakamunyi has joined #tripleo00:40
*** nati_ueno has joined #tripleo00:41
*** rcarrillocruz has joined #tripleo00:41
*** zackf has quit IRC00:49
*** shakamunyi has quit IRC00:50
*** nati_ueno has quit IRC00:51
tchaypoxuhaiwei: btw, i think my patch to fix the file:// index-url has landed in pip 1.5.600:54
tchaypocould you update your pip and see if it fixes your problems?00:54
xuhaiweitchaypo: thanks, i will check it00:55
*** matsuhashi has joined #tripleo00:55
xuhaiweitchaypo: now, i met another problem, when i destroyed the environment, and tried to build it again, it failed in building the undercloud, i checked the log, and found the 'cound' is 0 in seedcloud.00:59
*** nosnos has joined #tripleo00:59
xuhaiweitchaypo: i don't know the 'count' is configured by what01:00
*** zackf has joined #tripleo01:09
lifelessxuhaiwei: which count ?01:09
xuhaiweilifeless: with nova hypervisor-stats command01:10
*** ramishra has joined #tripleo01:13
xuhaiweilifeless: it failed in this line in devtest_undercloud.sh :wait_for 60 1 [ "\$(nova hypervisor-stats | awk '\$2==\"count\" { print \$4}')" != "0" ]01:14
lifelessthat means either a) you have not registered a node with the seed cloud or b) you already have an instance deployed on it01:15
*** shakamunyi has joined #tripleo01:16
xuhaiweilifeless: where to register the seedcloud01:17
*** killer_prince has quit IRC01:17
*** ramishra has quit IRC01:17
xuhaiweilifeless: i just ran the devtest.sh, and the seedcloud is running, when ssh seedcloud and run nova baremetal-node-list there is nothing01:19
lifelessif you ran devtest.sh it should have registered things itself01:21
lifelesssetup-baremetal is the script that gets called to configure a baremetal cloud01:21
lifelessit calls register-nodes which registers nodes with the cloud01:21
xuhaiweilifeless: yes, i see that , i think the register-nodes failed01:21
xuhaiweilifeless: i also got this log: + setup-baremetal --service-host seed --nodes /dev/fd/63    ERROR: The server has either erred or is incapable of performing the requested operation. (HTTP 500) (Request      -ID: req-8f685a8e-48b9-4387-84dd-6cd17a7e1fd3)01:22
xuhaiweilifeless: i checked the testenv.json, and the seedcloud information is not in it01:23
xuhaiweilifeless: is this the reason01:23
xuhaiwei?01:23
lifelessno01:24
lifelessthats a symptom not the cause01:24
lifelessthe 500 error - you'll need to look in the logs on the seed01:24
xuhaiweilifeless: yes, i saw it, it's a DBError01:25
xuhaiweilifeless: in the nova-api.log:  DBError: (DataError) (1406, "Data too long for column 'pm_password' at row 1") 'INSERT INTO bm_nodes (created_at, updated_at, deleted_at, deleted, uuid, service_host, instance_uuid, instance_name, cpus, memory_mb, local_gb, preserve_ephemeral, pm_address, pm_user, pm_password, task_state, terminal_port, image_path, pxe_config_path, deploy_key, root_mb, swap_mb, ephemeral_mb) VALUES (%s, %s, %s, %s, %01:26
*** shakamunyi has quit IRC01:26
tchaypoxuhaiwei: there was a post from rcarrillocruz on the openstack-dev list about that, he linked to what he thinks is the relevant bug01:27
openstackgerritSteve Kowalik proposed a change to openstack/os-cloud-config: Check for errors while parsing JSON  https://review.openstack.org/9539501:28
openstackgerritSteve Kowalik proposed a change to openstack/os-cloud-config: Check for relevant environment variables  https://review.openstack.org/9539601:28
xuhaiweitchaypo: can i have the link?01:29
tchaypoxuhaiwei: sorry, I don't have it handy. It was a post on openstack-dev within the last 24 hours though01:29
xuhaiweitchaypo: you mean the openstack-dev mailing list?01:30
tchaypoyes01:30
tchayposubject is [openstack-dev] [TripleO] devtest_seed.sh erroring with a 50001:30
xuhaiweitchaypo: thanks01:31
StevenKtchaypo: Which doesn't help01:31
StevenKIt shows the exact same symptom as what xuhaiwei is seeing, but no solution01:32
tchaypoit links to a bug01:32
tchaypowhich I've just been reading, and which links to a review in tripleo-incubator that purports to have a fix01:33
StevenKtchaypo: The message I have doesn't link to a bug, and then he moves it to the user list01:34
tchaypooh01:34
*** yamahata has joined #tripleo01:35
tchaypogood point01:38
StevenKHowever, I found the review01:38
xuhaiweiStevenK: could you show me the link please?01:45
StevenKxuhaiwei: https://review.openstack.org/#/c/95181/ is the review01:52
*** shakamunyi has joined #tripleo01:53
*** shakamunyi has quit IRC01:57
*** shakamunyi has joined #tripleo01:58
xuhaiweiStevenK: thank you01:59
*** shakamunyi has quit IRC02:05
*** ramishra has joined #tripleo02:14
*** ramishra has quit IRC02:17
*** ramishra has joined #tripleo02:17
openstackgerritSteve Baker proposed a change to openstack/diskimage-builder: Always bind-mount pypi mirror if dir exists  https://review.openstack.org/9539702:21
*** yamahata has quit IRC02:30
*** yamahata has joined #tripleo02:30
*** shakamunyi has joined #tripleo02:30
*** nati_ueno has joined #tripleo02:33
lifelessbrain fading from a day of reviews. Ugh.02:36
*** ramishra has quit IRC02:39
*** ramishra has joined #tripleo02:39
*** shakamunyi has quit IRC02:40
*** ramishra has quit IRC02:40
stevebakertchaypo: do you have a bug # for your pip fix?02:41
tchaypoI have a bug from xuhaiwei about the original behaviour02:41
tchaypohttps://bugs.launchpad.net/tripleo/+bug/130122002:43
uvirtbotLaunchpad bug 1301220 in tripleo "pip isn't case-insensitive when using a file:///-based index-url" [High,In progress]02:43
stevebakertchaypo: thanks02:43
*** ramishra has joined #tripleo02:43
*** nati_ueno has quit IRC02:43
*** fandi has joined #tripleo02:54
*** untriaged-bot has joined #tripleo03:00
untriaged-botUntriaged bugs so far:03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132194303:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/132156303:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/131876703:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/131497803:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/131535503:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/131667503:00
uvirtbotLaunchpad bug 1321943 in tripleo "Ceilometer Swift polling on overcloud control node fails with a 403 forbidden error" [Undecided,In progress]03:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132009003:00
uvirtbotLaunchpad bug 1321563 in diskimage-builder "Add support in ramdisk element to fetch IP from dhcp" [Undecided,New]03:00
uvirtbotLaunchpad bug 1318767 in tripleo "apache element SSL cert check fails" [Undecided,New]03:00
*** untriaged-bot has quit IRC03:00
uvirtbotLaunchpad bug 1314978 in tripleo "Cloud vm not pingable after overcloud upgrade " [Undecided,New]03:00
uvirtbotLaunchpad bug 1315355 in tripleo "Upgrade of overcloud failed with "Connection to neutron failed: Maximum attempts reached"" [Undecided,New]03:00
uvirtbotLaunchpad bug 1316675 in tripleo "Saving a devtest VM results in error" [Undecided,In progress]03:00
uvirtbotLaunchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New]03:00
*** shakamunyi has joined #tripleo03:06
*** fandi has quit IRC03:11
*** shakamunyi has quit IRC03:17
*** matsuhashi has quit IRC03:19
lifelesstchaypo: so are you doing a spec about rationalise UI ?03:25
tchaypoSomeone has started one that touches on some of it - putting data into heat templates instead of env vars03:27
*** fandi has joined #tripleo03:28
tchaypohttps://review.openstack.org/#/c/94910/03:31
lifelessyes, I am reviewing that03:32
lifelessI don't think it really gets into it. Its narrow. And AFAICT confused in its heart.03:32
*** nosnos has quit IRC03:33
openstackgerritSteve Baker proposed a change to openstack/tripleo-image-elements: Refresh heat-cfntools element  https://review.openstack.org/9203603:34
openstackgerritSteve Baker proposed a change to openstack/tripleo-image-elements: Prepare os-*-config for CI  https://review.openstack.org/9540303:34
tchaypoI don't think I've commented on it yet but I feel like it's only addressing one part of rationalise-ui03:34
tchaypoHeat is one place we might put data; but we also need a standard way to pass it been tools03:34
tchaypoSo they can be loosely joined03:35
tchaypoUnless all tools are going to rely on being able to read from heat03:35
openstackgerritSteve Baker proposed a change to openstack/tripleo-image-elements: Refresh heat-cfntools element  https://review.openstack.org/9203603:39
openstackgerritSteve Baker proposed a change to openstack/tripleo-image-elements: Prepare os-*-config for CI  https://review.openstack.org/9540303:39
*** tserong has quit IRC03:41
*** shakamunyi has joined #tripleo03:43
lifelesstchaypo: I have reviewed that spec now, FWIW03:44
*** nati_ueno has joined #tripleo03:47
tchaypoi saw the email03:48
lifelessheh :)03:50
*** shakamunyi has quit IRC03:53
*** cody-somerville has quit IRC04:06
*** nati_ueno has quit IRC04:14
*** nati_ueno has joined #tripleo04:14
*** shakamunyi has joined #tripleo04:19
*** matsuhashi has joined #tripleo04:21
*** nosnos has joined #tripleo04:21
openstackgerritJames Polley proposed a change to openstack/tripleo-incubator: Reformat README as ReST and include in built docs  https://review.openstack.org/9530804:29
*** ramishra has quit IRC04:29
*** shakamunyi has quit IRC04:29
*** ramishra has joined #tripleo04:30
openstackgerritJames Polley proposed a change to openstack/tripleo-incubator: Reformat README as ReST and include in built docs  https://review.openstack.org/9530804:33
tchaypoI was briefly curious about why we're not being bitten by https://bugs.launchpad.net/nova/+bug/1322599 in ci04:36
uvirtbotLaunchpad bug 1322599 in tripleo "nova baremetal-node-create fails with HTTP 500" [Critical,In progress]04:36
tchaypoactually on second thoughs i'm still curious04:37
tchaypoon third thougts is obvious - ci only tests building the seed with ironic, this is a nova-bm issue.04:38
*** lazy_prince has joined #tripleo04:42
*** nati_ueno has quit IRC04:45
StevenKtchaypo: The undercloud jobs without ironic use nova-bm04:53
StevenKtchaypo: Since the undercloud tests the seed as welll ...04:54
tchaypoOkay. So how are those tests passing?04:54
*** shakamunyi has joined #tripleo04:55
*** e0ne has joined #tripleo05:01
*** nati_ueno has joined #tripleo05:03
*** shakamunyi has quit IRC05:05
*** e0ne has quit IRC05:06
*** e0ne has joined #tripleo05:07
StevenKHuh, my seed is currently ISEing for every baremetal request because bm_nodes doesn't exist05:08
*** tserong has joined #tripleo05:10
tchaypoISE?05:10
*** dshulyak_ has joined #tripleo05:11
*** e0ne has quit IRC05:11
*** lazy_prince has quit IRC05:15
*** e0ne has joined #tripleo05:17
greghaynesISE ISE baby05:20
*** e0ne has quit IRC05:20
*** lazy_prince has joined #tripleo05:21
tchaypolo greghaynes05:21
*** e0ne has joined #tripleo05:21
*** nati_ueno has quit IRC05:21
mordredstevebaker: it _should_ default to using get-pip.py - I think if it doesn't it's just for hysterical raisins05:22
*** fandi has quit IRC05:23
*** nati_uen_ has joined #tripleo05:25
greghaynestchaypo: I believe we arent being bitten by that nova-bm bug because its a devtest_testenv thing which is entirely untested by our CI05:28
*** e0ne has quit IRC05:28
*** nati_ue__ has joined #tripleo05:28
tchaypoif it"s in _testenv that would certainly explain it05:28
*** e0ne has joined #tripleo05:28
tchaypoi thoutht it was happening in _seed05:28
xuhaiweican anyone see this bug report? https://bugs.launchpad.net/tripleo/+bug/132316705:29
uvirtbotLaunchpad bug 1323167 in tripleo "overcloud can't create a vm by neutron error" [Undecided,New]05:29
*** nati_ueno has joined #tripleo05:30
greghaynestchaypo: well, its data that were filling out in _testenv05:30
*** nati_uen_ has quit IRC05:30
*** e0ne has quit IRC05:31
greghaynesbut then that data is used in several places05:31
greghaynesso *shrug*05:31
*** shakamunyi has joined #tripleo05:31
tchaypoah05:32
tchaypothat makes sense05:32
tchaypoin CI we don05:32
tchaypot run _testenv, we have a static $TE_DATAFILE05:32
*** nati_ue__ has quit IRC05:33
tchayposo mayhap our $TE_DATAFILE was fixed or never had the problem for some other reason05:33
greghaynesyes... I was curious why we would ever put the ssh key in as that param in the first place05:34
*** shakamunyi_ has joined #tripleo05:37
StevenKHm05:38
StevenKtchaypo, greghaynes: ISE == Internal Server Error05:39
greghaynesyerp05:39
*** shakamunyi has quit IRC05:39
*** alexizz has joined #tripleo05:39
*** shakamunyi_ has quit IRC05:39
StevenKDue to a missing bm_nodes table05:40
StevenKAnd nova-manage db sync doesn't create it :-(05:40
*** nati_ueno has quit IRC05:42
*** nati_ueno has joined #tripleo05:43
*** rdopiera has joined #tripleo05:43
*** giulivo has joined #tripleo05:45
*** nati_uen_ has joined #tripleo05:46
*** giulivo has quit IRC05:46
*** giulivo has joined #tripleo05:46
*** nati_ueno has quit IRC05:47
*** nati_ueno has joined #tripleo05:48
StevenKHuh. Even nova-manage db sync doesn't create it with an empty database05:48
*** nati_uen_ has quit IRC05:50
*** nati_uen_ has joined #tripleo05:53
*** nati_ueno has quit IRC05:53
*** nati_uen_ has quit IRC05:57
*** nati_ueno has joined #tripleo06:06
*** dshulyak_ has quit IRC06:15
*** dshulyak_ has joined #tripleo06:16
*** dshulyak_ has quit IRC06:18
lifelessgreghaynes: whats the testenv issue ?06:24
tchaypolifeless: nova db format changed so that a password field is limited to 255 characters; we've always stuffed an ssh key in it, and that gets rejected now06:25
tchaypohttps://review.openstack.org/#/c/95181/ apparently has a fix06:26
StevenKThe db format didn't change, the db mode did06:26
StevenKBy olso06:26
*** jtomasek has joined #tripleo06:27
*** nati_ueno has quit IRC06:27
*** jtomasek has quit IRC06:27
*** jtomasek has joined #tripleo06:29
*** e0ne has joined #tripleo06:30
tchayposure, that thing.06:33
tchaypo*dies laughing*06:42
tchaypo│OSError: [Errno 2] No such file or directory: '/tmp/pypi/markupsafe/'06:42
tchayponot sure why i hit that, but i think it's because my system has 1.5.6 but virtualenv installs 1.5.406:46
*** nati_ueno has joined #tripleo06:48
*** e0ne has quit IRC06:48
*** e0ne has joined #tripleo06:48
StevenKRight, so rebuilding my seed from scratch fixed bm_nodes. Odd.06:52
*** e0ne has quit IRC06:53
*** rakesh_hs has joined #tripleo06:59
*** e0ne has joined #tripleo07:03
*** mrunge has joined #tripleo07:05
*** jtomasek has quit IRC07:07
*** nati_ueno has quit IRC07:11
*** nati_uen_ has joined #tripleo07:12
*** nati_uen_ has quit IRC07:16
GheRiveromorning all07:17
openstackgerritSteve Kowalik proposed a change to openstack/os-cloud-config: Stop passing pm_password for nova-bm  https://review.openstack.org/9544107:19
*** hashar has joined #tripleo07:22
lifelessStevenK: can you propose the same thing for incubator? https://review.openstack.org/#/c/95181/ is a non-starter (it changes the node definition, which is clearly the wrong layer).07:39
*** rlandy has joined #tripleo07:40
StevenKlifeless: Done.07:44
openstackgerritSteve Kowalik proposed a change to openstack/tripleo-incubator: Stop passing pm_password for nova-bm nodes  https://review.openstack.org/9544507:44
*** jp_at_hp has joined #tripleo07:49
*** fandi has joined #tripleo07:50
*** jprovazn has joined #tripleo07:50
*** e0ne has quit IRC07:59
*** e0ne_ has joined #tripleo07:59
*** e0ne_ has quit IRC08:02
*** e0ne has joined #tripleo08:02
tchayponow that I've used the DIB_REPO* variables to use 95441 i seem to be getting somewhere08:07
tchayposo perhaps I'll be able to test out the neutron patch now08:07
*** jistr has joined #tripleo08:13
*** Matt1 has quit IRC08:17
*** derekh_ has joined #tripleo08:23
*** ramishra has quit IRC08:24
*** ramishra_ has joined #tripleo08:27
*** lucasagomes has joined #tripleo08:33
*** e0ne has quit IRC08:33
*** e0ne has joined #tripleo08:33
marioslol lifeless review 'Something weird going on in this sentence.'08:35
*** lsmola has joined #tripleo08:39
openstackgerritColeman Corrigan proposed a change to openstack/diskimage-builder: Added a DIB_TIDY_IMAGE toggle for image tidy up  https://review.openstack.org/9359408:45
*** jtomasek has joined #tripleo08:48
lifelessmarios: :)08:49
*** vkdrao has joined #tripleo08:51
rlandyderekh_: hi ... wrt https://review.openstack.org/#/c/95026/1/specs/juno/tripleo-ci-improvements.rst.  - I could sign up for tasks that don't require direct access to the environment - probably the tasks closer to end of the list08:52
derekh_rlandy: sounds good, I'll probably have a new version push up later today based on the feedback from lifeless08:53
*** untriaged-bot has joined #tripleo09:00
untriaged-botUntriaged bugs so far:09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132194309:00
uvirtbotLaunchpad bug 1321943 in tripleo "Ceilometer Swift polling on overcloud control node fails with a 403 forbidden error" [Undecided,In progress]09:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/132156309:00
uvirtbotLaunchpad bug 1321563 in diskimage-builder "Add support in ramdisk element to fetch IP from dhcp" [Undecided,New]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/131876709:00
uvirtbotLaunchpad bug 1318767 in tripleo "apache element SSL cert check fails" [Undecided,New]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/131497809:00
uvirtbotLaunchpad bug 1314978 in tripleo "Cloud vm not pingable after overcloud upgrade " [Undecided,New]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/131535509:00
uvirtbotLaunchpad bug 1315355 in tripleo "Upgrade of overcloud failed with "Connection to neutron failed: Maximum attempts reached"" [Undecided,New]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132316709:00
uvirtbotLaunchpad bug 1323167 in tripleo "overcloud can't create a vm by neutron error" [Undecided,New]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/131667509:00
uvirtbotLaunchpad bug 1316675 in tripleo "Saving a devtest VM results in error" [Undecided,In progress]09:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132009009:00
uvirtbotLaunchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New]09:00
*** untriaged-bot has quit IRC09:00
openstackgerritA change was merged to openstack/os-cloud-config: Switch from subprocess to ironicclient  https://review.openstack.org/9396409:01
*** alexizz has quit IRC09:05
openstackgerritA change was merged to openstack/tripleo-image-elements: Create haproxy chroot directory if necessary  https://review.openstack.org/9011009:10
openstackgerritDmitry Shulyak proposed a change to openstack/tripleo-specs: Haproxy configuration options  https://review.openstack.org/9490709:11
*** matsuhashi has quit IRC09:13
openstackgerritSteve Kowalik proposed a change to openstack/tripleo-incubator: Stop setting pm_password for nova-bm nodes  https://review.openstack.org/9544509:13
*** jtomasek has quit IRC09:13
openstackgerritDmitry Shulyak proposed a change to openstack/tripleo-specs: Haproxy configuration options  https://review.openstack.org/9490709:16
*** ramishra_ has quit IRC09:19
*** ramishra has joined #tripleo09:19
*** IvanBerezovskiy has joined #tripleo09:19
*** matsuhashi has joined #tripleo09:20
*** akrivoka has joined #tripleo09:22
openstackgerritDmitry Shulyak proposed a change to openstack/tripleo-specs: Haproxy configuration options  https://review.openstack.org/9490709:25
*** e0ne__ has joined #tripleo09:31
*** matsuhashi has quit IRC09:33
*** e0ne has quit IRC09:35
*** jtomasek has joined #tripleo09:37
*** matsuhas_ has joined #tripleo09:38
*** ccorrigan has joined #tripleo09:40
*** akrivoka has quit IRC09:40
openstackgerritA change was merged to openstack/tripleo-incubator: Unify on _member_ instead of Member and _member_  https://review.openstack.org/9238809:43
*** ramishra has quit IRC09:50
*** ramishra has joined #tripleo09:50
openstackgerritA change was merged to openstack/tripleo-incubator: Clarify the requirements for the TRIPLEO_ROOT variable  https://review.openstack.org/9261109:52
*** jang1 has joined #tripleo09:54
*** ramishra has quit IRC09:55
*** ramishra has joined #tripleo09:57
openstackgerritJon-Paul Sullivan (jp_at_hp) proposed a change to openstack/tripleo-incubator: Revert to correctly determining nodes needed  https://review.openstack.org/9359710:01
*** shakamunyi has joined #tripleo10:01
openstackgerritJon-Paul Sullivan (jp_at_hp) proposed a change to openstack/tripleo-image-elements: Fix rabbit on ubuntu saucy  https://review.openstack.org/9394310:04
openstackgerritJon-Paul Sullivan (jp_at_hp) proposed a change to openstack/tripleo-image-elements: Fix rabbit on ubuntu saucy  https://review.openstack.org/9394310:07
*** mrunge has quit IRC10:11
*** hashar has quit IRC10:15
*** shakamunyi has quit IRC10:15
openstackgerritJon-Paul Sullivan (jp_at_hp) proposed a change to openstack/tripleo-image-elements: Standardise manifest creation and retrieval  https://review.openstack.org/8385010:21
openstackgerritJon-Paul Sullivan (jp_at_hp) proposed a change to openstack/diskimage-builder: Create an element to allow using a dpkg manifest  https://review.openstack.org/9248110:23
*** mrunge has joined #tripleo10:23
*** akrivoka has joined #tripleo10:24
*** SpamapS has quit IRC10:25
*** SpamapS has joined #tripleo10:26
*** SpamapS has quit IRC10:26
*** SpamapS has joined #tripleo10:26
SpamapSo/10:29
SpamapSNg: Hello from LHR Terminal 5 ;)10:30
StevenKSpamapS: I did wonder why you were closer to a sane timezone :-P10:30
SpamapSStevenK: in about 12 hours, I'll be just a few hours west of you10:31
StevenKSpamapS: NZ?10:32
SpamapSStevenK: isn't that east of you?10:32
StevenKCould be10:32
StevenKPerth10:32
SpamapSjust a bit further west10:32
SpamapSBalmy Bangalore10:32
*** shakamunyi has joined #tripleo10:33
StevenKSpamapS: Northwest, West of Perth is Madagascar and friends10:34
SpamapSI wasn't speaking in straight lines, but in timezones.10:34
StevenKHeh, right10:35
StevenKGood old, Twisted default timezone of UTC+53010:35
SpamapSYeah10:35
SpamapSI wonder what the rationale for the 30 is10:35
openstackgerritA change was merged to openstack/tripleo-incubator: Put maximum_object_size before cache_dir  https://review.openstack.org/9377810:36
StevenKSome older time zones were:10:37
StevenK    Bombay time (UTC+04:51),10:37
StevenKNow they're just pulling our leg, surely.10:37
*** andreaf has joined #tripleo10:39
SpamapSIt looks like they just decided they didn't want to change their mean time10:39
StevenKYeah10:40
StevenKIndia got independance from the British and two less timezones10:40
openstackgerritA change was merged to openstack/tripleo-incubator: Add a link to tripleo-newdev-notes  https://review.openstack.org/9294410:40
*** ramishra has quit IRC10:41
*** fandi has quit IRC10:44
*** ramishra has joined #tripleo10:47
SpamapSderekh_: g'morning10:54
SpamapSderekh_: I poked at the HP region a bit this weekend. I think something went horribly wrong, but running os-collect-config --force --one cleared the sweird switching errors10:54
SpamapS21447 packets transmitted, 21396 received, 0% packet loss, time 107344780ms10:55
SpamapSrtt min/avg/max/mdev = 17.496/18.350/62.235/2.650 ms10:55
SpamapSderekh_: so my guess is that the bridges on the controller were half configured or something10:55
*** martyntaylor has joined #tripleo10:55
*** pblaho has joined #tripleo10:56
SpamapSderekh_: or, are you on holiday today too?10:56
*** martyntaylor has left #tripleo10:57
*** jtomasek has quit IRC11:04
*** jtomasek has joined #tripleo11:09
derekh_SpamapS: I'm here, cool, looks like R1 currently isn't being used will see if I can find out why11:09
SpamapSderekh_: I think it might be best to leave it unused..11:09
SpamapSderekh_: and just go through the whole trusty rebuild11:09
derekh_SpamapS: ok, sounds good to me, instead I will see if I can find out why R2 is only running about 10 tests at a time instead of the configured 1811:10
SpamapSnetwork is still really weird to instances actually11:10
SpamapStrying to ssh into one is taking > 3 minutes11:10
derekh_ouch11:10
SpamapSyeah, and its on the trusty compute node11:10
TheJuliaI'm surprised it even connects at 3 minutes11:11
SpamapSTheJulia: well it is taking > 3 ... it may give up soon ;)11:11
SpamapSTheJulia: it has done the initial handshake... just waitin gto hear back from SSH2_MSG_KEXINIT11:11
TheJuliaheh, that happens to me from my house over one of my internet connections to my colo host sometimes when there is bad packet loss11:12
*** ramishra has quit IRC11:12
SpamapSthe gre tunnel seems to have decided to stop working11:12
SpamapSrestarting openvswitch-switch has at least started the tunnel working again11:13
*** boris-42 has quit IRC11:13
*** ramishra_ has joined #tripleo11:16
*** julim has joined #tripleo11:17
*** vkdrao has quit IRC11:18
SpamapSand tunnel is dead again11:19
SpamapShrm11:19
*** julim has quit IRC11:21
*** boris-42 has joined #tripleo11:22
*** jtomasek has quit IRC11:22
*** ramishra_ has quit IRC11:27
*** ramishra has joined #tripleo11:29
SpamapSwow.. neutron is.. almost ridiculous to use11:30
SpamapSconstantly grepping things out and scrolling back11:30
SpamapSnames matter.. <sigh>11:30
*** lucasagomes is now known as lucas-hungry11:31
openstackgerritA change was merged to openstack/tripleo-image-elements: Error out when complete waitcondition fails  https://review.openstack.org/9379911:33
openstackgerritJon-Paul Sullivan (jp_at_hp) proposed a change to openstack/tripleo-image-elements: Fix rabbit on ubuntu saucy  https://review.openstack.org/9394311:36
SpamapSjust seems like the overlay network goes up and down11:38
SpamapSalright.. enough tinkering.. time to head to gate11:39
StevenKSpamapS: Have a safe flight11:40
StevenKTheJulia: And before I forget (again), welcome. :-)11:40
*** andreaf has quit IRC11:42
*** jcoufal has joined #tripleo11:43
*** ifarkas has joined #tripleo11:43
*** jcoufal has quit IRC11:43
TheJuliaStevenK: thank you :)11:44
*** yamahata has quit IRC11:44
*** lazy_prince has quit IRC11:46
openstackgerritA change was merged to openstack/tripleo-incubator: Guard against DIB_COMMON_ELEMENTS not existing  https://review.openstack.org/9302511:46
*** nosnos has quit IRC12:00
*** mrunge has quit IRC12:01
*** dtantsur is now known as dtantsur|lunch12:07
*** matsuhas_ has quit IRC12:08
SpamapSTheJulia: oh right! Welcome!12:16
*** lazy_prince has joined #tripleo12:17
*** rakesh_hs has quit IRC12:21
*** akuznetsov has quit IRC12:27
*** matsuhashi has joined #tripleo12:28
giulivoStevenK, you around? I was looking at your change meant to fix the pm_password thing... thanks for picking up on that12:30
giulivoI think we need something merged to fix CI but at the same time, I realized we probably shouldn't ignore pm_password by default as it is just just the libvirt driver ignoring it, not nova-bm per se12:31
giulivoso in the long term, this looks like a bit more problematic, we should use a proper password when the nova-bm driver isn't libvirt12:31
*** lucas-hungry is now known as lucasagomes12:35
giulivoStevenK, maybe the right thing to do would be to merge your change and open a lower priority but mentioning we should somehow support proper password handling for non-libvit nova-bm drivers?12:37
*** lazy_prince has quit IRC12:37
*** ramishra has quit IRC12:42
*** akuznetsov has joined #tripleo12:48
*** yamahata has joined #tripleo12:56
*** ifarkas has quit IRC12:58
*** zackf has quit IRC13:01
*** dtantsur|lunch is now known as dtantsur13:01
*** jtomasek has joined #tripleo13:03
*** matsuhashi has quit IRC13:06
*** matsuhashi has joined #tripleo13:06
*** shakamunyi has quit IRC13:07
*** matsuhas_ has joined #tripleo13:10
*** matsuhashi has quit IRC13:10
*** jang1 has quit IRC13:14
*** shakamunyi has joined #tripleo13:33
*** ramishra has joined #tripleo13:37
*** shakamunyi has quit IRC13:47
*** rdopiera has quit IRC13:51
*** jtomasek has quit IRC13:52
*** jtomasek has joined #tripleo13:59
*** rpodolyaka has quit IRC13:59
*** rpodolyaka has joined #tripleo13:59
*** viktors has joined #tripleo14:00
*** shakamunyi has joined #tripleo14:04
jp_at_hpgreghaynes: Can you look at https://review.openstack.org/#/c/93597/ please - I believe that I have changed it sufficiently to address any concerns raised.  I also think it was probably a -1 and not a -2, which is why I now have to come looking to get it re-reviewed :(14:05
openstackgerritKiall Mac Innes proposed a change to openstack/tripleo-image-elements: Switch from upstart to os-refresh-config for MySQL server-id's  https://review.openstack.org/9303914:07
*** rpodolyaka has quit IRC14:07
*** rpodolyaka has joined #tripleo14:09
derekh_giulivo: StevenK ya loks like it should only be set to unsed ig using the virtual power driver,14:09
derekh_giulivo: btw, this isn't effecting CI as CI uses pre created test envs.14:10
openstackgerritKiall Mac Innes proposed a change to openstack/tripleo-image-elements: Install lsof during Percona install  https://review.openstack.org/9303114:11
openstackgerritKiall Mac Innes proposed a change to openstack/tripleo-image-elements: Switch from upstart to os-refresh-config for MySQL server-id's  https://review.openstack.org/9303914:11
openstackgerritKiall Mac Innes proposed a change to openstack/tripleo-image-elements: Improve percona cluster initialization  https://review.openstack.org/9303214:11
openstackgerritKiall Mac Innes proposed a change to openstack/tripleo-image-elements: rabbitmq-server depends on os-svc-install  https://review.openstack.org/9305514:13
*** wfoster is now known as wfoster_afk14:18
openstackgerritJon-Paul Sullivan (jp_at_hp) proposed a change to openstack/tripleo-specs: Create reusable image definitions  https://review.openstack.org/9553614:30
*** cody-somerville has joined #tripleo14:33
*** ramishra has quit IRC14:36
*** matsuhas_ has quit IRC14:37
*** wfoster_afk is now known as wfoster14:37
*** hashar has joined #tripleo14:38
*** jp_at_hp has quit IRC14:54
*** jtomasek has quit IRC14:57
*** untriaged-bot has joined #tripleo15:00
untriaged-botUntriaged bugs so far:15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132194315:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/132156315:00
uvirtbotLaunchpad bug 1321943 in tripleo "Ceilometer Swift polling on overcloud control node fails with a 403 forbidden error" [Undecided,In progress]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/131876715:00
uvirtbotLaunchpad bug 1321563 in diskimage-builder "Add support in ramdisk element to fetch IP from dhcp" [Undecided,New]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/131497815:00
uvirtbotLaunchpad bug 1318767 in tripleo "apache element SSL cert check fails" [Undecided,New]15:00
uvirtbotLaunchpad bug 1314978 in tripleo "Cloud vm not pingable after overcloud upgrade " [Undecided,New]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/131535515:00
uvirtbotLaunchpad bug 1315355 in tripleo "Upgrade of overcloud failed with "Connection to neutron failed: Maximum attempts reached"" [Undecided,New]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132316715:00
uvirtbotLaunchpad bug 1323167 in tripleo "overcloud can't create a vm by neutron error" [Undecided,New]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/131947315:00
uvirtbotLaunchpad bug 1319473 in tripleo "devtest_testenv.sh doesn't honour overcloud_computescale or overcloud_controlscale" [Undecided,In progress]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/131667515:00
uvirtbotLaunchpad bug 1316675 in tripleo "Saving a devtest VM results in error" [Undecided,In progress]15:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132009015:00
uvirtbotLaunchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New]15:00
*** untriaged-bot has quit IRC15:00
*** rcarrill` has joined #tripleo15:07
*** rcarrillocruz has quit IRC15:09
*** viktors is now known as viktors|afk15:16
*** shakamunyi has quit IRC15:16
*** pblaho has quit IRC15:18
*** IvanBerezovskiy has left #tripleo15:27
*** zackf has joined #tripleo15:28
greghaynesStevenK: lifeless commented on https://review.openstack.org/#/c/95441/ (hl because seems critical)15:30
*** zackf has quit IRC15:33
openstackgerritColeman Corrigan proposed a change to openstack/os-apply-config: Fix for string key traversal bug #1321827  https://review.openstack.org/9481015:35
uvirtbotLaunchpad bug 1321827 in os-apply-config "os-apply-config fails with  TypeError: string indices must be integers  when traversing strings " [Medium,Triaged] https://launchpad.net/bugs/132182715:35
*** hashar has quit IRC15:36
*** ramishra has joined #tripleo15:37
greghaynesAlso, although https://review.openstack.org/#/c/95181/ breaks ironic I think its a better method than https://review.openstack.org/#/c/95445. It seems like we are overloading the value of that key as being both ssh key or password, when really we should just have to keys15:38
*** ramishra has quit IRC15:41
giulivogreghaynes, I agree with you that we seem to be overloading that key and also the virtual_power driver is actually ignoring all (pm_user and pm_host as well) so this should probably be managed differently depending on the driver too15:44
giulivoon the other hand, looks like we need a pretty urgent fix so ... I think if we add enough comments in the code, it can be restructured later15:44
giulivogreghaynes, I can do some work about that now though if we agree on a "meaningful" approach15:46
greghaynesAgreed, maybe ill make a comment about adding a TODO for that15:48
giulivoonly I wonder if ironic is actually capable of parsing an ssh key from pm_password?15:49
*** e0ne__ has quit IRC15:50
*** e0ne has joined #tripleo15:50
*** BadCub has joined #tripleo15:51
greghaynesIts not, but my thinking is we should correctly set the value at a higher level - set the ssh key property for ironic and not for nova_bm15:52
greghaynesoh wait, as is we do just grap ironic ssh key from pm_password15:53
greghaynesick15:53
giulivoexactly it is quite confusing15:53
giulivoand given ironic doesn't expect a key as "ipmi_password", maybe we should firstly check what is the baremetal driver in use and then15:54
*** e0ne has quit IRC15:55
giulivoif virtual we ignore pm_*; if not we should really expect to find proper values for host,user,password in the json file15:55
GheRiveromutt15:55
giulivocause each physical node can use different credentials15:56
GheRiveroups.. not here. Morning @all!15:56
greghaynescommand mutt not found15:56
BadCubMorning all :-)15:56
greghaynesGheRivero: SpamapS hasnt converted you to notmuch MUA yet?15:59
GheRiverogreghaynes: not yet :)16:01
greghaynesI just switched this week, its pretty nice :)16:01
*** yamahata has quit IRC16:02
*** yamahata has joined #tripleo16:02
clarkbwait I thought you sup16:03
greghayneser, right16:03
greghaynessup16:03
greghaynesI get them confused16:03
*** akuznetsov has quit IRC16:27
*** rlandy has quit IRC16:27
*** lucasagomes is now known as lucas-afk16:36
giulivogreghaynes, I'm lost ... what/where is that we copy the actuall ssh key into the seed VM (mnt/state/var/lib/nova/virtual-power-key) ?16:46
greghaynesgiulivo: tripleo-image-elements16:47
greghaynesnova-ironic element16:48
*** derekh_ has quit IRC16:48
giulivogreghaynes, I meant the nova-bm key ... that seems to be a trick from os-apply-config https://github.com/openstack/tripleo-image-elements/blob/master/elements/nova-baremetal/os-apply-config/mnt/state/var/lib/nova/virtual-power-key16:51
*** BadCub has quit IRC17:00
*** akrivoka has quit IRC17:00
greghaynesright, I meant nova-bm ;)17:01
giulivothis key thing is unexpectedly hard :P17:04
giulivobut up to that point we'd be doing well17:06
*** jistr has quit IRC17:13
*** rcarrillocruz has joined #tripleo17:24
*** rcarrill` has quit IRC17:26
*** ramishra has joined #tripleo17:30
NobodyCamgood morning / afternoon TripleO just a dumb question: has DIB_REPOLOCATION_ and / or  DIB_REPOREF_ changed the way they are used? I'm getting "fatal: Couldn't find remote ref refs/changes/25/92625/7" but fetching seems to work from command line17:32
*** ramishra has quit IRC17:34
greghaynesI recall a patch saying we broke ref based cloning17:37
greghaynesIt would have been in diskimage-builder17:37
*** jang1 has joined #tripleo17:38
NobodyCamgreghaynes: patch or bug?17:39
NobodyCamgreghaynes: I see https://bugs.launchpad.net/tripleo/+bug/1301435 but I am not doing -c or --offline17:41
greghaynespatch - it said it fixed it.17:41
uvirtbotLaunchpad bug 1301435 in tripleo "devtest -c with --offline and specific DIB_REPOREF_<project>'s fails if cache is older than refs" [Medium,Confirmed]17:41
*** jprovazn has quit IRC17:41
greghayneshrm17:42
greghaynesOn phone right now so my grep abilities are limited :/17:42
NobodyCamgreghaynes: I could be out of date let me update everyting17:42
*** dshulyak_ has joined #tripleo17:42
*** akuznetsov has joined #tripleo17:54
*** jml has quit IRC18:03
NobodyCamgreghaynes: fyi: I was a few commits behind on DIB. updateed and pulled only to encounter the same error. :-p18:04
*** e0ne has joined #tripleo18:08
*** jml has joined #tripleo18:08
*** akuznetsov has quit IRC18:11
*** sballe has quit IRC18:29
*** sballe has joined #tripleo18:30
NobodyCamhas anyone been using DIB_REPOLOCATION_ and /  DIB_REPOREF_ ... just trying to see if its y env or something else?18:31
openstackgerritA change was merged to openstack/tuskar: Don't display passwords when listing overclouds  https://review.openstack.org/9464818:33
NobodyCams/y/my/ :-p18:33
lifelessNobodyCam: what error ?18:45
lifelessNobodyCam: https://review.openstack.org/#/c/90890/ may be missing from your dib18:45
lifelessNobodyCam: it fixes a regression, failed last nighton the ironic virtual job18:45
*** e0ne has quit IRC18:45
*** e0ne has joined #tripleo18:46
NobodyCamlifeless: yep would seem I am missing that patch :)18:49
*** e0ne has quit IRC18:49
*** hashar has joined #tripleo18:50
*** lucas-afk is now known as lucasagomes18:55
*** jang has quit IRC19:03
*** ddieterly has joined #tripleo19:04
*** hashar has quit IRC19:06
*** hashar has joined #tripleo19:06
*** ddieterl_ has joined #tripleo19:08
*** ddieterly has quit IRC19:10
*** rcarrillocruz has quit IRC19:12
*** jang has joined #tripleo19:16
*** rcarrillocruz has joined #tripleo19:18
NobodyCamfyi lifeless that patch seems to be working for me. I will +1 if it in fact dose :)20:01
*** lucasagomes has left #tripleo20:01
*** dtantsur is now known as dtantsur|afk20:03
*** e0ne has joined #tripleo20:04
*** e0ne has quit IRC20:05
*** ddieterl_ has quit IRC20:05
*** ddieterly has joined #tripleo20:06
*** ddieterly has quit IRC20:07
*** e0ne has joined #tripleo20:26
*** e0ne has quit IRC20:32
*** Hefeweiz1n has quit IRC20:35
*** jang1 has quit IRC20:38
*** yamahata has quit IRC20:58
*** yamahata has joined #tripleo20:58
*** untriaged-bot has joined #tripleo21:00
untriaged-botUntriaged bugs so far:21:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132194321:00
untriaged-bothttps://bugs.launchpad.net/diskimage-builder/+bug/132156321:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/131876721:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/131497821:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/131535521:00
uvirtbotLaunchpad bug 1321943 in tripleo "Ceilometer Swift polling on overcloud control node fails with a 403 forbidden error" [Undecided,In progress]21:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132316721:00
uvirtbotLaunchpad bug 1321563 in diskimage-builder "Add support in ramdisk element to fetch IP from dhcp" [Undecided,New]21:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/131947321:00
uvirtbotLaunchpad bug 1318767 in tripleo "apache element SSL cert check fails" [Undecided,New]21:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/131667521:00
uvirtbotLaunchpad bug 1314978 in tripleo "Cloud vm not pingable after overcloud upgrade " [Undecided,New]21:00
uvirtbotLaunchpad bug 1315355 in tripleo "Upgrade of overcloud failed with "Connection to neutron failed: Maximum attempts reached"" [Undecided,New]21:00
untriaged-bothttps://bugs.launchpad.net/tripleo/+bug/132009021:00
uvirtbotLaunchpad bug 1323167 in tripleo "overcloud can't create a vm by neutron error" [Undecided,New]21:00
*** untriaged-bot has quit IRC21:00
uvirtbotLaunchpad bug 1319473 in tripleo "devtest_testenv.sh doesn't honour overcloud_computescale or overcloud_controlscale" [Undecided,In progress]21:00
uvirtbotLaunchpad bug 1316675 in tripleo "Saving a devtest VM results in error" [Undecided,In progress]21:00
uvirtbotLaunchpad bug 1320090 in tripleo "pxe_ephemeral_format': u'ext4 left on nodes after deploys deleted" [Undecided,New]21:00
tchaypogood morning party people!21:05
tchaypoand also greghaynes21:05
lifelesso/21:07
*** aboutGod has joined #tripleo21:10
*** aboutGod has left #tripleo21:15
tchaypoStevenK: now I'm at the stage of getting the ISEs too, yay~21:16
*** giulivo has quit IRC21:19
*** hashar has quit IRC21:19
tchaypoin happier news, the daily batch job has noticed that I'm now velocity gold21:20
mordredtchaypo: what's a velocity?21:22
tchaypovirgin australia's loyalty program21:22
tchaypoor in other contexts, it's a vector comprised of a speed plus a direction21:22
mordredtchaypo: ah. well, I now appreciate and understand both definitions21:25
mordredtchaypo: congrats on the gold status in the loyalty program - I believe that's probably the more relevant thing here21:26
tchaypooh goody. my undercloud has 127.0.0.1 as the only server in /etc/resolv.conf21:31
tchaypoand dnsmasq is configured to get its list of upstream servers from resolv.conf21:31
tchayposo that's working well for me21:31
lifelessStevenK: when you're about, I'd like to point you at eliminating the manual ssh key file from ironic in favour of passing the key value to ironic21:32
openstackgerritAdam Gandelman proposed a change to openstack/diskimage-builder: Add new cloud-init-datasources element  https://review.openstack.org/9559821:42
*** dshulyak_ has quit IRC21:42
*** d0ugal_ has joined #tripleo21:56
lifelessgreghaynes: if you are about https://review.openstack.org/#/c/90890/22:07
openstackgerritAdam Gandelman proposed a change to openstack/diskimage-builder: Add new cloud-init-datasources element  https://review.openstack.org/9559822:27
-openstackstatus- NOTICE: stopping gerrit briefly to rebuild its search index in an attempt to fix post-rename oddities (will update with notices every 10 minutes until completed)22:31
*** ChanServ changes topic to "stopping gerrit briefly to rebuild its search index in an attempt to fix post-rename oddities (will update with notices every 10 minutes until completed)"22:31
*** ChanServ changes topic to "ci-overcloud for HP region is down due to network issues | https://etherpad.openstack.org/p/tripleo-ci-r1-trusty | tripleo-cd running preserve-ephemeral WIP patches and https://review.openstack.org/#/c/62042/ | Using OpenStack to deploy OpenStack;meetings Tuesday 1900 UTC in #openstack-meeting-alt"22:36
*** yamahata has quit IRC22:45
openstackgerritKiall Mac Innes proposed a change to openstack/tripleo-image-elements: Switch from upstart to os-refresh-config for MySQL server-id's  https://review.openstack.org/9303922:58
openstackgerritKiall Mac Innes proposed a change to openstack/tripleo-image-elements: Improve percona cluster initialization  https://review.openstack.org/9303222:58
StevenKgreghaynes: The os-cloud-config branch isn't critical, but it is needed. The -incubator change is, however.22:58
lifelessStevenK: I just reviewed it23:01
StevenKlifeless: Right, so for -incubator just cut -b1-255 on pm_password ?23:06
lifelessyeah23:15
openstackgerritJames Polley proposed a change to openstack/tripleo-incubator: Restore formatting of _undercloud instructions  https://review.openstack.org/9560523:16
tchaypoStevenK: did you figure out the ISEs perchance?23:17
openstackgerritSteve Kowalik proposed a change to openstack/tripleo-incubator: Clamp pm_password to 255 for nova-bm nodes  https://review.openstack.org/9544523:19
StevenKtchaypo: Nope. I rebuilt my seed which stopped the ISEs23:19
tchaypowheee23:20
tchaypookay, while that's happening I'll make another stab at getting the hpcs vpn up and running so i can try to be useful there.23:29
*** noslzzp has quit IRC23:42
xuhaiweimorning23:46

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!