| *** kota_ has joined #openstack-swift | 00:02 | |
| *** ChanServ sets mode: +v kota_ | 00:02 | |
| kota_ | hello | 00:02 |
|---|---|---|
| mattoliverau | kota_: morning | 00:03 |
| kota_ | mattoliverau: morning | 00:03 |
| notmyname | hmm...looks like http://www.ramenshow.com/ is at the same time as the summit | 00:09 |
| kota_ | Ramen Festival? | 00:10 |
| notmyname | http://www.gotokyo.org/en/index.html seems to have some good info | 00:16 |
| notmyname | well, it has info. I don't know if it's good | 00:17 |
| kota_ | I see. | 00:18 |
| openstackgerrit | Matthew Oliver proposed openstack/swift: Fix memcached exception out of range stacktrace https://review.openstack.org/219963 | 00:20 |
| mattoliverau | ^ pesky pep8 :P | 00:20 |
| kota_ | IMO, http://www.ramenshow.com/ seems for expert travelers because (maybe) no English direction in the festival. | 00:21 |
| notmyname | kota_: you forgot the part where I'm not letting you out of my sight for the week ;-) | 00:22 |
| notmyname | you and ho | 00:22 |
| kota_ | that's perfect, no problem to go there :P | 00:23 |
| notmyname | lol | 00:23 |
| kota_ | notmyname: so quick question, in tokyo | 00:24 |
| redbo | I'm hoping that pointing at food and handing random amounts of money to people works. | 00:24 |
| clayg | redbo: that's the universal laungage | 00:25 |
| clayg | pointing and money will get you all the things you want in life | 00:25 |
| kota_ | notmyname: are your interested in the dinner cruise called 'yakatabune'? | 00:25 |
| notmyname | kota_: yes, i probably am. what is it? | 00:25 |
| *** darrenc is now known as darrenc_afk | 00:25 | |
| notmyname | https://en.wikipedia.org/wiki/Yakatabune | 00:25 |
| kota_ | googling english direction, right now. | 00:25 |
| kota_ | that is | 00:26 |
| kota_ | more phots, https://www.google.co.jp/search?q=yakatabune&rlz=1C1SNJC_jaJP478JP478&espv=2&biw=1132&bih=734&source=lnms&tbm=isch&sa=X&ved=0CAYQ_AUoAWoVChMIlI7bgI_cxwIVyaiUCh0POwMF | 00:27 |
| notmyname | I'm pretty sure I'll be happy with most things ;-) | 00:27 |
| notmyname | kota_: if you're talking about something you're planning for swift people with ntt, that looks pretty nice | 00:28 |
| kota_ | notmyname: thanks, that's what I am considering. | 00:29 |
| notmyname | http://www.nissan.co.jp/GALLERY/HQ/ looks nice too | 00:30 |
| kota_ | Maybe we can ride on the boat from closer place from summit conference hotel. | 00:30 |
| kota_ | I know the gallery. | 00:31 |
| *** m_kazuhiro has joined #openstack-swift | 00:32 | |
| kota_ | the gallery is crowed with students in vacation season but not sure around summit week. | 00:32 |
| kota_ | but exactly populer place, I guess. | 00:33 |
| kota_ | notmyname: if you are interested in Japanese historical museum, my recomendation is Edo tokyo museum. | 00:35 |
| kota_ | here, https://www.edo-tokyo-museum.or.jp/en/ | 00:35 |
| kota_ | It's the largest museum in Japan to explain old tokyo, 'Edo'. | 00:36 |
| kota_ | It's not technological :/ | 00:37 |
| notmyname | cool thanks | 00:42 |
| * notmyname packs up, bikes home | 00:43 | |
| *** darrenc_afk is now known as darrenc | 00:44 | |
| *** zaitcev has quit IRC | 00:48 | |
| *** lyrrad has quit IRC | 00:51 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Don't send commits for quorum *BAD* requests on EC https://review.openstack.org/220059 | 01:11 |
| *** jkugel has joined #openstack-swift | 01:23 | |
| *** jrichli has joined #openstack-swift | 01:43 | |
| openstackgerrit | Merged openstack/python-swiftclient: Log and report trace on service operation fails https://review.openstack.org/171692 | 01:46 |
| *** annegentle has quit IRC | 02:01 | |
| *** annegentle has joined #openstack-swift | 02:23 | |
| mattoliverau | notmyname: just realized my LCA paper was accepted too.. Looks like I better get sharding working and learn to speak to real life peoples :p | 02:39 |
| *** darrenc is now known as darrenc_afk | 02:40 | |
| *** petertr7 is now known as petertr7_away | 02:47 | |
| *** ho has quit IRC | 02:51 | |
| *** darrenc_afk is now known as darrenc | 02:58 | |
| *** ho has joined #openstack-swift | 02:58 | |
| notmyname | mattoliverau: that's great!! | 03:00 |
| *** kairo_ has joined #openstack-swift | 03:08 | |
| *** kairo has quit IRC | 03:09 | |
| *** openstackgerrit_ has joined #openstack-swift | 03:11 | |
| *** proteusguy_ has quit IRC | 03:17 | |
| *** jroll has quit IRC | 03:28 | |
| *** tsubic has quit IRC | 03:28 | |
| *** tsubic has joined #openstack-swift | 03:28 | |
| *** xnox has quit IRC | 03:28 | |
| *** flwang1 has quit IRC | 03:28 | |
| *** sudorandom has quit IRC | 03:28 | |
| *** proteusguy_ has joined #openstack-swift | 03:30 | |
| *** sudorandom has joined #openstack-swift | 03:33 | |
| *** xnox has joined #openstack-swift | 03:34 | |
| *** jroll has joined #openstack-swift | 03:34 | |
| *** flwang1 has joined #openstack-swift | 03:34 | |
| *** sanchitmalhotra has joined #openstack-swift | 03:37 | |
| *** kota_ has quit IRC | 03:50 | |
| *** wbhuber has quit IRC | 03:59 | |
| *** kei_yama has quit IRC | 04:04 | |
| *** km has quit IRC | 04:04 | |
| *** links has joined #openstack-swift | 04:12 | |
| *** openstackgerrit_ has quit IRC | 04:14 | |
| *** kei_yama has joined #openstack-swift | 04:14 | |
| *** kei_yama has quit IRC | 04:16 | |
| *** ppai has joined #openstack-swift | 04:16 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-swiftclient: Updated from global requirements https://review.openstack.org/89250 | 04:19 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 04:19 |
| *** kei_yama has joined #openstack-swift | 04:20 | |
| *** trifon has joined #openstack-swift | 04:24 | |
| openstackgerrit | Tim Burke proposed openstack/swift: Add the ability to specify ranges for SLO segments https://review.openstack.org/211726 | 04:30 |
| *** trifon has quit IRC | 04:30 | |
| *** rjaiswal has joined #openstack-swift | 04:33 | |
| *** trifon has joined #openstack-swift | 04:37 | |
| *** trifon has quit IRC | 04:42 | |
| jrichli | congrats, mattoliverau! You will do great. Ok, I am off to bed. | 04:46 |
| jrichli | debugging can be exhausing. especially when values from old requests somehow make it into the response headers of a new wsgiContext :-( | 04:46 |
| mattoliverau | jrichli: thanks, you should come to LCA ;) | 04:47 |
| mattoliverau | jrichli: and night :) | 04:47 |
| jrichli | I am just happy they are paying my way to Tokyo :-) | 04:47 |
| mattoliverau | jrichli: good point :) | 04:47 |
| *** prometheanfire has left #openstack-swift | 04:47 | |
| *** jrichli has quit IRC | 04:49 | |
| *** annegentle has quit IRC | 04:56 | |
| *** haomaiwa_ has quit IRC | 04:57 | |
| *** bkopilov has quit IRC | 05:19 | |
| *** _hrou_ has quit IRC | 05:21 | |
| *** m_kazuhiro has quit IRC | 05:21 | |
| *** trifon has joined #openstack-swift | 05:26 | |
| *** bkopilov has joined #openstack-swift | 05:30 | |
| *** zhill has joined #openstack-swift | 05:33 | |
| *** jamielennox is now known as jamielennox|away | 05:35 | |
| *** m_kazuhiro has joined #openstack-swift | 05:50 | |
| *** jamielennox|away is now known as jamielennox | 05:58 | |
| *** zhill has quit IRC | 05:59 | |
| *** dimasot has joined #openstack-swift | 05:59 | |
| *** kei_yama has quit IRC | 06:00 | |
| *** ttrumm has joined #openstack-swift | 06:09 | |
| *** SkyRocknRoll has joined #openstack-swift | 06:10 | |
| *** ttrumm_ has joined #openstack-swift | 06:11 | |
| *** ttrumm has quit IRC | 06:13 | |
| *** xnox has quit IRC | 06:23 | |
| *** xnox has joined #openstack-swift | 06:25 | |
| *** ttrumm_ has quit IRC | 06:30 | |
| *** rjaiswal has quit IRC | 06:50 | |
| *** kota_ has joined #openstack-swift | 06:54 | |
| *** ChanServ sets mode: +v kota_ | 06:54 | |
| ho | mattoliverau: you wil speak at LCA! that's great!!! | 06:57 |
| mattoliverau | ho: yeah, you should come :) | 06:57 |
| ho | mattoliverau: :-) | 06:58 |
| *** jamielennox is now known as jamielennox|away | 07:08 | |
| *** rledisez has joined #openstack-swift | 07:10 | |
| *** kota_ has quit IRC | 07:29 | |
| *** jamielennox|away is now known as jamielennox | 07:32 | |
| *** geaaru has joined #openstack-swift | 07:40 | |
| *** sanchitmalhotra has quit IRC | 08:28 | |
| *** jistr has joined #openstack-swift | 08:44 | |
| *** dimasot has quit IRC | 08:55 | |
| *** mahatic has joined #openstack-swift | 08:55 | |
| *** sanchitmalhotra has joined #openstack-swift | 08:56 | |
| *** joeljwright has joined #openstack-swift | 09:08 | |
| *** ChanServ sets mode: +v joeljwright | 09:08 | |
| *** sanchitmalhotra1 has joined #openstack-swift | 09:11 | |
| *** SkyRocknRoll_ has joined #openstack-swift | 09:12 | |
| *** sanchitmalhotra has quit IRC | 09:14 | |
| *** lazy_prince has joined #openstack-swift | 09:24 | |
| lazy_prince | Hi all.. looks like Ironic gates are broken due to swift in devstack.. | 09:24 |
| lazy_prince | is anyone looking at it..? | 09:25 |
| *** tsubic has quit IRC | 09:30 | |
| *** nadeem has joined #openstack-swift | 09:31 | |
| *** nadeem has quit IRC | 09:36 | |
| *** SkyRocknRoll_ has quit IRC | 09:43 | |
| *** marzif has joined #openstack-swift | 09:43 | |
| cschwede | lazy_prince: do you have a link to a log or failed test? | 09:48 |
| cschwede | lazy_prince: ah, this one? http://logs.openstack.org/68/217068/14/check/gate-tempest-dsvm-ironic-agent_ssh/18d8590/logs/devstacklog.txt.gz#_2015-09-04_09_04_55_994 | 09:48 |
| lazy_prince | You can start with http://logs.openstack.org/44/206244/19/check/gate-tempest-dsvm-ironic-agent_ssh/3d4d287/ | 09:49 |
| lazy_prince | yeah.. that is another one.. | 09:49 |
| cschwede | looking into it | 09:50 |
| *** marzif has quit IRC | 10:08 | |
| openstackgerrit | Victor Stinner proposed openstack/swift: py3: Fix Python 3 issues in utils https://review.openstack.org/217426 | 10:14 |
| openstackgerrit | Victor Stinner proposed openstack/swift: Disable temporary the rfc822 import on Python 3 https://review.openstack.org/220444 | 10:14 |
| *** haypo has joined #openstack-swift | 10:16 | |
| cschwede | lazy_prince: i don’t think the error is related to a problem in swift. in fact, there is not even a POST request in the swift proxy server log: http://logs.openstack.org/68/217068/14/check/gate-tempest-dsvm-ironic-agent_ssh/18d8590/logs/screen-s-proxy.txt.gz | 10:18 |
| cschwede | lazy_prince: that might be an issue with keystone then | 10:19 |
| cschwede | lazy_prince: and looking at the last line in http://logs.openstack.org/68/217068/14/check/gate-tempest-dsvm-ironic-agent_ssh/18d8590/logs/apache/keystone_access.txt.gz | 10:21 |
| cschwede | that should probably be „/v2.0/tokens“, not „/tokens“ | 10:21 |
| lazy_prince | cschwede: okay.. can you follow up with the keystone team then..? | 10:27 |
| openstackgerrit | Merged openstack/swift: Better scoping for tempurls, especially container tempurls https://review.openstack.org/217255 | 10:32 |
| *** tsubic has joined #openstack-swift | 10:38 | |
| cschwede | lazy_prince: i think it’s an issue in keystone itself. not sure why this pops up now, but the os_auth_url set in devstack is wrong | 10:39 |
| cschwede | lazy_prince: it’s set here: http://logs.openstack.org/68/217068/14/check/gate-tempest-dsvm-ironic-agent_ssh/18d8590/logs/devstacklog.txt.gz#_2015-09-04_09_03_45_104 | 10:39 |
| cschwede | note the missing „/v2.0“ at the end of OS_AUTH_URL | 10:40 |
| cschwede | that results in exactly the same error that is shown in the swift log. i can reproduce this in my local devstack | 10:40 |
| cschwede | compare that too http://logs.openstack.org/03/220003/2/gate/gate-tempest-dsvm-ironic-agent_ssh/a23d537/logs/devstacklog.txt.gz#_2015-09-03_21_00_16_153 | 10:42 |
| cschwede | lazy_prince: i found the reason | 10:47 |
| cschwede | creating a bug report | 10:47 |
| *** acoles_ is now known as acoles | 10:48 | |
| *** links has quit IRC | 10:49 | |
| *** joeljwright has quit IRC | 10:50 | |
| *** joeljwright has joined #openstack-swift | 10:51 | |
| *** ChanServ sets mode: +v joeljwright | 10:51 | |
| acoles | mattoliverau: did you silence your phone tonight? :P | 10:54 |
| *** SkyRocknRoll_ has joined #openstack-swift | 10:57 | |
| lazy_prince | cschwede: please share the bug id once you have it.. | 10:57 |
| *** trifon has quit IRC | 10:59 | |
| cschwede | lazy_prince: https://bugs.launchpad.net/devstack/+bug/1492216 | 11:00 |
| openstack | Launchpad bug 1492216 in devstack "gate-tempest-dsvm-ironic-agent_ssh fails with a Swift error" [Undecided,New] - Assigned to Christian Schwede (cschwede) | 11:00 |
| cschwede | lazy_prince: fix for devstack: https://review.openstack.org/220458 | 11:11 |
| lazy_prince | so it was a devstack fix for swift afterall.. | 11:15 |
| *** thurloat_isgone is now known as thurloat | 11:25 | |
| cschwede | lazy_prince: no, for python-keystoneclient | 11:27 |
| cschwede | because that is the part that failed | 11:27 |
| *** pbourke has quit IRC | 11:37 | |
| *** pbourke has joined #openstack-swift | 11:37 | |
| mattoliverau | acoles: yes, yes I did ;) | 11:41 |
| *** sanchitmalhotra1 has quit IRC | 11:41 | |
| *** xnox has quit IRC | 11:42 | |
| acoles | mattoliverau: heh. i just booked tokyo flights :) | 11:42 |
| mattoliverau | acoles: \o/, Tokyo will be fun! cschwede you going? | 11:43 |
| acoles | mattoliverau: cschwede has to come to collect all the bbq meat you saved for him in austin! | 11:44 |
| mattoliverau | Lol, yeah that's right. Nothing ages as well as Texas BBQ meat :p | 11:45 |
| *** xnox has joined #openstack-swift | 11:45 | |
| cschwede | mattoliverau: acoles: i won’t eat the week before Tokyo ;) Yep, I’ll be there, hotel & flights are booked :D | 11:54 |
| mattoliverau | \o/ | 11:54 |
| mattoliverau | Mine aren't, but flights from oz isn't too bad, should poke the powers that be tho | 11:55 |
| *** delattec has quit IRC | 12:00 | |
| *** cdelatte has quit IRC | 12:00 | |
| *** cdelatte has joined #openstack-swift | 12:15 | |
| *** delattec has joined #openstack-swift | 12:15 | |
| *** kairo_ has quit IRC | 12:19 | |
| *** links has joined #openstack-swift | 12:21 | |
| *** SkyRocknRoll has quit IRC | 12:24 | |
| *** trifon has joined #openstack-swift | 12:32 | |
| *** jkugel has quit IRC | 12:34 | |
| *** m_kazuhiro has quit IRC | 12:39 | |
| *** delattec has quit IRC | 12:41 | |
| *** cdelatte has quit IRC | 12:41 | |
| *** cdelatte has joined #openstack-swift | 12:43 | |
| *** silor has joined #openstack-swift | 12:44 | |
| *** annegentle has joined #openstack-swift | 12:46 | |
| *** lazy_prince has quit IRC | 12:57 | |
| *** dustins has joined #openstack-swift | 12:57 | |
| *** NM has joined #openstack-swift | 12:58 | |
| *** proteusguy__ has joined #openstack-swift | 13:01 | |
| *** proteusguy_ has quit IRC | 13:03 | |
| *** mahatic has quit IRC | 13:09 | |
| *** mahatic has joined #openstack-swift | 13:09 | |
| *** jkugel has joined #openstack-swift | 13:11 | |
| *** petertr7_away is now known as petertr7 | 13:12 | |
| *** esker has quit IRC | 13:15 | |
| *** esker has joined #openstack-swift | 13:15 | |
| *** _hrou_ has joined #openstack-swift | 13:16 | |
| *** janonymous_ has joined #openstack-swift | 13:16 | |
| *** esker has quit IRC | 13:20 | |
| *** haomaiwang has joined #openstack-swift | 13:21 | |
| *** vinsh_ is now known as Vinsh | 13:24 | |
| *** links has quit IRC | 13:27 | |
| *** ppai has quit IRC | 13:31 | |
| *** thumpba has joined #openstack-swift | 13:44 | |
| *** jbonjean has joined #openstack-swift | 13:44 | |
| *** jbonjean has quit IRC | 13:44 | |
| *** jbonjean has joined #openstack-swift | 13:45 | |
| *** jbonjean has quit IRC | 13:45 | |
| *** jbonjean has joined #openstack-swift | 13:45 | |
| *** jbonjean has quit IRC | 13:45 | |
| *** jbonjean has joined #openstack-swift | 13:46 | |
| *** mahatic has quit IRC | 13:55 | |
| *** esker has joined #openstack-swift | 13:56 | |
| *** esker has quit IRC | 13:56 | |
| *** ctrath has joined #openstack-swift | 13:56 | |
| *** esker has joined #openstack-swift | 13:56 | |
| *** haomaiwang has quit IRC | 14:01 | |
| *** haomaiwang has joined #openstack-swift | 14:01 | |
| *** wbhuber has joined #openstack-swift | 14:11 | |
| *** aix has quit IRC | 14:15 | |
| *** mahatic has joined #openstack-swift | 14:21 | |
| *** aix has joined #openstack-swift | 14:23 | |
| *** jlhinson has joined #openstack-swift | 14:26 | |
| notmyname | good morning | 14:34 |
| *** NM has quit IRC | 14:34 | |
| peluse | morning | 14:37 |
| notmyname | looks like we have 2 fishbowls, 12 workrooms, and a full "meetup" day in tokyo | 14:38 |
| mahatic | good morning | 14:39 |
| janonymous_ | I might get to meet "The swift team " :) | 14:41 |
| notmyname | you've already met us. now you get to put faces with names ;-) | 14:41 |
| janonymous_ | Haha.. ryt :) | 14:45 |
| *** annegentle has quit IRC | 14:45 | |
| notmyname | today I'm going to look at any last things for python-swiftclient. we'll be doing a release there very soon /cc joeljwright timburke | 14:45 |
| *** annegentle has joined #openstack-swift | 14:46 | |
| joeljwright | notmyname: I had a look this morning and couldn't see anything pressing | 14:46 |
| notmyname | joeljwright: thanks | 14:46 |
| wbhuber | peluse: ctennis: can you check somethinig for me on your cluster.... what version of socket.py are you using in the cluster - usr/local/lib/python2.7/dist-packages/eventlet-0.17.3-py2.7.egg/eventlet/green/socket.py? it might tell us why we are getting traceback dump | 14:46 |
| joeljwright | the keystone toekn auth patch has a negative review, so it's unlikely to be ready in time | 14:46 |
| notmyname | cschwede: what's going on with ironic/devstack? | 14:46 |
| cschwede | notmyname: morning! seems to get fixed right now: https://review.openstack.org/#/c/220532/ | 14:47 |
| cschwede | notmyname: i proposed a different fix, but that wasn’t accepted - don’t know why: https://review.openstack.org/#/c/220458/ | 14:48 |
| *** jrichli has joined #openstack-swift | 14:48 | |
| notmyname | cschwede: is it something that affects swift? | 14:48 |
| cschwede | notmyname: whatever, there was a patch to devstack that broke the keystone auth, and the failure was only appearing within ironic tests and related to swift tempurl. it looked like a swift error, but it isn’t | 14:49 |
| notmyname | ok | 14:49 |
| joeljwright | notmyname: I also wanted to ask about patch 217255 'Better scoping for tempurls, especially container tempurls' | 14:50 |
| patchbot | joeljwright: https://review.openstack.org/#/c/217255/ | 14:50 |
| notmyname | joeljwright: looks like it finally merged | 14:50 |
| notmyname | joeljwright: what about it? | 14:50 |
| *** kevinc_ has joined #openstack-swift | 14:50 | |
| wbhuber | peluse: ctennis: this is wrt to LP 1488610, evidently, the most current socket.py has a check point to see if err has been used or not before raising it. | 14:50 |
| openstack | Launchpad bug 1488610 in OpenStack Object Storage (swift) "EC connection refused exception when storage node is offline" [Undecided,Confirmed] https://launchpad.net/bugs/1488610 - Assigned to Bill Huber (wbhuber) | 14:50 |
| joeljwright | I understand the reason for the patch, but it will break the default behaviour of segmented objects and python-swiftclient | 14:50 |
| notmyname | joeljwright: ie that DLOs are no longer allowed for tempurl PUTs? | 14:51 |
| joeljwright | well segments on upload go into a different container by default | 14:51 |
| joeljwright | so gets will fail for any large object uploaded with the swiftclient | 14:51 |
| joeljwright | it does apply to GETs too? | 14:51 |
| joeljwright | or am I reading the description of the patch wrong? | 14:53 |
| notmyname | for container-level tempurls, yes. | 14:53 |
| notmyname | ...checking something locally.. | 14:53 |
| acoles | notmyname: when are you planning to do the swiftclient release? | 14:55 |
| *** j_king has joined #openstack-swift | 14:56 | |
| notmyname | acoles: Real Soon Now | 14:56 |
| notmyname | acoles: ie no set time, but I'd like to have it tagged or ready to tag by my monday morning | 14:57 |
| notmyname | but if it's ready now, then now | 14:57 |
| acoles | notmyname: cool. i would -2 https://review.openstack.org/172791 but don't want to stand in way of it merging IF it gets fixed in time for release | 14:58 |
| acoles | joeljwright: ^^ | 14:59 |
| *** minwoob has joined #openstack-swift | 14:59 | |
| notmyname | joeljwright: container-level tempurl is denies access (because the segments are outside of the contianer where access was granted. account-level tempurl still works (both for GETs) | 14:59 |
| acoles | hence i only put -1 | 14:59 |
| *** janonymous_ has quit IRC | 14:59 | |
| joeljwright | acoles: I will keep an eye on it | 15:00 |
| joeljwright | notmyname: that seems a shame because it makes container-level tempurls very inconvenient for *LO | 15:00 |
| peluse | wbhuber, I cant right now but I will check later today sometime | 15:01 |
| *** haomaiwang has quit IRC | 15:01 | |
| wbhuber | peluse: np | 15:01 |
| MooingLemur | notmyname: doing some straces, I found the main problem causing stalls was that I needed to lower the mb_per_sync. I also disabled fallocate because sometimes (most of the time) that was taking ~10s to return on large files. | 15:01 |
| notmyname | acoles: ah, ok. so you really really don't want to see that land as-is, but if it does get fixed when you aren't online, you don't want to stand in the way | 15:01 |
| *** haomaiwang has joined #openstack-swift | 15:01 | |
| joeljwright | notmyname: have to put all the segments in the single container, but then I can't just download the container without data doubling on download | 15:01 |
| joeljwright | :( | 15:01 |
| MooingLemur | Not sure when it became a problem, but it's either because the disks are fuller, busier, or most likely both. | 15:01 |
| acoles | notmyname: correct | 15:02 |
| joeljwright | nomyname: it might be nice if we could explicitly allow some segment containers for container-level tempurls | 15:02 |
| MooingLemur | but I'm still running into issues where the open() syscall on DELETEs is occasionally taking a very long time, which stalls the entire worker. | 15:02 |
| notmyname | MooingLemur: interesting. so it's all about the fsync to disk. | 15:03 |
| *** jbonjean has quit IRC | 15:03 | |
| *** jbonjean has joined #openstack-swift | 15:03 | |
| *** jbonjean has quit IRC | 15:03 | |
| *** jbonjean has joined #openstack-swift | 15:03 | |
| joeljwright | notmyname: although I haven't thought that through and it might have the same security issue :( | 15:03 |
| notmyname | joeljwright: unfortunately, the way tempurls work is that you can HEAD anything you have PUT access to. and container tempurl keys are intended to be handed out to someone who has access to that container but not the rest of the account | 15:04 |
| notmyname | joeljwright: so if DLOs are allowed via container tempurls, then the holder of the container-level key can probe the entire rest of the account to see what's there (an info leak) | 15:05 |
| joeljwright | notmyname: yeah, I understand that issue | 15:05 |
| *** jistr is now known as jistr|call | 15:05 | |
| openstackgerrit | paul luse proposed openstack/swift: EC GET path: support use of alternate fragments on object servers https://review.openstack.org/213147 | 15:06 |
| joeljwright | notmyname: what I was suggesting is a mechanism by which an admin could explicitly allow tempurls for container1 to also access container1_segments (or some other named container but no others) | 15:06 |
| peluse | acoles, clayg: ^^ rebased and updated to jive w/how things were tweaked on etag buckets... let me know what you think | 15:06 |
| acoles | peluse: ack | 15:07 |
| joeljwright | notmyname: just some way to enable downloading an entire container without downloading every large object's data twice :S | 15:07 |
| acoles | peluse: i wanted to check in with you - are we still looking to do the optimistic GETs thing? I'll need to rebase the patch I had depnding on yours | 15:07 |
| peluse | acoles, but I just now saw your comment on the prev patch so that's not in there :( | 15:07 |
| acoles | peluse: np | 15:08 |
| peluse | acoles, yes I believe so... lets see what clay thinks of the alt frag stuff first though | 15:08 |
| acoles | peluse: k | 15:09 |
| openstackgerrit | John Dickinson proposed openstack/python-swiftclient: do hand-curated authors/changelog files https://review.openstack.org/220550 | 15:18 |
| notmyname | joeljwright: acoles: ^ that's one I'd prefer to resurrect and have in this swiftclient release | 15:18 |
| notmyname | just marked as WIP | 15:18 |
| notmyname | I need to fill in the AUTHORS file and update the ChangeLog for this release | 15:19 |
| notmyname | perhaps I can get that done today :-) | 15:19 |
| joeljwright | :) there's nothing like a deadline to motivate you | 15:21 |
| *** silor has quit IRC | 15:21 | |
| *** jistr|call is now known as jistr | 15:28 | |
| *** janonymous_ has joined #openstack-swift | 15:34 | |
| openstackgerrit | Romain LE DISEZ proposed openstack/swift: Allows to configure the rsync modules where the replicators will send data https://review.openstack.org/192186 | 15:38 |
| *** rledisez has quit IRC | 15:41 | |
| *** marzif has joined #openstack-swift | 15:42 | |
| *** pgbridge has joined #openstack-swift | 15:43 | |
| minwoob | notmyname: Good morning. Could you take a quick look at this and verify whether this bug actually needs a patch? | 15:54 |
| minwoob | notmyname: https://bugs.launchpad.net/swift/+bug/1491883 | 15:54 |
| openstack | Launchpad bug 1491883 in OpenStack Object Storage (swift) "Reconstructor complains 404 is invalid responce when building object" [Undecided,New] | 15:54 |
| *** chsc has joined #openstack-swift | 15:54 | |
| *** early has quit IRC | 15:55 | |
| *** early has joined #openstack-swift | 15:56 | |
| openstackgerrit | Tim Burke proposed openstack/swift: Add the ability to specify ranges for SLO segments https://review.openstack.org/211726 | 16:00 |
| *** haomaiwang has quit IRC | 16:01 | |
| *** haomaiwang has joined #openstack-swift | 16:01 | |
| minwoob | notmyname: I think the 404 logging should stay around, but ctennis brings up a good point in response. | 16:02 |
| notmyname | minwoob: for a cursory high-level look, it seems that replication has the same log message in update() but not update_deleted() | 16:02 |
| notmyname | on the other hand, I don't see where REPLICATE even has a code path to return 404 | 16:04 |
| notmyname | minwoob: so my initial response is that the reconstructor logging should be similar to replication. and ctennis makes a very good point about (1) the 404s being available in the object server logs and (2) not filling up the logs as a matter of course every time there is a ring change | 16:07 |
| notmyname | minwoob: I know that's not a very precise answer to your questions, but that's what I've got for now ;-) | 16:08 |
| notmyname | ...and it's commute time for me. be back online in about 30-40 minutes | 16:09 |
| openstackgerrit | Tim Burke proposed openstack/python-swiftclient: Add --decode-content option https://review.openstack.org/184957 | 16:10 |
| openstackgerrit | Tim Burke proposed openstack/python-swiftclient: Prompt for missing password https://review.openstack.org/189387 | 16:14 |
| *** jistr has quit IRC | 16:18 | |
| ctennis | notmyname: minwoob what would be cooler is that during the 5 minute stats output, it reported how many 404's it got..that would give you a similar amount of aggregate information without bloating the logs | 16:18 |
| *** NM has joined #openstack-swift | 16:22 | |
| *** petertr7 is now known as petertr7_away | 16:22 | |
| minwoob | ctennis: +1 | 16:23 |
| *** esker has quit IRC | 16:29 | |
| *** lyrrad has joined #openstack-swift | 16:37 | |
| notmyname | ctennis: yeah, but you lose the per-transaction resolution. roll-up should (probably) happen via statsd and/or some management software (ctennis, know of any good swift management software? ;-) | 16:46 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Trivial Key Master for encryption https://review.openstack.org/193749 | 16:49 |
| acoles | jrichli: ^^ | 16:49 |
| *** marzif has quit IRC | 16:51 | |
| openstackgerrit | janonymous proposed openstack/swift: Python 3 deprecated the logger.warn method in favor of warning https://review.openstack.org/208222 | 16:58 |
| openstackgerrit | John Dickinson proposed openstack/python-swiftclient: do hand-curated authors/changelog files https://review.openstack.org/220550 | 16:59 |
| *** haomaiwang has quit IRC | 17:01 | |
| *** haomaiwang has joined #openstack-swift | 17:01 | |
| *** openstackgerrit_ has joined #openstack-swift | 17:02 | |
| acoles | notmyname: should *all* authors appear in the mailmap? | 17:03 |
| notmyname | acoles: just those who need it, actually. ie those who have contributed under more than one email address (or prefer a different email address listed than the one they contributed with) | 17:04 |
| notmyname | acoles: IIRC, I ended up copying over the one from swift because swift authors are a superset (in generat) | 17:04 |
| acoles | notmyname: got it | 17:04 |
| notmyname | last thing to do there is to update the changelog. working on that now | 17:07 |
| minwoob | ctennis: For https://bugs.launchpad.net/swift/+bug/1491908 the logic for the 202 response handling should be in swift/proxy/controllers/obj.py right? | 17:07 |
| openstack | Launchpad bug 1491908 in OpenStack Object Storage (swift) "proxy server reports 202 even though EC backend didn't accept object" [Undecided,New] | 17:07 |
| *** marzif has joined #openstack-swift | 17:07 | |
| acoles | notmyname: oic - i was about to comment that stuff is missing from changelog :) | 17:08 |
| minwoob | ctennis: It's the bug that you filed for obj servers returning 409 which then bubble up as 202's by the proxy. | 17:09 |
| ctennis | minwoob: yeah, I talked to clayg about it last night. There is a special "feature" in there that changes 409s to 202s to make container sync stuff work better | 17:09 |
| ctennis | minwoob: the side effect being that in the case of a majority of 409's the proxy server acts like everything was OK | 17:09 |
| ctennis | even though it isn't | 17:09 |
| *** annegentle has quit IRC | 17:10 | |
| hugespoon | sorry all, i dont think i spotted a response yesterday, do the Swift ACLs only work between Keystone projects and not within them? just wanted to confirm that im reading the docs correctly | 17:11 |
| *** rjaiswal has joined #openstack-swift | 17:11 | |
| minwoob | ctennis: What's the reasoning behind it? | 17:11 |
| notmyname | acoles: joeljwright: looking at what's landed since the last release, I think a 2.5.1 tag is appropriate instead of 2.6.0 | 17:15 |
| openstackgerrit | John Dickinson proposed openstack/python-swiftclient: do hand-curated authors/changelog files https://review.openstack.org/220550 | 17:15 |
| notmyname | there's the changes | 17:15 |
| notmyname | acoles: timburke: joeljwright: ok, I think that wraps up the last needed thing for swiftclient ^ | 17:17 |
| minwoob | ctennis: clayg: So are we calling that one a non-bug, then? | 17:17 |
| *** petertr7_away is now known as petertr7 | 17:19 | |
| *** jlhinson has quit IRC | 17:23 | |
| *** zhill has joined #openstack-swift | 17:26 | |
| notmyname | yikes. just had a kernel panic on my mac | 17:28 |
| notmyname | torgomatic: did you mean to not leave a vote on https://review.openstack.org/#/c/220550/ | 17:29 |
| torgomatic | notmyname: I tried to put a +1, but I guess it didn't work | 17:29 |
| openstackgerrit | Alistair Coles proposed openstack/python-swiftclient: do hand-curated authors/changelog files https://review.openstack.org/220550 | 17:29 |
| torgomatic | probably pilot error | 17:29 |
| acoles | notmyname: ^^ couple of typos fixed | 17:29 |
| notmyname | torgomatic: oh. looks like you did on patch set 1 | 17:30 |
| notmyname | acoles: thanks | 17:30 |
| torgomatic | notmyname: that's offline reviewing for you | 17:30 |
| notmyname | ah ok | 17:30 |
| ctennis | minwoob: no it's a bug, it's jst that the current behavior fixes a specific problem btu also introduces another, so someone (clay?) may need to weigh in on a good solution | 17:32 |
| ctennis | minwoob: my understanding is that when we container sync since the timestamps are the same, we expect 409s. | 17:32 |
| ctennis | minwoob: so in the container sync case, 409s are a sign that we are "ok" | 17:32 |
| ctennis | minwoob: however in the general case, 409s shouldn't translate back to the proxy server as OKs. Otherwise we're giving the user a false sense that they updated the object when nothing changed. | 17:33 |
| *** thurloat is now known as thurloat_isgone | 17:33 | |
| *** esker has joined #openstack-swift | 17:34 | |
| *** esker has quit IRC | 17:34 | |
| *** esker has joined #openstack-swift | 17:35 | |
| *** geaaru has quit IRC | 17:35 | |
| minwoob | ctennis: clayg: Why can't we then have it so that 409's are handled properly in the normal case, but for container sync have some logic that checks for say, a flag or header indicating that 409 is expected? | 17:38 |
| openstackgerrit | janonymous proposed openstack/swift: Python 3 using builtins instead of __builtin__, rename raw_input() to input() from six.moves https://review.openstack.org/200932 | 17:39 |
| openstackgerrit | Tim Burke proposed openstack/swift: Add the ability to specify ranges for SLO segments https://review.openstack.org/211726 | 17:45 |
| *** acoles is now known as acoles_ | 17:47 | |
| *** annegentle has joined #openstack-swift | 17:49 | |
| *** jrichli has quit IRC | 17:49 | |
| *** devlaps has joined #openstack-swift | 17:56 | |
| *** haomaiwang has quit IRC | 18:01 | |
| *** haomaiwang has joined #openstack-swift | 18:01 | |
| *** gyee has joined #openstack-swift | 18:08 | |
| openstackgerrit | janonymous proposed openstack/swift: Python 3 deprecated the logger.warn method in favor of warning https://review.openstack.org/208222 | 18:13 |
| notmyname | minwoob: clayg: FYI clayg is out today | 18:13 |
| *** jbonjean has left #openstack-swift | 18:13 | |
| minwoob | notmyname: No prob. I suppose that such a solution would add a lot of overhead. | 18:19 |
| *** trifon has quit IRC | 18:22 | |
| ctennis | minwoob: that's probably the right approach | 18:34 |
| *** mgarza_ has joined #openstack-swift | 18:42 | |
| *** annegentle has quit IRC | 18:44 | |
| *** annegentle has joined #openstack-swift | 18:45 | |
| *** SkyRocknRoll_ has quit IRC | 18:54 | |
| *** haomaiwang has quit IRC | 19:01 | |
| *** 5EXAB2JHN has joined #openstack-swift | 19:01 | |
| wbhuber | ctennis: do you have the link to the LP bug where reconstructor is spitting out stack trace similarly to what we saw from proxy.controller.obj.py? i couldn't find it myself. | 19:06 |
| *** resker has joined #openstack-swift | 19:06 | |
| *** esker has quit IRC | 19:10 | |
| ctennis | wbhuber: let me see | 19:15 |
| wbhuber | clayg: peluse: top of my head, concurrency is not to be picked up by object-reconstructor, right? | 19:15 |
| ctennis | wbhuber: I thought I put both outputs in the same ticket | 19:19 |
| ctennis | wbhuber: yeah it's comment 5 in https://bugs.launchpad.net/swift/+bug/1488610 | 19:20 |
| openstack | Launchpad bug 1488610 in OpenStack Object Storage (swift) "EC connection refused exception when storage node is offline" [Undecided,Invalid] - Assigned to Bill Huber (wbhuber) | 19:20 |
| peluse | wbhuber, why do you say that? | 19:20 |
| *** joeljwright has quit IRC | 19:20 | |
| peluse | wbhuber, it is coded to use concurrency (a little differently than the repl) but we haven't really fully tested/characterized how different values affect things | 19:21 |
| wbhuber | ctennis: thanks but i remember seeing a longer output than what was put in the comment 5. maybe peluse did it or my mind was astray. | 19:21 |
| wbhuber | peluse: i was running some one node removal test and cosbench workload performance stuff. seems reconstructor is running as one process with one speed. i changed concurrency to 10 but it seemed not to be picked up unless i am doing it wrong in the configuraiton. | 19:22 |
| peluse | wbhuber, concurrency doesn't mean multi-process. it means that the two types of jobs created intenrally will run on green threads | 19:22 |
| wbhuber | peluse: what are those two types of jobs? | 19:23 |
| peluse | wbhuber, sync and revert | 19:23 |
| wbhuber | peluse: oh right. | 19:24 |
| peluse | wbhuber, the only difference in this area wrt the replicator is that the ec recon does a bunch more work in creating the jobs before sending them off to threads so depending on what the ec recon is doing (sprocessing jobs or building) concurrency may or may not impact anything | 19:24 |
| *** _hrou_ has quit IRC | 19:25 | |
| peluse | wbhuber, but, again, we didn't totally verify that concurrency setting is working correclty via real cluster test (it should but I wouldn't trust it til someone proves it, or otherwise) | 19:25 |
| wbhuber | peluse: good to know. i'd try and see if i can check that one out in my cluster test. | 19:27 |
| peluse | wbhuber, that'd be great! | 19:29 |
| wbhuber | peluse: let me back up a bit. does concurrency play a role in replicator? has anyone characterized / test the values that are impacted by concurrency for replicator? | 19:33 |
| *** mgarza_ has quit IRC | 19:34 | |
| peluse | wbhuber, yeah, it controls the number of threads in the pool for sync and delete jobs in very much the same way... no idea if anyone has really characterized the impacr though | 19:35 |
| *** petertr7 is now known as petertr7_away | 19:36 | |
| *** mahatic has quit IRC | 19:38 | |
| wbhuber | peluse: wrt the new bugs you and ctennis opened that you found from running EC on the real cluster, which of those that require immediate attention? or better this way, which of those would be good if we fixed it before end of month when we ship 2.4.0? | 19:43 |
| ctennis | not entirely sure on the tirage wbhuber, I was hoping the swift experts would chime in and prioritize them | 19:47 |
| *** cdelatte has quit IRC | 19:49 | |
| *** thumpba has quit IRC | 19:50 | |
| peluse | wbhuber, I need to go through them as we were jump logging and moving forward | 19:50 |
| *** 5EXAB2JHN has quit IRC | 20:01 | |
| *** haomaiwang has joined #openstack-swift | 20:01 | |
| *** openstackgerrit_ has quit IRC | 20:03 | |
| *** openstackgerrit_ has joined #openstack-swift | 20:04 | |
| *** portante_ has joined #openstack-swift | 20:15 | |
| *** EmilienM has quit IRC | 20:16 | |
| *** portante has quit IRC | 20:16 | |
| *** wolsen has quit IRC | 20:16 | |
| *** eikke has quit IRC | 20:16 | |
| *** eikke has joined #openstack-swift | 20:16 | |
| *** EmilienM_ has joined #openstack-swift | 20:16 | |
| *** wolsen has joined #openstack-swift | 20:16 | |
| *** EmilienM_ is now known as EmilienM | 20:16 | |
| *** dustins has quit IRC | 20:21 | |
| *** dustins has joined #openstack-swift | 20:22 | |
| notmyname | wbhuber: yeah, the concurrency testing to see the changes would be great! that's one of the things that we'd love to see in EC testing. what happens if you double the concurrency? how much does it help/hurt? playing with the config values is an important thing | 20:30 |
| *** zhill has quit IRC | 20:35 | |
| *** zhill_desktop has quit IRC | 20:36 | |
| *** openstackgerrit_ has quit IRC | 20:47 | |
| *** openstackgerrit_ has joined #openstack-swift | 20:47 | |
| *** openstackgerrit_ has quit IRC | 20:48 | |
| *** openstackgerrit_ has joined #openstack-swift | 20:48 | |
| notmyname | I merged the manual changelog/authors patch to swiftclient | 20:50 |
| *** gyee has quit IRC | 20:50 | |
| *** gyee has joined #openstack-swift | 20:50 | |
| notmyname | when it lands, I'll initiate the release for 2.5.1 | 20:51 |
| *** openstackgerrit_ has quit IRC | 20:53 | |
| *** annegent_ has joined #openstack-swift | 21:00 | |
| *** marzif has quit IRC | 21:00 | |
| *** haomaiwang has quit IRC | 21:01 | |
| *** haomaiwang has joined #openstack-swift | 21:01 | |
| *** annegentle has quit IRC | 21:03 | |
| *** HenryG has quit IRC | 21:07 | |
| *** j_king has quit IRC | 21:08 | |
| *** gyee has quit IRC | 21:08 | |
| *** j_king has joined #openstack-swift | 21:09 | |
| *** changbl has quit IRC | 21:10 | |
| *** HenryG has joined #openstack-swift | 21:11 | |
| *** whydidyoustealmy has quit IRC | 21:11 | |
| *** dosaboy has quit IRC | 21:11 | |
| *** dosaboy has joined #openstack-swift | 21:12 | |
| *** changbl has joined #openstack-swift | 21:12 | |
| *** ho has quit IRC | 21:13 | |
| *** dustins has quit IRC | 21:16 | |
| *** ho has joined #openstack-swift | 21:17 | |
| *** whydidyoustealmy has joined #openstack-swift | 21:18 | |
| *** wbhuber_ has joined #openstack-swift | 21:19 | |
| *** gyee has joined #openstack-swift | 21:20 | |
| *** wbhuber has quit IRC | 21:23 | |
| *** marzif has joined #openstack-swift | 21:31 | |
| *** trifon has joined #openstack-swift | 21:32 | |
| *** marzif has quit IRC | 21:41 | |
| *** ctrath has quit IRC | 21:43 | |
| minwoob | notmyname: In previous hackathons, has anyone proposed any ideas for dynamically changing config values in an operational cluster? | 21:52 |
| minwoob | based on workloads, etc. | 21:53 |
| minwoob | The discussion on concurrency (as an example) reminded me of that question. | 21:55 |
| openstackgerrit | Pete Zaitcev proposed openstack/swift: Remove rfc822 import for compatibility with Python 3 https://review.openstack.org/220687 | 21:57 |
| openstackgerrit | Zack M. Davis proposed openstack/python-swiftclient: absolute expiry option for tempURL generation https://review.openstack.org/220688 | 21:59 |
| *** haomaiwang has quit IRC | 22:01 | |
| *** NM has quit IRC | 22:01 | |
| *** 17SADHVEF has joined #openstack-swift | 22:01 | |
| *** pberis has joined #openstack-swift | 22:04 | |
| *** chsc has quit IRC | 22:09 | |
| *** pberis has quit IRC | 22:12 | |
| openstackgerrit | Merged openstack/python-swiftclient: do hand-curated authors/changelog files https://review.openstack.org/220550 | 22:13 |
| redbo | locally sourced, atisinally crafted changelogs | 22:14 |
| MooingLemur | :) | 22:15 |
| *** jkugel has quit IRC | 22:15 | |
| *** zhill has joined #openstack-swift | 22:23 | |
| *** zhill_desktop has joined #openstack-swift | 22:23 | |
| notmyname | minwoob: nope. actually, people (ops people) have been pretty opposed to that. we've tried to keep swift fairly non-dynamic. ie no automatic ring changes. no auto config changes. keeps ops life easier | 22:23 |
| notmyname | redbo: shade-grown | 22:24 |
| notmyname | free range | 22:24 |
| *** haypo has left #openstack-swift | 22:29 | |
| *** trifon has quit IRC | 22:30 | |
| *** zaitcev has joined #openstack-swift | 22:32 | |
| *** ChanServ sets mode: +v zaitcev | 22:32 | |
| zaitcev | LOL | 22:32 |
| zaitcev | zackmdavis: Are you around? | 22:32 |
| *** zhill_desktop has quit IRC | 22:32 | |
| *** zhill has quit IRC | 22:33 | |
| *** zhill has joined #openstack-swift | 22:33 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/python-swiftclient: Updated from global requirements https://review.openstack.org/89250 | 22:34 |
| *** marcusvrn_ has quit IRC | 22:35 | |
| *** _hrou_ has joined #openstack-swift | 22:35 | |
| *** trifon has joined #openstack-swift | 22:40 | |
| *** zhill has quit IRC | 22:43 | |
| *** zhill_desktop has joined #openstack-swift | 22:45 | |
| notmyname | Friday fun: https://www.surveymonkey.com/r/5WQ7Y9F <-- tshirt size/color survey | 22:46 |
| *** zhill has joined #openstack-swift | 22:47 | |
| *** annegent_ has quit IRC | 22:47 | |
| *** minwoob has quit IRC | 22:54 | |
| *** 17SADHVEF has quit IRC | 23:01 | |
| *** haomaiwang has joined #openstack-swift | 23:01 | |
| *** zaitcev has left #openstack-swift | 23:09 | |
| *** jamielennox is now known as jamielennox|away | 23:13 | |
| *** wbhuber has joined #openstack-swift | 23:15 | |
| notmyname | that's cool. HP has a women-in-tech scholarship program http://h30499.www3.hp.com/t5/Grounded-in-the-Cloud/OpenStack-Scholarship-Program-advancing-the-next-generation-of/ba-p/6780601 | 23:17 |
| *** lyrrad has quit IRC | 23:23 | |
| *** lyrrad has joined #openstack-swift | 23:24 | |
| openstackgerrit | Zack M. Davis proposed openstack/python-swiftclient: make ClientException.http_status default to None rather than 0 https://review.openstack.org/220697 | 23:25 |
| *** zhill_ has joined #openstack-swift | 23:26 | |
| *** zhill has quit IRC | 23:26 | |
| *** zhill_desktop has quit IRC | 23:27 | |
| *** hrou has joined #openstack-swift | 23:32 | |
| *** _hrou_ has quit IRC | 23:34 | |
| *** lyrrad_ has joined #openstack-swift | 23:39 | |
| *** zhill_desktop has joined #openstack-swift | 23:40 | |
| *** lyrrad has quit IRC | 23:41 | |
| *** lyrrad_ is now known as lyrrad | 23:41 | |
| *** gyee has quit IRC | 23:41 | |
| *** zhill_ has quit IRC | 23:42 | |
| *** zhill has joined #openstack-swift | 23:43 | |
| *** trifon has quit IRC | 23:44 | |
| *** zhill has quit IRC | 23:50 | |
| *** zhill has joined #openstack-swift | 23:51 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!