| *** haomaiwang has quit IRC | 00:01 | |
| *** haomaiwang has joined #openstack-swift | 00:01 | |
| *** m_kazuhiro has joined #openstack-swift | 00:24 | |
| *** zhill has quit IRC | 00:34 | |
| *** garthb__ has quit IRC | 00:35 | |
| *** garthb__ has joined #openstack-swift | 00:39 | |
| *** forcebanana has joined #openstack-swift | 00:44 | |
| *** haomaiwang has quit IRC | 01:01 | |
| *** haomaiwang has joined #openstack-swift | 01:01 | |
| openstackgerrit | Takashi Kajinami proposed openstack/swift: Allow asynchronous update to account in container PUT/DELETE https://review.openstack.org/87882 | 01:05 |
|---|---|---|
| ho_ | takashi: nice! | 01:09 |
| *** wasmum has joined #openstack-swift | 01:15 | |
| openstackgerrit | Takashi Kajinami proposed openstack/swift: Fix 404 in container PUT when the majority of the accounts are missing https://review.openstack.org/87882 | 01:21 |
| *** takashi has joined #openstack-swift | 01:27 | |
| *** haomaiwang has quit IRC | 01:32 | |
| *** 14WAAOU1P has joined #openstack-swift | 01:33 | |
| *** garthb__ has quit IRC | 01:34 | |
| *** lcurtis has joined #openstack-swift | 01:37 | |
| *** tsg has quit IRC | 01:37 | |
| *** asettle has quit IRC | 01:45 | |
| *** Jeffrey4l has joined #openstack-swift | 01:48 | |
| *** 14WAAOU1P has quit IRC | 02:01 | |
| *** haomaiwang has joined #openstack-swift | 02:01 | |
| *** thumpba has joined #openstack-swift | 02:10 | |
| *** klrmn has quit IRC | 02:12 | |
| *** Lickitysplitted has quit IRC | 02:16 | |
| *** Lickitysplitted has joined #openstack-swift | 02:16 | |
| openstackgerrit | Samuel Merritt proposed openstack/swift: Speed up get_more_nodes() when there is an empty zone https://review.openstack.org/267273 | 02:16 |
| *** esker has joined #openstack-swift | 02:18 | |
| *** lcurtis has quit IRC | 02:19 | |
| *** klrmn has joined #openstack-swift | 02:20 | |
| *** esker has quit IRC | 02:23 | |
| *** mingdang1 has joined #openstack-swift | 02:27 | |
| *** lcurtis has joined #openstack-swift | 02:31 | |
| *** thumpba has quit IRC | 02:32 | |
| *** Jeffrey4l has quit IRC | 02:38 | |
| *** Jeffrey4l has joined #openstack-swift | 02:38 | |
| *** thumpba has joined #openstack-swift | 02:42 | |
| openstackgerrit | Takashi Kajinami proposed openstack/swift: Fix 404 in container PUT when the majority of the accounts are missing https://review.openstack.org/87882 | 02:42 |
| *** klrmn has quit IRC | 02:43 | |
| *** McMurlock1 has joined #openstack-swift | 02:52 | |
| *** _fortis has quit IRC | 02:52 | |
| *** lpabon_ has quit IRC | 02:52 | |
| *** ntt has quit IRC | 02:52 | |
| *** McMurlock has quit IRC | 02:52 | |
| *** ekarlso has quit IRC | 02:52 | |
| *** ChubYann has quit IRC | 02:52 | |
| *** ChubYann has joined #openstack-swift | 02:53 | |
| *** proteusguy has quit IRC | 02:53 | |
| *** lpabon has joined #openstack-swift | 02:53 | |
| *** proteusguy has joined #openstack-swift | 02:53 | |
| *** forcebanana has quit IRC | 02:59 | |
| *** haomaiwang has quit IRC | 03:01 | |
| *** haomaiwa_ has joined #openstack-swift | 03:01 | |
| *** haomaiwa_ has quit IRC | 03:04 | |
| *** diogogmt has joined #openstack-swift | 03:04 | |
| *** _fortis has joined #openstack-swift | 03:05 | |
| *** ekarlso has joined #openstack-swift | 03:05 | |
| *** esker has joined #openstack-swift | 03:07 | |
| *** sanchitmalhotra has joined #openstack-swift | 03:10 | |
| *** esker has quit IRC | 03:12 | |
| *** thumpba has quit IRC | 03:13 | |
| *** thumpba has joined #openstack-swift | 03:16 | |
| *** thumpba has quit IRC | 03:17 | |
| *** links has joined #openstack-swift | 03:17 | |
| *** esker has joined #openstack-swift | 03:20 | |
| *** esker has quit IRC | 03:25 | |
| *** thumpba has joined #openstack-swift | 03:26 | |
| *** thumpba has quit IRC | 03:28 | |
| *** thumpba has joined #openstack-swift | 03:28 | |
| *** daemontool has quit IRC | 03:30 | |
| *** dimasot has joined #openstack-swift | 03:31 | |
| *** klrmn has joined #openstack-swift | 03:44 | |
| *** esker has joined #openstack-swift | 03:45 | |
| *** esker has quit IRC | 03:50 | |
| *** lcurtis has quit IRC | 04:05 | |
| *** esker has joined #openstack-swift | 04:09 | |
| *** esker has quit IRC | 04:14 | |
| *** thumpba has quit IRC | 04:15 | |
| *** esker has joined #openstack-swift | 04:23 | |
| *** esker has quit IRC | 04:28 | |
| *** thumpba has joined #openstack-swift | 04:28 | |
| *** mragupat has joined #openstack-swift | 04:35 | |
| *** thumpba has quit IRC | 04:44 | |
| *** janonymous_ has joined #openstack-swift | 04:45 | |
| *** janonymous has quit IRC | 04:46 | |
| *** esker has joined #openstack-swift | 04:47 | |
| *** esker has quit IRC | 04:52 | |
| *** thumpba has joined #openstack-swift | 04:54 | |
| openstackgerrit | Merged openstack/swift-bench: Pass environment variables of proxy to tox https://review.openstack.org/264134 | 05:09 |
| *** esker has joined #openstack-swift | 05:12 | |
| *** itlinux has joined #openstack-swift | 05:12 | |
| *** haomaiwang has joined #openstack-swift | 05:16 | |
| *** esker has quit IRC | 05:16 | |
| *** klrmn has quit IRC | 05:19 | |
| *** silor has joined #openstack-swift | 05:22 | |
| *** gyee has quit IRC | 05:30 | |
| *** thumpba has quit IRC | 05:30 | |
| *** asettle has joined #openstack-swift | 05:33 | |
| *** trifon has joined #openstack-swift | 05:36 | |
| openstackgerrit | dangming proposed openstack/swift: fix bug 1513955 https://review.openstack.org/267308 | 05:37 |
| openstack | bug 1513955 in OpenStack Object Storage (swift) ""swift-ring-builder set_info" command doesn't recognize 0 as a search value when "--id" is used as a key" [Low,In progress] https://launchpad.net/bugs/1513955 | 05:37 |
| *** thumpba has joined #openstack-swift | 05:41 | |
| *** asettle is now known as asettle-ask | 05:51 | |
| *** asettle-ask is now known as asettle-afk | 05:51 | |
| *** haomaiwa_ has joined #openstack-swift | 05:56 | |
| *** haomaiwang has quit IRC | 05:56 | |
| *** haomaiwa_ has quit IRC | 06:01 | |
| *** haomaiwang has joined #openstack-swift | 06:01 | |
| *** janonymous_ has quit IRC | 06:09 | |
| *** itlinux has quit IRC | 06:09 | |
| *** SkyRocknRoll_ has joined #openstack-swift | 06:10 | |
| *** thumpba has quit IRC | 06:14 | |
| *** mattoliverau has left #openstack-swift | 06:21 | |
| *** mattoliverau has joined #openstack-swift | 06:21 | |
| *** ChanServ sets mode: +v mattoliverau | 06:21 | |
| openstackgerrit | Merged openstack/swift: Get better at closing WSGI iterables. https://review.openstack.org/217750 | 06:24 |
| *** thumpba has joined #openstack-swift | 06:25 | |
| *** ChubYann has quit IRC | 06:31 | |
| *** nakagawamsa has joined #openstack-swift | 06:31 | |
| *** nakagawamsa has quit IRC | 06:31 | |
| *** esker has joined #openstack-swift | 06:41 | |
| openstackgerrit | Bing Hu proposed openstack/python-swiftclient: Support uploading to an object in swift from stdin update code and add unittest use mkstemp instead of NamedTemporaryFile, upadte unittest Closes-bug: #1521342 https://review.openstack.org/259410 | 06:44 |
| openstack | bug 1521342 in python-swiftclient "support uploading from a stdin stream" [Wishlist,In progress] https://launchpad.net/bugs/1521342 - Assigned to Bing Hu (hubingsh) | 06:44 |
| *** esker has quit IRC | 06:46 | |
| *** haomaiwang has quit IRC | 07:01 | |
| *** haomaiwang has joined #openstack-swift | 07:01 | |
| *** daemontool has joined #openstack-swift | 07:14 | |
| *** thumpba has quit IRC | 07:16 | |
| *** asettle-afk has quit IRC | 07:21 | |
| *** daemontool has quit IRC | 07:22 | |
| *** ppai has joined #openstack-swift | 07:23 | |
| *** thumpba has joined #openstack-swift | 07:26 | |
| *** nakagawamsa has joined #openstack-swift | 07:41 | |
| *** mragupat has quit IRC | 07:42 | |
| *** eranrom has joined #openstack-swift | 07:42 | |
| *** jmccarthy has joined #openstack-swift | 07:50 | |
| *** joker_ has joined #openstack-swift | 07:56 | |
| *** asettle-afk has joined #openstack-swift | 07:59 | |
| *** haomaiwang has quit IRC | 08:01 | |
| *** haomaiwang has joined #openstack-swift | 08:01 | |
| *** asettle-afk has quit IRC | 08:03 | |
| *** rledisez has joined #openstack-swift | 08:04 | |
| *** arnox has joined #openstack-swift | 08:10 | |
| *** zaitcev has quit IRC | 08:14 | |
| *** mrmoje has joined #openstack-swift | 08:18 | |
| *** geaaru has joined #openstack-swift | 08:23 | |
| *** yatin has joined #openstack-swift | 08:24 | |
| *** nakagawamsa has joined #openstack-swift | 08:29 | |
| *** nakagawamsa has quit IRC | 08:29 | |
| *** thumpba has quit IRC | 08:30 | |
| *** jordanP has joined #openstack-swift | 08:53 | |
| openstackgerrit | Jude Augustine Job proposed openstack/python-swiftclient: Error with uploading large object includes unicode path https://review.openstack.org/265544 | 08:56 |
| *** haomaiwang has quit IRC | 09:01 | |
| *** haomaiwang has joined #openstack-swift | 09:01 | |
| *** jistr has joined #openstack-swift | 09:03 | |
| *** SkyRocknRoll_ has quit IRC | 09:03 | |
| *** SkyRocknRoll_ has joined #openstack-swift | 09:16 | |
| *** ig0r_ has joined #openstack-swift | 09:17 | |
| *** jamielennox is now known as jamielennox|away | 09:18 | |
| *** daemontool has joined #openstack-swift | 09:23 | |
| *** mrmoje has quit IRC | 09:26 | |
| *** mrmoje has joined #openstack-swift | 09:26 | |
| *** proteusguy has quit IRC | 09:37 | |
| *** thumpba has joined #openstack-swift | 09:38 | |
| *** SkyRocknRoll has quit IRC | 09:40 | |
| *** ig0r_ has quit IRC | 09:41 | |
| *** thumpba has quit IRC | 09:48 | |
| *** acoles_ is now known as acoles | 09:52 | |
| *** proteusguy has joined #openstack-swift | 09:54 | |
| acoles | clayg: torgomatic_ kota_ thanks for reviewing patch 266599, i can cross first item off today's todo list :) | 09:58 |
| patchbot | acoles: https://review.openstack.org/#/c/266599/ - Make object-auditor storage-policy-aware | 09:58 |
| *** yatin has quit IRC | 10:00 | |
| *** haomaiwang has quit IRC | 10:01 | |
| *** haomaiwang has joined #openstack-swift | 10:01 | |
| *** yatin has joined #openstack-swift | 10:02 | |
| *** aix has joined #openstack-swift | 10:32 | |
| *** SkyRocknRoll_ has quit IRC | 10:42 | |
| *** trifon has quit IRC | 10:49 | |
| *** aix has quit IRC | 10:52 | |
| *** SkyRocknRoll_ has joined #openstack-swift | 10:55 | |
| *** yatin has quit IRC | 10:59 | |
| *** admin6 has joined #openstack-swift | 10:59 | |
| *** haomaiwang has quit IRC | 11:01 | |
| *** haomaiwang has joined #openstack-swift | 11:01 | |
| *** Jeffrey4l has quit IRC | 11:04 | |
| *** mingdang1 has quit IRC | 11:15 | |
| *** yatin has joined #openstack-swift | 11:17 | |
| *** yatin has quit IRC | 11:23 | |
| *** lcurtis has joined #openstack-swift | 11:26 | |
| *** yatin has joined #openstack-swift | 11:29 | |
| *** yatin has quit IRC | 11:29 | |
| *** sanchitmalhotra has quit IRC | 11:33 | |
| *** d0ugal has quit IRC | 11:40 | |
| openstackgerrit | Ondřej Nový proposed openstack/swift: Show UTC time in swift-recon. https://review.openstack.org/251151 | 11:49 |
| openstackgerrit | Ondřej Nový proposed openstack/swift: Allow to change auditor sleep interval in config https://review.openstack.org/266561 | 11:53 |
| *** dudi has joined #openstack-swift | 11:57 | |
| dudi | hi, can someone help me on swift container-sync plzzz: https://ask.openstack.org/en/question/87351/swift-container-sync-not-working/ | 11:58 |
| *** d0ugal has joined #openstack-swift | 11:58 | |
| dudi | hi, can someone help me on swift container-sync plzzz: https://ask.openstack.org/en/question/87351/swift-container-sync-not-working/ | 11:58 |
| *** SkyRocknRoll_ has quit IRC | 11:59 | |
| *** haomaiwang has quit IRC | 12:01 | |
| *** haomaiwang has joined #openstack-swift | 12:01 | |
| dudi | hi, can someone help me on swift container-sync plzzz: https://ask.openstack.org/en/question/87351/swift-container-sync-not-working/ | 12:02 |
| dudi | hi, can someone help me on swift container-sync plzzz: https://ask.openstack.org/en/question/87351/swift-container-sync-not-working/ | 12:08 |
| *** hseipp has joined #openstack-swift | 12:11 | |
| *** hseipp has quit IRC | 12:11 | |
| *** hseipp has joined #openstack-swift | 12:11 | |
| *** km__ has quit IRC | 12:13 | |
| *** fthiagogv has quit IRC | 12:13 | |
| *** lcurtis has quit IRC | 12:16 | |
| *** mingdang1 has joined #openstack-swift | 12:16 | |
| dudi | hi, can someone help me on swift container-sync plzzz: https://ask.openstack.org/en/question/87351/swift-container-sync-not-working/ | 12:18 |
| mahatic | dudi: you didnt specify other config options like, if you have added it to the middleware and other conf for container syc. | 12:21 |
| mahatic | dudi: you may cross check a few things here: http://docs.openstack.org/developer/swift/overview_container_sync.html | 12:21 |
| *** aix has joined #openstack-swift | 12:22 | |
| *** hugespoon has quit IRC | 12:22 | |
| eranrom | dudi: Can you elaborate on your problem with container sync? | 12:23 |
| eranrom | dudi: oh missed the link :/ looking | 12:23 |
| dudi | thanks mahatic for responding..i did all things as given on http://docs.openstack.org/developer/swift/overview_container_sync.html | 12:24 |
| dudi | i can share my all config files if u want.. | 12:24 |
| dudi | for more details @mahatic | 12:24 |
| *** fthiagogv has joined #openstack-swift | 12:25 | |
| *** trifon has joined #openstack-swift | 12:25 | |
| *** McMurlock1 has quit IRC | 12:26 | |
| *** McMurlock has joined #openstack-swift | 12:27 | |
| eranrom | dudi: I assume cluster_ip1 and cluster_ip2 are resolvable on both nodes - right? | 12:27 |
| dudi | sorry but i dint understand resolvable? | 12:28 |
| eranrom | dudi: In the container-sync-realms.conf you specify cluster_cluster1 = http://cluster_ip1:8080/v1/ which means that the container servers from cluster 2 need to 'see' that address both in terms of DNS and or routing | 12:30 |
| *** McMurlock has quit IRC | 12:31 | |
| dudi | ya.. | 12:31 |
| dudi | eranrom: yes | 12:32 |
| *** forcebanana has joined #openstack-swift | 12:33 | |
| eranrom | dudi: alright. Lets try have a look at the logs. Do you see errors coming from any of the Swift services (proxy-server or container-server). In the source from which you upload we are interested in container-server logs in the target we are interested in the proxy-serverlogs | 12:34 |
| dudi | eranrom: i followed http://www.szemtsov.net/books/OpenStackSwift.pdf for cluster setup..please look page 159 what they doing for logging..because i'm not getting logs separately.. | 12:37 |
| dudi | in all.log file in /var/log/swift dir there is no logs related to container-server | 12:38 |
| *** mrmoje has quit IRC | 12:45 | |
| *** m_kazuhiro has quit IRC | 12:45 | |
| ho_ | cschwede: thanks for the review. i wrote my understanding for delete container in gerrit. i think other patches are alomost similar (i will put logs for them tomorrow) | 12:46 |
| *** mrmoje has joined #openstack-swift | 12:47 | |
| *** thumpba has joined #openstack-swift | 12:47 | |
| *** ho_ has quit IRC | 12:50 | |
| cschwede | ho: you’re welcome! alright, i will have a look at your comments | 12:50 |
| *** thumpba has quit IRC | 12:53 | |
| openstackgerrit | Merged openstack/swift: Make object-auditor storage-policy-aware https://review.openstack.org/266599 | 12:57 |
| *** haomaiwang has quit IRC | 12:59 | |
| *** lpabon_ has joined #openstack-swift | 13:05 | |
| *** CaioBren_ has joined #openstack-swift | 13:07 | |
| *** CaioBrentano has quit IRC | 13:10 | |
| *** CaioBren_ has quit IRC | 13:10 | |
| *** CaioBrentano has joined #openstack-swift | 13:11 | |
| *** diogogmt has quit IRC | 13:14 | |
| *** diogogmt has joined #openstack-swift | 13:17 | |
| *** admin6 has left #openstack-swift | 13:27 | |
| *** mrmoje has quit IRC | 13:29 | |
| *** haomaiwang has joined #openstack-swift | 13:30 | |
| *** silor has quit IRC | 13:33 | |
| *** dslevin has quit IRC | 13:39 | |
| *** dimasot has quit IRC | 13:39 | |
| *** diogogmt has quit IRC | 13:41 | |
| *** lpabon has quit IRC | 13:41 | |
| openstackgerrit | dangming proposed openstack/swift: Fix "--id" is used as a key https://review.openstack.org/267308 | 13:42 |
| openstackgerrit | dangming proposed openstack/swift: Fix "--id" is used as a key https://review.openstack.org/267308 | 13:43 |
| *** lcurtis has joined #openstack-swift | 13:45 | |
| openstackgerrit | dangming proposed openstack/swift: Fix "--id" is used as a key https://review.openstack.org/267308 | 13:45 |
| *** lcurtis has quit IRC | 13:51 | |
| *** haomaiwang has quit IRC | 14:01 | |
| *** haomaiwa_ has joined #openstack-swift | 14:01 | |
| eranrom | dudi: you still here? Do you see any logs from any of the Swift servers? | 14:20 |
| *** thumpba has joined #openstack-swift | 14:21 | |
| *** dimasot has joined #openstack-swift | 14:21 | |
| *** yatin has joined #openstack-swift | 14:29 | |
| *** changbl has quit IRC | 14:34 | |
| *** yatin has quit IRC | 14:35 | |
| *** blmartin has joined #openstack-swift | 14:37 | |
| *** petertr7_away is now known as petertr7 | 14:38 | |
| dudi | eranrom: still trying to get logs separately for different swift servers..btw i'm testing with single node cluster, means all services running in same node (object, acc etc)....just testing if get succeed will implement on large cluster.. | 14:40 |
| *** jistr has quit IRC | 14:40 | |
| *** haomaiwa_ has quit IRC | 14:40 | |
| *** jistr has joined #openstack-swift | 14:41 | |
| dudi | eranrom: my container-sync-realm: https://drive.google.com/a/contentsphere.com/file/d/0B5CsOqqijbyBVlBMNllQb0N4Vmc/view?usp=sharing | 14:42 |
| dudi | eranrom: proxy-server conf : https://drive.google.com/a/contentsphere.com/file/d/0B5CsOqqijbyBQWdDOWppMU9Ba2c/view?usp=sharing | 14:42 |
| eranrom | dudi: It does not matter if the logs are seperated or not. Any logs coming from the proxy will start with proxy-server: and the other serverof interest is container-sync: | 14:43 |
| dudi | ok i'll share my log files also | 14:44 |
| eranrom | dudi: no need to share them. Just grep for proxy-server and container-sync and look for errors | 14:45 |
| eranrom | dudi: Again in the source cluster (where you did the upload) we are interested in container-sync logs and on the target cluster in proxy-server logs | 14:46 |
| dudi | ok let me see | 14:46 |
| dudi | on main server: ERROR Syncing /srv/node/d2/containers/223/2b6/37ffec704d5dd80c1994b8e69380a2b6/37ffec704d5dd80c1994b8e69380a2b6.db {'name': 'pic007', 'deleted': 0, 'created_at': '1452510767.24668', 'storage_policy_index': 0, 'etag': '6bde2aa6394fde37e21748bc0578113b', 'content_type': 'application/octet-stream', 'ROWID': 1, 'size': 2000000}: #012Traceback (most recent call last):#012 File "/usr/local/lib/python2.7/dist-packa | 14:48 |
| dudi | on target server : proxy-server: ERROR WSGI: code 400, message Bad request syntax ('\x16\x03\x01\x01(\x01\x00\x01$\x03\x03\xe6\xbc\x92x$Km\x0f\xecSh\xf6;\x8c\xfa\x9fs\xd6\xb7\xd7\x1c\xf0[\xfdC2\xae^\xe4\x9b\xfd\x87\x00\x00\x8e\xc00\xc0,\xc0(\xc0$\xc0\x14\xc0') | 14:48 |
| dudi | container-sync errors : return self.do_open(httplib.HTTPSConnection, req)#012 File "/usr/lib/python2.7/urllib2.py", line 1184, in do_open#012 raise URLError(err)#012URLError: <urlopen error [Errno 1] _ssl.c:510: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol> (txn: tx862846328196489f8062c-0056939d29) | 14:50 |
| *** yatin has joined #openstack-swift | 14:51 | |
| *** links has quit IRC | 14:52 | |
| eranrom | dudi: ok. looks like an SSL problem, which is funny because all of the commands in your example are with http and port 8080 | 14:53 |
| *** blmartin has quit IRC | 14:56 | |
| *** blmartin has joined #openstack-swift | 14:58 | |
| eranrom | dudi: Can you tell me if there is an http_proxy definition in your container-server.conf file under the [container-sync] section? | 14:58 |
| dudi | eranrom: sorry for my last logs ... actually i reinstall swift and did some mistake.. | 15:00 |
| *** haomaiwa_ has joined #openstack-swift | 15:00 | |
| dudi | container-sync: ERROR Syncing /srv/node/d2/containers/223/2b6/37ffec704d5dd80c1994b8e69380a2b6/37ffec704d5dd80c1994b8e69380a2b6.db {'name': 'pic006', 'deleted': 0, 'created_at': '1452510767.30658', 'storage_policy_index': 0, 'etag': '6bde2aa6394fde37e21748bc0578113b', 'content_type': 'application/octet-stream', 'ROWID': 4, 'size': 2000000}: #012Traceback | 15:00 |
| *** haomaiwa_ has quit IRC | 15:01 | |
| dudi | File "/usr/lib/python2.7/urllib2.py", line 531, in http_error_default#012 raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)#012HTTPError: HTTP Error 401: Unauthorized (txn: tx4a03e3a54d41402c8753f-0056939fa4) | 15:01 |
| dudi | "proxy-server 26449" 0.0004 "-" 26443 0 Jan 11 17:59:20 ubuntu object-server: 192.168.132.65 - - [11/Jan/2016:12:29:20 +0000] "GET /d1/563/AUTH_.auth/USA/.services" 200 117 "GET http://192.168.132.65:8080/v1/AUTH_.auth/USA/.services" "txcd8dfd1d27514e4cbd317-005693a020" "proxy-server 26449" 0.0005 "-" 26443 0 Jan 11 17:59:20 ubuntu container-server: 192.168.132.65 - - [11/Jan/2016:12:29:20 +0000] "GET /d2/878/AUTH_f7f633d3-be | 15:01 |
| dudi | showing unauthorized..!! | 15:01 |
| *** ppai has quit IRC | 15:01 | |
| *** esker has joined #openstack-swift | 15:03 | |
| *** SkyRocknRoll has joined #openstack-swift | 15:05 | |
| *** esker has quit IRC | 15:05 | |
| eranrom | dudi: Right so we see unauthorized on the source side, wenow need to find the cause for this on the target side. Loog for 401 in the proxy-server in the other cluster. | 15:10 |
| dudi | eranrom: i think reason for SSL problem was: "cluster_USA = https://192.168.132.65:8080/v1/" which i corrected to "cluster_USA = http://192.168.132.65:8080/v1/".....above logs are after correcting this.. | 15:11 |
| eranrom | dudi: what happens here, if that the container sync daemon running on the source is trying to replicate the object to the target, and it gets this 401. | 15:11 |
| eranrom | dudi: hopefully, we will see on the target the reason for this. | 15:11 |
| dudi | eranrom: i tried with one way also and syncing b/w container on same cluster ..that also dint waork | 15:12 |
| *** Jeffrey4l has joined #openstack-swift | 15:13 | |
| *** klamath has joined #openstack-swift | 15:13 | |
| eranrom | dudi: ok, so you might have the same problem on both sides :-) Lets find out what it is. | 15:13 |
| dudi | need to share logs or not? | 15:16 |
| eranrom | dudi: I suggest 2 things:\ | 15:16 |
| *** mingdang1 has quit IRC | 15:17 | |
| eranrom | dudi: 1. Valiudate that the container-sync middleware is in the proxy-server pipeline. For now make sure it is in the target cluster pipeline, and appearing before swauth or any other auth middleware. | 15:17 |
| eranrom | dudi: 2. Search for a log from proxy-server with PUT ... 401 in the target cluster | 15:18 |
| dudi | its just after [pipeline:main] and swauth middleware is at end of file. | 15:19 |
| dudi | ok | 15:19 |
| eranrom | dudi: Can you paste pls the pipeline line? | 15:21 |
| dudi | pipeline = catch_errors cache swauth proxy-server | 15:22 |
| eranrom | dudi: ok so this is definately a problem :-) one sec. | 15:23 |
| *** silor has joined #openstack-swift | 15:24 | |
| eranrom | dudi: which swift version are you using? 2.7.1 is probably the swiftclient version. | 15:25 |
| *** mragupat has joined #openstack-swift | 15:25 | |
| eranrom | dudi: you basically need to add the container-sync middleware before swauth | 15:27 |
| eranrom | dudi: I need to go, will try and have a look later today. What time zone are you? | 15:27 |
| dudi | Kalkutta , India | 15:28 |
| dudi | thanks for ur help eranrom | 15:28 |
| *** lcurtis has joined #openstack-swift | 15:30 | |
| dudi | i have container-sync middleware .... [pipeline:main] pipeline = catch_errors cache swauth proxy-server [filter:container_sync] use = egg:swift#container_sync | 15:31 |
| *** diazjf has joined #openstack-swift | 15:32 | |
| *** esker has joined #openstack-swift | 15:36 | |
| *** esker has quit IRC | 15:37 | |
| *** silor1 has joined #openstack-swift | 15:37 | |
| *** silor has quit IRC | 15:41 | |
| *** silor has joined #openstack-swift | 15:42 | |
| *** silor1 has quit IRC | 15:44 | |
| *** dudi has quit IRC | 15:44 | |
| openstackgerrit | Alistair Coles proposed openstack/python-swiftclient: Fix the http request headers being overwritten in logging https://review.openstack.org/252261 | 15:45 |
| *** itlinux has joined #openstack-swift | 15:48 | |
| *** petertr7 is now known as petertr7_away | 15:50 | |
| *** dudi has joined #openstack-swift | 15:53 | |
| *** dudi has quit IRC | 15:54 | |
| *** eranrom has quit IRC | 15:55 | |
| lcurtis | morning all...another swift question. can i use swift-ring-builder to find the nodes a partition lives on? | 15:55 |
| ahale | lcurtis: you can use swift-get-nodes for that | 15:56 |
| *** diazjf has quit IRC | 15:57 | |
| *** Jeffrey4l has quit IRC | 16:00 | |
| *** petertr7_away is now known as petertr7 | 16:01 | |
| *** esker has joined #openstack-swift | 16:03 | |
| *** diazjf has joined #openstack-swift | 16:06 | |
| *** petertr7 is now known as petertr7_away | 16:10 | |
| *** diogogmt has joined #openstack-swift | 16:21 | |
| *** nadeem has joined #openstack-swift | 16:22 | |
| *** garthb__ has joined #openstack-swift | 16:23 | |
| *** itlinux has quit IRC | 16:24 | |
| *** petertr7_away is now known as petertr7 | 16:25 | |
| *** trifon has quit IRC | 16:30 | |
| *** hyakuhei has joined #openstack-swift | 16:30 | |
| hyakuhei | Hey, is there anyone around I can ask about encryption, we (Barbican and the Security Project) are having a joint sprint on a proposal for BYOK in the core OpenStack services. | 16:33 |
| openstackgerrit | ke liang proposed openstack/swift: Drop python 2.6 support https://review.openstack.org/267688 | 16:33 |
| lcurtis | ahale: thank you! | 16:37 |
| jrichli | hyakuhei: i may be able to help you. Our encryption effort right now is using a pretty simple key solution. there are plans to use barbican in the future, however. | 16:38 |
| jrichli | hyakuhei: diazjf in the barbican community might be a good resource for you | 16:39 |
| notmyname | good morning | 16:39 |
| jrichli | good morning! | 16:39 |
| hyakuhei | jrichli: thanks buddy, yeah, diazjf is in the physical room here too :) | 16:39 |
| jrichli | hyakuhei: ah, ok. so you are at the meetup. :-) | 16:40 |
| hyakuhei | So we’re looking at working with the various IaaS services to provide a BYOK mechanism, similar to those offered by Azure, AWS and Google today. It’s a simple model where a client provides a key for a given operation (store/retreive etc) and Swift would use that key for the crypto ops rather than talking to Barbican etc. | 16:40 |
| hyakuhei | The general idea is that we’d get this functionality into Castellan, so all a project has to do is get the extra API call data from the front end to Castellan. | 16:41 |
| hyakuhei | but it’s early days and we’re still gathering info at the moment | 16:41 |
| hyakuhei | So I guess as an easy ask: Are there any recent crypto specs/blueprints or updates you could point me to? | 16:42 |
| jrichli | hyakuhei: sorry for the delay - i am in a workshop at the same time | 16:43 |
| hyakuhei | No worries :) | 16:43 |
| *** esker has quit IRC | 16:43 | |
| hyakuhei | Yeah Security and Barbican have an overlapping mid-cycle so we can work on things like this | 16:43 |
| gmmaha | good morning | 16:46 |
| jrichli | hyakuhei: I will post the crypto spec link in a moment. It doesn't address details of the key management that you are probably looking for. | 16:46 |
| *** esker has joined #openstack-swift | 16:46 | |
| jrichli | hyakuhei: there are some etherpads i can point you to that capture discussions about the future "keymaster". diazjf is familiar with the prototyping that has been done. | 16:47 |
| hyakuhei | That would be extremely helpful thank you | 16:47 |
| pdardeau | good morning | 16:49 |
| jrichli | hyakuhei: http://specs.openstack.org/openstack/swift-specs/specs/in_progress/at_rest_encryption.html | 16:49 |
| jrichli | hyakuhei https://etherpad.openstack.org/p/Swift_SmartKeyMaster | 16:50 |
| jrichli | hyakuhei https://etherpad.openstack.org/p/swift_production_keymaster_issues | 16:50 |
| jrichli | hyakuhei https://etherpad.openstack.org/p/swifjt-keymaster-with-castellan | 16:50 |
| *** hyakuhei has quit IRC | 16:50 | |
| *** hyakuhei has joined #openstack-swift | 16:51 | |
| *** esker has quit IRC | 16:51 | |
| hyakuhei | Thank you jrichli | 16:52 |
| *** pgbridge has joined #openstack-swift | 16:54 | |
| acoles | hyakuhei: sounds interesting. can you clarify - if the client provides the key for swift operations, why would swift need to pass any data to Castellan? | 16:56 |
| *** yatin has quit IRC | 16:56 | |
| acoles | hyakuhei: did I misunderstand "all a project has to do is get the extra API call data from the front end to Castellan"? | 16:56 |
| *** petertr7 is now known as petertr7_away | 16:57 | |
| *** hyakuhei has quit IRC | 16:59 | |
| *** hyakuhei has joined #openstack-swift | 17:02 | |
| hyakuhei | acoles: It wouldn’t have to | 17:03 |
| hyakuhei | It’s just that if castellan is there and already doing “key things” | 17:04 |
| hyakuhei | then checking if a BYOK was provided and just levergaging that might make sense. Other approaches include simply branching away from castellan when a BYOK is present | 17:04 |
| diazjf | jrichli, notmyname, acoles, feel free to add to https://etherpad.openstack.org/p/cEA79A5fG1 | 17:05 |
| * hyakuhei probably isn’t articulating this very well | 17:05 | |
| hyakuhei | -I fail without a a whiteboard. | 17:05 |
| acoles | hyakuhei: oic. so castellan would understand and BYOK headers for example and provide an API for Swift to just retrieve the key? | 17:06 |
| hyakuhei | Yeah | 17:07 |
| hyakuhei | If you’re already using castellan that might be the path of least resistance. | 17:07 |
| notmyname | that seems reasonable. my only concern is trying to solve this use case before the other one. scope creep and all that :-) | 17:07 |
| acoles | got it | 17:07 |
| hyakuhei | Sure, so we’re just documenting approaches right now. BYOK is a lovely feature to have and is pretty low effort for something that’s really very useful - in theory | 17:08 |
| hrou | Slight point of clarification; The the key reside in castellan and we're just indicating which one to fetch by index or some other means through a header ? | 17:09 |
| lcurtis | is there a way to replicate a single partition? | 17:10 |
| hyakuhei | So with BYOK keys should exist for as short a time as possible. | 17:10 |
| lcurtis | so in the case of swapping disks, i know that there is now only one copy, i can replicate to handoffs before i swap disks? | 17:10 |
| diazjf | hrou, key wont live anywhere it will just be used for direct encryption/decryption | 17:11 |
| hyakuhei | It’s quite possible (and a documented use case) that a user might push an object into Swift with a key but choose to retreive it in it’s encrpyted form (without providing the key) | 17:11 |
| hyakuhei | diazjf: +1 | 17:11 |
| hrou | diazjf, ah gotcha, so the castllan front end is just to have a consistent keymaster in this case (as it does not need to do anything, you could interface directly with swift in this scenario) ? | 17:11 |
| diazjf | hrou, correct since it is already being used in other services and will make its way to swift. | 17:12 |
| hrou | diazjf, gotcha but its essentially a no-op there's nothing for it to do in that scenario; But I do see talk of "Ability to provide a key id. So actual key doesn't need to be provided." in the etherpad ? | 17:13 |
| hrou | diazjf, hyakuhei is that a different mode of operation ? Where the key is stored in castellan (which can store it anywhere) and you are indicating the ID ? | 17:13 |
| acoles | hrou: "no-op" might actually be to map some header(s) to a key value, so Swift would not need to understand BYOK service header names | 17:14 |
| hyakuhei | So what we’re talking about is probably most easily described here: https://cloud.google.com/compute/docs/disks/customer-supplied-encryption | 17:14 |
| hyakuhei | If you’re already using castellan then it could make sense to add the BYOK logic there and just ensure that the extra headers get to castellan | 17:15 |
| hyakuhei | or you could have some BYOK logic before the keymaster/castellan | 17:15 |
| hrou | acoles, Ah gotcha, so from swift there's just one interface to get keys at the end of the day regardless of the source | 17:15 |
| hyakuhei | Yeah | 17:15 |
| hrou | Perfect ! Thanks acoles hyakuhei diazjf for the clarification that helped; Reading that doc now too. | 17:15 |
| acoles | hrou: thats just my guess. | 17:15 |
| acoles | hrou: i may have been accidentally right :) | 17:16 |
| hyakuhei | lol | 17:17 |
| hrou | hyakuhei, thanks for the doc I do understand that portion :) i.e. the byok operation, the confusion is what Castellan serves in that scenario, the answer may be nothing effectively (or maybe a re-mapping of headers) such that swift only needs to know about 1 way to access keys. | 17:17 |
| hrou | hyakuhei, I guess for the key-id approach it could end up looking the same, the difference being castellan will fetch said key and place it in the same header | 17:18 |
| acoles | jrichli: hrou we might want to bear in mind the use case mentioned above of the client wanting to retrieve the encrypted object | 17:18 |
| *** SkyRocknRoll has quit IRC | 17:20 | |
| jrichli | acoles: will think about that, thx. | 17:20 |
| *** gyee has joined #openstack-swift | 17:22 | |
| *** dslevin has joined #openstack-swift | 17:23 | |
| *** jordanP has quit IRC | 17:28 | |
| *** jistr has quit IRC | 17:29 | |
| *** geaaru has quit IRC | 17:30 | |
| *** rledisez has quit IRC | 17:31 | |
| *** haomaiwang has joined #openstack-swift | 17:41 | |
| *** zhill has joined #openstack-swift | 17:43 | |
| *** silor has quit IRC | 17:43 | |
| *** haomaiwang has quit IRC | 17:44 | |
| *** arnox has quit IRC | 17:48 | |
| *** diazjf has quit IRC | 17:50 | |
| *** hseipp has quit IRC | 17:51 | |
| *** arnox has joined #openstack-swift | 17:52 | |
| *** arnox has quit IRC | 17:52 | |
| *** klrmn has joined #openstack-swift | 17:54 | |
| *** zaitcev has joined #openstack-swift | 17:59 | |
| *** ChanServ sets mode: +v zaitcev | 17:59 | |
| *** lpabon_ has quit IRC | 18:01 | |
| *** daemontool has quit IRC | 18:01 | |
| zaitcev | Anyone has tips on getting to Bristol, England from abroad? | 18:05 |
| zaitcev | "Bristol has good transport connections to London and, closer by, the World Heritage city of Bath." | 18:05 |
| zaitcev | That... gives me too much choice. Kind of like Swift itself. | 18:06 |
| notmyname | lol | 18:06 |
| notmyname | I'm planning on plane to LHR and train to bristol. although the flight to galway or amsterdam then bristol sounded interesting | 18:06 |
| zaitcev | thanks | 18:07 |
| notmyname | I'm expecting travel logistics this morning from our corporate travel person | 18:07 |
| zaitcev | ballin' | 18:07 |
| *** tsg has joined #openstack-swift | 18:07 | |
| jrichli | corporate travel person - that's nice. we just get a tool. :-( | 18:08 |
| notmyname | it's a service we use (rather than an in-house thing). it's great. "hey, I need flights to FOO from BAR and a hotel too" "ok, here" done. | 18:09 |
| jrichli | sounds great | 18:11 |
| lcurtis | all...is it possible to replicate a single partition? | 18:13 |
| lcurtis | manually? | 18:13 |
| *** changbl has joined #openstack-swift | 18:13 | |
| *** blmartin has quit IRC | 18:13 | |
| acoles | zaitcev: fly into LHR, bus to Bristol is probably easiest. pre-book a car to take you from LHR to Bristol if you have larger budget. scroll down the eventbrite page for details. | 18:16 |
| zaitcev | acoles: thanks, got it | 18:17 |
| acoles | zaitcev: I assumed that since you asked the question, abroad==US :) | 18:17 |
| zaitcev | acoles: Dunno if anyone wants to get a skull cracked by a migrant's brick while taking the chunnel :-) | 18:18 |
| acoles | lol | 18:19 |
| *** diazjf has joined #openstack-swift | 18:21 | |
| *** petertr7_away is now known as petertr7 | 18:24 | |
| timburke | notmyname: if we want to backport patch 266599, i think we'll also need to backport patch 231121 (or at least parts of it) | 18:24 |
| patchbot | timburke: https://review.openstack.org/#/c/266599/ - Make object-auditor storage-policy-aware (MERGED) | 18:24 |
| patchbot | timburke: https://review.openstack.org/#/c/231121/ - Make ECDiskFile report all fragments found on disk (MERGED) | 18:24 |
| *** changbl has quit IRC | 18:35 | |
| openstackgerrit | Béla Vancsics proposed openstack/swift: Reduce code duplication https://review.openstack.org/267767 | 18:36 |
| *** eranrom has joined #openstack-swift | 18:37 | |
| *** dmorita has joined #openstack-swift | 18:38 | |
| eranrom | dudi: What you have if the container-sync configuration part in proxy-server.conf - This does not mean that it is actually in the pipeline. you need to have: pipeline = catch_errors cache container-sync swauth proxy-server | 18:39 |
| eranrom | dudi: one changed oyu need to restart the proxy (swift-init proxy-server restart). You need this on the target cluster. | 18:40 |
| *** dmorita has quit IRC | 18:43 | |
| *** blmartin has joined #openstack-swift | 18:46 | |
| *** diazjf has quit IRC | 18:47 | |
| *** forcebanana has quit IRC | 18:47 | |
| *** dmorita has joined #openstack-swift | 18:48 | |
| *** aix has quit IRC | 18:49 | |
| *** dmorita has quit IRC | 18:50 | |
| *** pdardeau has left #openstack-swift | 18:53 | |
| *** pdardeau has joined #openstack-swift | 18:53 | |
| *** dmorita has joined #openstack-swift | 18:56 | |
| *** dmorita has quit IRC | 18:56 | |
| *** dmorita has joined #openstack-swift | 18:57 | |
| *** hyakuhei has quit IRC | 18:57 | |
| eranrom | Dear Swift community. I thought this short writeup may help us in our everyday conversation over metadata. I first wrote it as a spec, but then thought it not really the right place. so there it is: | 18:58 |
| eranrom | https://github.com/eranrom/swift-thoughts/blob/master/metadata_taxonomy.rst | 18:58 |
| *** esker has joined #openstack-swift | 19:01 | |
| notmyname | eranrom: that's pretty good | 19:01 |
| notmyname | eranrom: point two under content-type, you suddenly mention content-length. was that intentional? | 19:02 |
| eranrom | notmyname,: that was fast :-) | 19:02 |
| *** esker has quit IRC | 19:02 | |
| eranrom | notmyname,: no that's an error. I will fix :-) | 19:02 |
| notmyname | ok | 19:02 |
| *** esker has joined #openstack-swift | 19:02 | |
| notmyname | I'd agree that content-length should never be updated ;-) | 19:02 |
| *** diazjf has joined #openstack-swift | 19:03 | |
| eranrom | notmyname,: even for SLOs? | 19:03 |
| notmyname | the use case for updating content-type is when it's set incorrectly on initial upload. eg default to application/octet-stream when it's really a jpeg or video or text/plain | 19:03 |
| notmyname | eranrom: well, everything sorta changes for *LO ;-) | 19:03 |
| notmyname | eranrom: but yeah, with SLOs, the length only changes with a change to the manifest. for DLOs, it's different | 19:04 |
| eranrom | notmyname,: ok, getting phylosophical, why not requiring to upload the object again? | 19:04 |
| eranrom | notmyname,: right | 19:04 |
| notmyname | because updating the content-type is a lot less bytes than reuploading a 5GB file | 19:04 |
| notmyname | mostly it's the "fault" of poor clients | 19:05 |
| notmyname | in swift's early days, we had a lot of requests for that at RAX. users needed to change the content type because it was set incorrectly (by our own SDKs) | 19:05 |
| eranrom | notmyname,: I see. I would guess that its more of a case where they already uploaded a lot of pics and not that those pics are 5GB... | 19:06 |
| notmyname | right | 19:06 |
| notmyname | eranrom: the other nitpick from that doc is about x-timestamp. turns out that it's user-settable (fun! [not really fun]). and it's also visible on a read. but it's almost always a bad idea to set it | 19:07 |
| *** forcebanana has joined #openstack-swift | 19:08 | |
| notmyname | it's probably more technically correct to call it data system metadata in your taxonomy. but it's also set by the system if the user doesn't set it (which is the normal case) | 19:08 |
| *** forcebanana has quit IRC | 19:08 | |
| *** dmorita has quit IRC | 19:08 | |
| *** forcebanana has joined #openstack-swift | 19:09 | |
| *** dmorita has joined #openstack-swift | 19:09 | |
| eranrom | notmyname,: interesting. I wonder if splitting this into a data system metadata and a storage system metadata would be helpful (referring to your comments which inclue the word 'fun') | 19:09 |
| *** hyakuhei has joined #openstack-swift | 19:10 | |
| notmyname | eranrom: turns out that right now you're probably the person who cares most about settable x-timestamp. it's a key part of container sync ;-) | 19:10 |
| eranrom | notmyname,: indeed :-) | 19:10 |
| *** dmorita has quit IRC | 19:11 | |
| *** dmorita has joined #openstack-swift | 19:11 | |
| eranrom | notmyname,: even more when we add fast-post support I guess. | 19:11 |
| timburke | notmyname: do i at least get to be a runner-up for submitting patch 265488? :P | 19:11 |
| patchbot | timburke: https://review.openstack.org/#/c/265488/ - Validate X-Timestamps | 19:11 |
| notmyname | eranrom: what do you mean by "splitting this"? splitting the definitions? or splitting the x-timestamp concept? | 19:11 |
| eranrom | notmyname,: having two of them. One that can / is set by the user and another one that is (probably) always set by the system. They will carry different key names. | 19:12 |
| notmyname | timburke: :-) | 19:12 |
| pdardeau | notmyname: is there a place where swift developer question/answer stuff is archived for future reference? | 19:13 |
| pdardeau | other than eavesdrop archive of channel | 19:15 |
| notmyname | eranrom: well, that sounds complicated and full of corner cases. as much as user-settable x-timestamp seems and sounds weird, I havne't actually seen any realy problems with it | 19:17 |
| notmyname | pdardeau: you mean stuff like "how does X work?" | 19:17 |
| pdardeau | notmyname: sorta, here's an example from last few days https://gist.github.com/pauldardeau/0f312cb03abda770702f | 19:18 |
| pdardeau | format is poor, but thinking of content | 19:20 |
| eranrom | notmyname,: well in that case no reason split... | 19:20 |
| notmyname | pdardeau: nope there's nothing like that today | 19:22 |
| pdardeau | notmyname: ok. i'll volunteer to start pulling some things together | 19:24 |
| notmyname | pdardeau: that's great! | 19:24 |
| notmyname | pdardeau: and I'd suggest using the openstack wiki (rooted at https://wiki.openstack.org/wiki/Swift) as a starting point | 19:25 |
| pdardeau | notmyname: cool, thx | 19:25 |
| pdardeau | notmyname: is the wiki content handled by the doc folks? | 19:27 |
| *** haomaiwang has joined #openstack-swift | 19:30 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Fix inconsistent suffix hashes after ssync of tombstone https://review.openstack.org/267788 | 19:34 |
| *** haomaiwang has quit IRC | 19:34 | |
| acoles | clayg: remember you spotted the inconsistent suffix hashes with the fast-post patch when the meta file names differed? | 19:35 |
| acoles | clayg: well in the course of fixing it I ran into a related problem on master, patch above ^^ | 19:36 |
| acoles | clayg: you described some thoughts you and/or torgomatic had about how hashing could be changed, and *if* I understood them then thats where this patch is headed. | 19:37 |
| acoles | i.e hash what diskfile cares about rather than whatever is on disk. | 19:37 |
| *** ChubYann has joined #openstack-swift | 19:41 | |
| acoles | clayg: maybe catch you tomorrow. ttfn | 19:42 |
| *** acoles is now known as acoles_ | 19:43 | |
| *** klamath has quit IRC | 19:43 | |
| *** diogogmt has quit IRC | 19:43 | |
| *** diogogmt has joined #openstack-swift | 19:53 | |
| Zyric | Good Morning | 20:04 |
| *** dmorita has quit IRC | 20:05 | |
| *** petertr7 is now known as petertr7_away | 20:08 | |
| *** gyee has quit IRC | 20:09 | |
| *** petertr7_away is now known as petertr7 | 20:18 | |
| *** nadeem has quit IRC | 20:34 | |
| *** nadeem has joined #openstack-swift | 20:34 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 20:37 |
| *** whitebyte has joined #openstack-swift | 20:38 | |
| *** torgomatic_ has quit IRC | 20:42 | |
| *** torgomatic has joined #openstack-swift | 20:44 | |
| *** ChanServ sets mode: +v torgomatic | 20:44 | |
| *** niya has joined #openstack-swift | 20:47 | |
| *** eranrom has quit IRC | 20:51 | |
| niya | hello, | 20:56 |
| whitebyte | niya: just ask your question | 20:59 |
| niya | I am new to IRC and Swift object storage. Currently, I am working on a project Bluebox using Swift object storage and Bluemix infrastructure. Front end of this app is java script and the back end is python. I am trying to make streaming upload and download using the iterator function and resp_chunk_size parameter of swift | 21:00 |
| *** dmorita has joined #openstack-swift | 21:05 | |
| niya | http://paste.openstack.org/show/483929/ | 21:06 |
| *** whitebyte has quit IRC | 21:08 | |
| niya | but somehow streaming upload and download is not working | 21:08 |
| *** dmorita has quit IRC | 21:09 | |
| *** dmorita has joined #openstack-swift | 21:13 | |
| lcurtis | hello all...question | 21:20 |
| lcurtis | running swift-object-replicator /tmp/tmp.conf -v -o <part#> | 21:22 |
| lcurtis | to try and replicate single partition but it is doing a walk of whole node it seems | 21:24 |
| lcurtis | is that correct behavior? | 21:25 |
| *** dmorita has quit IRC | 21:26 | |
| *** dmorita has joined #openstack-swift | 21:27 | |
| *** changbl has joined #openstack-swift | 21:27 | |
| notmyname | niya: so your backend is a server-side thing that is talking to swift? | 21:27 |
| clayg | waho - hey acoles! | 21:27 |
| *** diogogmt has quit IRC | 21:27 | |
| notmyname | niya: and your backend is using the swiftclient SDK to talk to swift? | 21:27 |
| *** aix has joined #openstack-swift | 21:28 | |
| niya | notmyname: yes | 21:28 |
| timburke | niya: what's the observed behavior? from the code you pasted, encodedOutputFile will be a generator that yields out the contents of the object 10 bytes at a time. is that what downloadingObject expects? what does *that* code do? | 21:29 |
| *** dmorita has quit IRC | 21:29 | |
| *** StevenK has quit IRC | 21:30 | |
| niya | notmyname:that code is not working properly. it is supposed to start downloading 10 bytes of data every time. | 21:31 |
| *** StevenK has joined #openstack-swift | 21:31 | |
| niya | notmyname: I dont know how to properly make use of generator function here with swift. | 21:31 |
| notmyname | niya: timburke is asking the right questions. what are you seeing now? | 21:32 |
| niya | timburke: Usually when we make a download, the process will first make space for that object on the server and after creating that space only the download starts. By this code I want to make Streamin Download, that is download should start as soon as it is requested without creatin space on server | 21:34 |
| mattoliverau | morning, got a baby check up this morning, so might be in and out a bit. | 21:36 |
| notmyname | mattoliverau: cool. hope it goes well | 21:38 |
| blmartin | mattoliverau: Morning, There is another pull request waiting for you! | 21:38 |
| blmartin | it just never ends, eh? | 21:38 |
| *** shakamunyi has quit IRC | 21:40 | |
| clayg | tsg: notmyname: so the saio docs still say jerasure -> http://docs.openstack.org/developer/swift/development_saio.html#configuring-each-node | 21:40 |
| clayg | tsg: notmyname: but like with the latest breakage they don't have too right? | 21:41 |
| notmyname | right | 21:41 |
| notmyname | IIRC tsg pulled out the jerasure references in tests (and most docs/configs). probably saio docs just got skipped | 21:41 |
| *** diogogmt has joined #openstack-swift | 21:42 | |
| clayg | k, i'll add a wishlist | 21:43 |
| mattoliverau | notmyname: thanks | 21:44 |
| mattoliverau | blmartin: again :P | 21:44 |
| *** thumpba has quit IRC | 21:44 | |
| timburke | niya: yes. and it seems like the code you posted is a good step in that direction; it seems like getObject will do the work of using swiftclient to talk to swift and then yielding out chunks, without needing to buffer the entire object. so where is it falling down, and how? | 21:44 |
| notmyname | clayg: thanks. looking at a global search for "jerasure", it seems like that's the only place where it's not couched in "here's an example of one of these options" language | 21:44 |
| *** diazjf has quit IRC | 21:45 | |
| notmyname | clayg: ie it's not exactly wrong now, but it could be better | 21:45 |
| *** thumpba has joined #openstack-swift | 21:45 | |
| openstackgerrit | Peter Chng proposed openstack/swift: Ensure sysmeta is written/updated on (fast) POST https://review.openstack.org/266545 | 21:46 |
| clayg | notmyname: ok, sweet so wishlist is fine -> lp bug #1534325 | 21:46 |
| openstack | Launchpad bug 1534325 in OpenStack Object Storage (swift) "remove jerasure from swift docs" [Wishlist,New] https://launchpad.net/bugs/1534325 | 21:46 |
| notmyname | clayg: yeah. perfect. thanks | 21:46 |
| *** blmartin has quit IRC | 21:47 | |
| *** blmartin has joined #openstack-swift | 21:47 | |
| *** dmorita has joined #openstack-swift | 21:49 | |
| *** thumpba has quit IRC | 21:49 | |
| *** jamielennox|away is now known as jamielennox | 21:49 | |
| openstackgerrit | Samuel Merritt proposed openstack/swift: Speed up get_more_nodes() when there is an empty zone https://review.openstack.org/267273 | 21:50 |
| niya | timburke: in downloadObject again I am using a loop in the server and collecting the chunks, I want to downoad this chunk every time instead of collecting it, and then swift should call the next chunk to download | 21:50 |
| *** cdelatte has quit IRC | 21:52 | |
| clayg | torgomatic: you suggested that when you debugged the issue with the slow affinity on zero weight regions you instrumented the probing in some way as to "count" the number of times you did useless work? | 21:57 |
| clayg | torgomatic: I think a test would be useful mainly to hint that future maintainers of the handoff function that even when there are zero wegith devices the selection must still be quick | 21:57 |
| torgomatic | clayg: yeah, I basically put a bunch of "probes += 1" in the code and printed its value out | 21:57 |
| clayg | torgomatic: so a test that a) sets up the secenario to describe the desired behavior and b) proxies "quick" as "counting # of expensive things we do" could be useful | 21:58 |
| clayg | although granting it my not prevent a regression of the next thing we break | 21:58 |
| clayg | torgomatic: ok, anyway - i might have to write something like that to review it fully - so maybe I can throw it up in a follow on - thanks! | 21:59 |
| torgomatic | np | 21:59 |
| clayg | AssertionError 100 != 10000 would be a great test failure to see if I revert common.ring on a patch that says "makes faster" | 21:59 |
| torgomatic | this is certainly true; the trick is finding something to hook into | 22:00 |
| torgomatic | all we've got in there is a list of array('H') and some dicts | 22:01 |
| clayg | OHHHH - ok | 22:01 |
| timburke | niya: as it's currently written, each call to getObject will create a new connection to swift and return a generator that yields out content 10 bytes at a time. downloadObject kind of looks like it expects each call to getObject to return the next chunk; might you want something like `for chunk in swift.getObject(...): downloadingObject(..., chunk)`? it's kind of hard for me to tell what downloadingObject expects or does | 22:01 |
| *** thumpba has joined #openstack-swift | 22:01 | |
| clayg | timburke: oh oh oh - are we staring at java code!? | 22:02 |
| *** thumpba has quit IRC | 22:02 | |
| timburke | clayg: i hope not! i don't see *nearly* enough semicolons! | 22:02 |
| clayg | oh how strange - i was thrown by the camel case | 22:05 |
| Zyric | Hi everyone, I've been having a lot of trouble with new SAIOs the last few days. I can't figure out why so many functests and probetests keep failing on fresh installs and would really appreciate some help. From what I can tell there's come kind of cleanup that isn't happening and maybe character encoding issues? Pastebin links incoming. | 22:09 |
| Zyric | Account info: http://paste.openstack.org/show/483931/ /var/log/swift/storage1.error: http://paste.openstack.org/show/483932/ Functests Results: http://paste.openstack.org/show/483933/ http://paste.openstack.org/show/483934/ | 22:10 |
| niya | timburke: http://paste.openstack.org/show/483937/ | 22:12 |
| *** bsdkurt has quit IRC | 22:13 | |
| notmyname | Zyric: the `resetswift` command in the saio should clean everything up. (although I though the functests were all self-cleaning now) | 22:13 |
| niya | timburke: This was the original code for downloadObject, it was written to download whole object and not only chunks, now i want to change this somehow to download inchunks. | 22:13 |
| notmyname | Zyric: any chance you've downloaded a patch and are still running code from that? are you on master and seeing there | 22:14 |
| notmyname | *these | 22:14 |
| *** nadeem has quit IRC | 22:14 | |
| Zyric | notmyname: I suspected that at first, but since then I've set up three new Ubuntu Server VMs and all of them have had similar issues. | 22:15 |
| *** dmorita has quit IRC | 22:15 | |
| notmyname | Zyric: next up is to check swift.conf and proxy-server.conf to see if you've made any changes that could affect what's going on | 22:15 |
| notmyname | eg you might have a middleware in there doing something? | 22:16 |
| Zyric | They're absolutely fresh installs, so should have default values for everything but I'll double check. | 22:16 |
| *** bsdkurt has joined #openstack-swift | 22:17 | |
| notmyname | Zyric: ok, then next step to to confirm functionality by hand and to isolate the failing tests. eg run just a failing test | 22:18 |
| timburke | niya: it looks like that's doing chunking; you've already got resp_chunk_size plumbed through. did it seem to be slurping the whole response into memory somewhere? | 22:19 |
| tsg | notmyname, clayg: I didn't submit my doc update yet .. was wondering about my question from yesterday - whether we need a note in the saio doc about enabling the "*-backports" repos | 22:20 |
| tsg | notmyname, clayg: will submit other changes | 22:21 |
| *** dmorita has joined #openstack-swift | 22:23 | |
| niya | timburke: yes it is taking the whole response into memory, (it should not have this return statement in downloadObject as this function will stop immediately after return and will not go for the next chunk) , | 22:23 |
| dimasot | dudi: you need o change your pipeline definition from "pipeline = catch_errors cache swauth proxy-server" to "pipeline = catch_errors cache container_sync swauth proxy-server " | 22:23 |
| *** dmorita has quit IRC | 22:23 | |
| *** petertr7 is now known as petertr7_away | 22:24 | |
| *** dmorita has joined #openstack-swift | 22:24 | |
| *** bsdkurt has quit IRC | 22:24 | |
| dimasot | dudi: otherwise your contaner_sync would not be a part of pipeline - and would be not used | 22:24 |
| *** dmorita has quit IRC | 22:25 | |
| dimasot | dudi: then your auhtentication would failed, and you will get 401 http error | 22:25 |
| tsg | clayg: I read "latest breakage" above :-) what was that about? | 22:25 |
| *** dmorita has joined #openstack-swift | 22:28 | |
| *** petertr7_away is now known as petertr7 | 22:28 | |
| *** bsdkurt has joined #openstack-swift | 22:29 | |
| *** mingdang1 has joined #openstack-swift | 22:33 | |
| *** blmartin has quit IRC | 22:36 | |
| openstackgerrit | Richard Hawkins proposed openstack/swift: _get_info_cache should encode unicode metadata https://review.openstack.org/267016 | 22:36 |
| dimasot | lcurtis: you can change the ring manualy - I am not sure if it is what you want - but you can set for any partition the devices that should hold it - and hten replicator would move it | 22:37 |
| dimasot | lcurtis: if you need more help with ring rebalancing ping me | 22:38 |
| lcurtis | dimasot: do you have a moment to discuss? | 22:42 |
| Zyric | notmyname: Neat, looks like if I resetswift first, then the functests all pass! Trying probetests now. Why might the cleanups not be working though? | 22:42 |
| *** petertr7 is now known as petertr7_away | 22:44 | |
| notmyname | Zyric: some of it may not not actually be cleaned up | 22:44 |
| Zyric | It actually looks like almost none of it is cleaned up, if I get the account info before it's completely empty and after I run functests there's lots of data there. | 22:46 |
| notmyname | the metadata might be sticking around | 22:46 |
| Zyric | Hmm. Is there a way to check that? Seems strange. | 22:48 |
| *** bsdkurt has left #openstack-swift | 22:48 | |
| notmyname | which may mean that subsequent test runs are runnign into limits or something | 22:48 |
| *** bsdkurt has joined #openstack-swift | 22:48 | |
| notmyname | yeah, just run it once and see what's there ;-) | 22:48 |
| Zyric | Hokey doke, I'll have another poke around once the probetests are done. Thanks for the tips :) | 22:49 |
| clayg | tsg: i'm sure it's all good breakage - but apparently something changed upstream with py-eclib and folks have a new build depends when installing via pip? | 22:49 |
| zaitcev | It used to be that functests pre-cleaned. I, for one, never bothered with resetswift. | 22:50 |
| *** forcebanana has quit IRC | 22:50 | |
| clayg | tsg: that is to say *I* only noticed it in response to some folks complaining about vsaio being broken - but the whole transistion was probably carefully planned and announced had I been paying attention there probably would have been a method by which I could have made the build still work before the release; but would have seemlessly transistioned to the new requirements w/o having any period where it was not possible to b | 22:51 |
| *** bsdkurt has left #openstack-swift | 22:51 | |
| *** bsdkurt has joined #openstack-swift | 22:52 | |
| tsg | clayg: :-) details will be useful .. were people complaining about vsaio after the pyeclib 1.2.0 release (Tuesday)? | 22:52 |
| *** bsdkurt has left #openstack-swift | 22:52 | |
| notmyname | I think it's the unbundling of liberasurecode | 22:53 |
| tsg | notmyname: it is likely that folks could not install liberasurecode because they didn't have backports repos enabled .. | 22:53 |
| *** bsdkurt1 has joined #openstack-swift | 22:53 | |
| tsg | that's pretty much the only dependency at the moment for pyeclib | 22:54 |
| clayg | tsg: not sure when exactly - i haven't puttogether a timeline. | 22:54 |
| clayg | tsg: pchng might have more details | 22:54 |
| *** bsdkurt1 has quit IRC | 22:54 | |
| *** bsdkurt has joined #openstack-swift | 22:54 | |
| clayg | tsg: I believe vsaio already add's backports - but was missing the install for the liberasure-dev package (from my current understanding) | 22:55 |
| bsdkurt | . | 22:55 |
| tsg | clayg: ok - that would cause problems with pyeclib 1.2.0 | 22:55 |
| clayg | tsg: I belive that's all there was to it; but if the saio docs don't currently describe a procedure that would result in a working swift they probably should be updated soon-ish (might be worth filing a bug) | 22:56 |
| tsg | clayg: ack - I will submit a few doc updates I have, this evening | 22:57 |
| clayg | tsg: oh thank you very much! | 22:57 |
| openstackgerrit | Samuel Merritt proposed openstack/swift: Let developers/operators add watchers to object audit https://review.openstack.org/212824 | 22:58 |
| *** hyakuhei has quit IRC | 22:58 | |
| *** km_ has joined #openstack-swift | 22:59 | |
| *** bsdkurt has quit IRC | 23:03 | |
| *** bsdkurt1 has joined #openstack-swift | 23:03 | |
| tsg | clayg: do you also want me to update vsaio? or does it pull from the swift repo from time to time? | 23:03 |
| *** bsdkurt1 has left #openstack-swift | 23:06 | |
| *** bsdkurt has joined #openstack-swift | 23:06 | |
| *** haomaiwang has joined #openstack-swift | 23:07 | |
| *** bsdkurt has quit IRC | 23:07 | |
| *** bsdkurt has joined #openstack-swift | 23:07 | |
| *** haomaiwang has quit IRC | 23:08 | |
| clayg | tsg: as I understand it pchng already squared it - so it's only the upstream saio docs | 23:12 |
| tsg | clayg: great - thanks | 23:13 |
| tsg | thanks pchng! | 23:13 |
| *** dmorita has quit IRC | 23:14 | |
| *** mrmoje has joined #openstack-swift | 23:15 | |
| *** zhill has quit IRC | 23:15 | |
| kota_ | tsg: https://review.openstack.org/#/c/266618/ | 23:15 |
| kota_ | tsg: upstream docs already updated :) | 23:16 |
| tsg | kota_: yep I saw that yesterday (which is why I didn't push my doc changes right away) | 23:16 |
| *** dmorita has joined #openstack-swift | 23:17 | |
| tsg | kota_: there are some more updates to do with removal of jerasure references | 23:17 |
| *** dmorita has quit IRC | 23:17 | |
| kota_ | Ah, yeah gotcha | 23:17 |
| *** dmorita has joined #openstack-swift | 23:18 | |
| tsg | kota_: I am moving jerasure to just the samples and also adding an ISA-L example | 23:18 |
| kota_ | Cool | 23:19 |
| notmyname | has anyone played with locking cpu frequencies on swift nodes? | 23:19 |
| *** dmorita has quit IRC | 23:19 | |
| *** dmorita has joined #openstack-swift | 23:20 | |
| *** gyee has joined #openstack-swift | 23:21 | |
| *** diogogmt has quit IRC | 23:23 | |
| *** dmorita has quit IRC | 23:23 | |
| *** dmorita has joined #openstack-swift | 23:25 | |
| *** mrmoje has quit IRC | 23:33 | |
| openstackgerrit | dangming proposed openstack/swift: Fix "--id" is used as a key https://review.openstack.org/267308 | 23:34 |
| openstackgerrit | dangming proposed openstack/swift: Fix "--id" is used as a key https://review.openstack.org/267308 | 23:36 |
| *** forcebanana has joined #openstack-swift | 23:36 | |
| pchng | tsg: No problem :) | 23:36 |
| *** mragupat has quit IRC | 23:39 | |
| *** mingdang1 has quit IRC | 23:41 | |
| *** mrmoje has joined #openstack-swift | 23:42 | |
| Zyric | notmyname: appears resetting swift didn't help with the failing probetests http://paste.openstack.org/show/483943/ That's all the output it gave me, repeat tries show more of the same. Now functests aren't running at all now with a "local variable 'run_subunit_content' referenced before assignment" error, but that wasn't happening 15 minutes ago. Wish I knew what was going on. | 23:44 |
| *** niya has quit IRC | 23:46 | |
| *** mrmoje has quit IRC | 23:55 | |
| *** bsdkurt has quit IRC | 23:55 | |
| *** bsdkurt has joined #openstack-swift | 23:55 | |
| *** bsdkurt has quit IRC | 23:56 | |
| *** bsdkurt has joined #openstack-swift | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!