*** henriqueof has quit IRC | 00:00 | |
*** ianychoi has joined #openstack-swift | 00:19 | |
*** gyee has quit IRC | 00:38 | |
*** rcernin has quit IRC | 01:27 | |
*** rcernin has joined #openstack-swift | 01:27 | |
*** BjoernT has joined #openstack-swift | 03:14 | |
*** BjoernT has quit IRC | 03:19 | |
*** BjoernT has joined #openstack-swift | 03:20 | |
*** mugsie has quit IRC | 03:22 | |
*** dosaboy has quit IRC | 03:22 | |
*** szaher has quit IRC | 03:22 | |
*** mvkr has quit IRC | 03:22 | |
*** mgagne has quit IRC | 03:22 | |
*** dasp has quit IRC | 03:22 | |
*** guimaluf has quit IRC | 03:22 | |
*** d34dh0r53 has quit IRC | 03:22 | |
*** psachin has joined #openstack-swift | 03:31 | |
openstackgerrit | zhufl proposed openstack/swift master: Fix :param: in docstring https://review.opendev.org/665284 | 03:33 |
---|---|---|
*** mugsie has joined #openstack-swift | 03:37 | |
*** dosaboy has joined #openstack-swift | 03:37 | |
*** szaher has joined #openstack-swift | 03:38 | |
*** mvkr has joined #openstack-swift | 03:38 | |
*** mgagne has joined #openstack-swift | 03:38 | |
*** dasp has joined #openstack-swift | 03:38 | |
*** guimaluf has joined #openstack-swift | 03:38 | |
*** d34dh0r53 has joined #openstack-swift | 03:38 | |
*** irclogbot_2 has quit IRC | 03:41 | |
openstackgerrit | Merged openstack/swift master: Update mailmap for Kota Tsuyuzaki https://review.opendev.org/664200 | 03:41 |
*** irclogbot_2 has joined #openstack-swift | 03:42 | |
*** dasp has quit IRC | 03:58 | |
*** dasp has joined #openstack-swift | 04:21 | |
openstackgerrit | Merged openstack/swift feature/losf: Merge branch 'master' into feature/losf https://review.opendev.org/665170 | 04:23 |
*** pcaruana has joined #openstack-swift | 05:09 | |
*** e0ne has joined #openstack-swift | 05:21 | |
*** BjoernT has quit IRC | 05:33 | |
*** takamatsu has joined #openstack-swift | 06:34 | |
*** viks__ has joined #openstack-swift | 06:57 | |
*** rdejoux has joined #openstack-swift | 07:07 | |
*** tesseract has joined #openstack-swift | 07:08 | |
viks__ | I see the following in my swift storage node logs at times and fails gets automatically resolved: | 07:10 |
viks__ | ``` | 07:10 |
viks__ | object-replicator: 229243 successes, 1 failures | 07:10 |
viks__ | object-replicator: 212547 successes, 15 failures | 07:10 |
viks__ | object-replicator: 229225 successes, 19 failures | 07:10 |
viks__ | object-replicator: 211476 successes, 24 failures | 07:10 |
viks__ | object-replicator: 229219 successes, 25 failures | 07:10 |
viks__ | object-replicator: 211190 successes, 2 failures | 07:10 |
viks__ | object-replicator: 229241 successes, 3 failures | 07:10 |
viks__ | ``` | 07:10 |
viks__ | But i'm unable to find what is causing this failure. Is it expected. Can someone help me how to troubleshoot this issue? | 07:10 |
*** rcernin has quit IRC | 07:22 | |
*** e0ne has quit IRC | 07:35 | |
*** ccamacho has joined #openstack-swift | 08:30 | |
*** tkajinam has quit IRC | 08:38 | |
*** e0ne has joined #openstack-swift | 09:08 | |
*** tdasilva has quit IRC | 10:04 | |
*** psachin has quit IRC | 10:37 | |
*** psachin has joined #openstack-swift | 10:39 | |
*** tdasilva has joined #openstack-swift | 12:46 | |
*** ChanServ sets mode: +v tdasilva | 12:46 | |
*** BjoernT has joined #openstack-swift | 13:25 | |
*** tdasilva has quit IRC | 13:32 | |
*** hseipp has joined #openstack-swift | 13:34 | |
*** mrjk has joined #openstack-swift | 14:02 | |
*** tdasilva has joined #openstack-swift | 14:06 | |
*** ChanServ sets mode: +v tdasilva | 14:06 | |
*** BjoernT has quit IRC | 14:08 | |
*** e0ne has quit IRC | 14:13 | |
*** edausq has quit IRC | 14:16 | |
*** baojg has joined #openstack-swift | 14:18 | |
*** hseipp has quit IRC | 14:18 | |
*** hseipp has joined #openstack-swift | 14:19 | |
openstackgerrit | Thiago da Silva proposed openstack/swift master: Enable s3api in saio docker container https://review.opendev.org/665403 | 14:20 |
*** tdasilva has quit IRC | 14:22 | |
*** tdasilva has joined #openstack-swift | 14:22 | |
*** ChanServ sets mode: +v tdasilva | 14:22 | |
timburke | viks__, most likely it's just that the remote nodes were overloaded and replication hit a timeout. to confirm that, look for messages in the replicator log like "Error syncing handoff partition" or "Error syncing partition" | 14:52 |
timburke | something like "Exception in top-level replication loop" would be more troubling | 14:52 |
*** e0ne has joined #openstack-swift | 14:57 | |
*** e0ne has quit IRC | 15:00 | |
timburke | basically, as long as there don't seem to be persistent failures, it's normal operations and nothing to worry about | 15:00 |
*** mrjk has quit IRC | 15:02 | |
*** mrjk has joined #openstack-swift | 15:02 | |
timburke | (man, now i want to track time-since-last-successful-sync on a per-partition basis -- looking at outliers would probably be interesting) | 15:03 |
viks__ | timburke: I'm not seeing any of the above mentioned error | 15:03 |
*** baojg has quit IRC | 15:06 | |
viks__ | but it resolves on it's own after many iterations | 15:06 |
viks__ | when checked with `/recon/replication/object` endpoint, it says rsync error.. but rsync log also does not show any error. | 15:13 |
*** e0ne has joined #openstack-swift | 15:15 | |
*** e0ne has quit IRC | 15:21 | |
openstackgerrit | Tim Burke proposed openstack/swift master: Fix up how we memcache on py3 https://review.opendev.org/662115 | 15:21 |
openstackgerrit | Tim Burke proposed openstack/swift master: py3: Port the tempurl middleware https://review.opendev.org/652928 | 15:21 |
*** e0ne has joined #openstack-swift | 15:24 | |
*** tdasilva has quit IRC | 15:26 | |
*** e0ne has quit IRC | 15:27 | |
*** e0ne has joined #openstack-swift | 15:29 | |
*** e0ne has quit IRC | 15:33 | |
*** takamatsu has quit IRC | 16:07 | |
*** hseipp has quit IRC | 16:23 | |
*** hseipp has joined #openstack-swift | 16:43 | |
*** hseipp has quit IRC | 17:08 | |
*** e0ne has joined #openstack-swift | 17:41 | |
openstackgerrit | Clay Gerrard proposed openstack/swift master: Allow "harder" symlinks https://review.opendev.org/633094 | 17:42 |
*** guimaluf has quit IRC | 17:42 | |
*** e0ne has quit IRC | 17:46 | |
*** rdejoux has quit IRC | 18:08 | |
*** psachin has quit IRC | 18:08 | |
*** tesseract has quit IRC | 18:18 | |
*** e0ne has joined #openstack-swift | 18:42 | |
*** e0ne has quit IRC | 19:16 | |
*** e0ne has joined #openstack-swift | 19:16 | |
*** e0ne has quit IRC | 19:21 | |
*** e0ne has joined #openstack-swift | 19:23 | |
*** e0ne has quit IRC | 19:33 | |
*** viks__ has quit IRC | 19:36 | |
*** pcaruana has quit IRC | 19:39 | |
openstackgerrit | Tim Burke proposed openstack/swift master: py3: port the test of locale https://review.opendev.org/665042 | 20:31 |
timburke | 🤮 | 20:33 |
openstackgerrit | Tim Burke proposed openstack/swift master: py3: Specify an encoding when loading db.pending pickles https://review.opendev.org/665486 | 22:44 |
openstackgerrit | Tim Burke proposed openstack/swift master: Stop sending non-UTF-8 prefixes in func tests https://review.opendev.org/665487 | 22:45 |
timburke | on ^^^ there are probably others -- i think we'll be playing whack-a-mole for a while as we do dev work and receive bug reports against py3 | 22:46 |
timburke | er, vvv | 22:47 |
openstackgerrit | Tim Burke proposed openstack/swift master: Move calls to self.app outside of error handling https://review.opendev.org/665488 | 22:47 |
openstackgerrit | Tim Burke proposed openstack/swift master: Move call to global_conf_callback after loadapp() https://review.opendev.org/665494 | 23:20 |
timburke | i feel like ^^^ needs some functional validation that replication_concurrency is still respected | 23:20 |
timburke | (sorry for the flurry of activity -- i'm trying to break up https://review.opendev.org/#/c/663794/ into independent patches that each make sense on their own) | 23:22 |
patchbot | patch 663794 - swift - py3: get func tests passing - 1 patch set | 23:22 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!