| Voidwalker | I believe I've found a solution to update the missing container listing information. It seems that by recovering the container and object names with swift-object-info, we can then just copy the file onto itself, and it will preserve all the file (meta)data and add it to the listing. | 02:22 |
|---|---|---|
| Voidwalker | Once disk space is sorted, we're a simple bash script away from having that all back. | 02:25 |
| opendevreview | OpenStack Proposal Bot proposed openstack/swift master: Imported Translations from Zanata https://review.opendev.org/c/openstack/swift/+/886694 | 03:37 |
| opendevreview | Philippe SERAPHIN proposed openstack/swift master: Adjust column width https://review.opendev.org/c/openstack/swift/+/886704 | 04:45 |
| opendevreview | Philippe SERAPHIN proposed openstack/swift master: Add Major:Minor detection https://review.opendev.org/c/openstack/swift/+/886733 | 09:48 |
| reid_g | Good morning. I have a question about https://opendev.org/openstack/swift/src/tag/2.29.2/swift/common/wsgi.py#L210 . What does it solve? I keep running into issues where we stop object and start it again and end up getting messages that some of the ports are already in use (port per device config). The only way I have found to fix it is to reboot the server. Sometimes we get | 12:53 |
| reid_g | the message after a fresh reboot as the services are starting. It is happening in Ussuri and Yoga. | 12:53 |
| opendevreview | ASHWIN A NAIR proposed openstack/swift master: format sysmeta in swift-container-info https://review.opendev.org/c/openstack/swift/+/881936 | 16:20 |
| timburke | reid_g, does turning off keepalive help with that? what version of eventlet are you running? it should be setting SO_REUSEPORT since 0.20.0 back in 2016, so i would've thought the rebind should be fine... what user is starting the main worker-management process? do you have something different configured as `user` in your conf file? | 16:46 |
| timburke | is it just the object server that exhibits this, or do you see it with the other servers, too? | 16:48 |
| opendevreview | ASHWIN A NAIR proposed openstack/swift master: combine duplicated code in replication and EC GET paths https://review.opendev.org/c/openstack/swift/+/886645 | 17:54 |
| opendevreview | ASHWIN A NAIR proposed openstack/swift master: combine duplicated code in replication and EC GET paths https://review.opendev.org/c/openstack/swift/+/886645 | 17:58 |
| opendevreview | ASHWIN A NAIR proposed openstack/swift master: unify duplicate code in replication, EC GET paths https://review.opendev.org/c/openstack/swift/+/886645 | 18:06 |
| opendevreview | Tim Burke proposed openstack/swift master: py3: Quiet RemoteDisconnected tracebacks https://review.opendev.org/c/openstack/swift/+/886780 | 18:39 |
| opendevreview | ASHWIN A NAIR proposed openstack/swift master: unify duplicate code in replication, EC GET paths https://review.opendev.org/c/openstack/swift/+/886645 | 18:56 |
| reid_g | I really only see the issue with swift-object. I will check the other questions tomorrow. I got a bit distracted today and have to run to pickup my kid now. | 20:12 |
| timburke | no worries, i know how that is :-) | 20:18 |
| timburke | Voidwalker / paladox copying data back on itself should work, provided (1) you don't need to worry about overwrites/consistency issues, (2) you don't need to try to preserve last-modified times, and (3) you don't mind incurring a bunch of disk I/O to copy the data | 20:29 |
| paladox | timburke: thanks for weight suggestion, changing it to match disk space seems to have fixed the out of storage. | 20:30 |
| paladox | now seems to load balance much better | 20:30 |
| timburke | given the single replica and the small cluster, (1) probably isn't much of an issue. (3) makes me nervous given your disk-full state, though | 20:30 |
| timburke | yay! glad to hear it | 20:30 |
| timburke | another option would be to just write down fresh async pendings for everything using something like https://paste.opendev.org/show/bdHCdLOoJQoNAPC0pLyL/ -- though you'd want to up your reclaim age a bunch to make sure the updater actually processed all the ancient asyncs | 20:31 |
| paladox | i lowered the reclaim age from the default 7 days to 6 hours as 7 days was too long to keep around objects we deleted and we don't store more then one copy so it was pointless as my understanding is the feature is for replicators incase one went out temporarily and rejoined | 20:33 |
| timburke | paladox, yes -- that seems appropriate for resolving the disk-full in particular -- *after* that, when we're fixing the objects not appearing in listings, it may be worth increasing it (depending on how you go about fixing it) | 20:38 |
| paladox | oh | 20:39 |
| opendevreview | Tim Burke proposed openstack/swift master: py3: Quiet RemoteDisconnected tracebacks https://review.opendev.org/c/openstack/swift/+/886780 | 20:46 |
| opendevreview | Merged openstack/swift master: Green GreenDBConnection.execute https://review.opendev.org/c/openstack/swift/+/866051 | 22:27 |
| paladox | https://www.irccloud.com/pastebin/EFDO7YgY/ | 23:21 |
| paladox | not sure why i get that as swift list works. | 23:21 |
| paladox | timburke: any idea what could cause ^? | 23:23 |
| paladox | oh, think i found it. Wish the error was a bit more descriptive | 23:30 |
| paladox | swift list -l --debug helped | 23:30 |
| paladox | https://www.irccloud.com/pastebin/DR2KE7Zc/ | 23:30 |
| paladox | fixed that with a swift post. | 23:31 |
| kota | good morning | 23:59 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!