| opendevreview | Matthew Oliver proposed openstack/swift master: WIP - db: shard up the DatabaseBroker pending files https://review.opendev.org/c/openstack/swift/+/830551 | 06:20 |
|---|---|---|
| Nicolas | Hello Swift peeps ! A quick question for you folks if you have the time. We are managing a rather important cluster and we start to face an issue where the container db size start to be quite big for come customers ( ~ 100GB) which is triggering a lot of side issues on the rebalance. (lack of space, delay of the replication for example) | 14:38 |
| Nicolas | We know that container-sharding is there in our version (Stein 2.21.0) and we plan to implement it but before that we would like to make sure it's not possible to compact / prune a container db with existing tools. We asked customers to move / delete their older set of datas to other tiers, but it doesn't seems to reduce the size of the container db sadly. Is is expected ? | 14:40 |
| zaitcev | I only have a very limited experience with a situation like this, but yes, Swift does not vacuuum the databases. However, notice that the deleted entries stay in the DB for a week while tombstones are around. It's known as reclaim time. | 14:45 |
| zaitcev | So, objects are only marked deleted when they are deleted. A week later they are deleted from the database. After they are deleted, the responsiveness of the database must improve despite the large size. | 14:46 |
| zaitcev | You can see it in how long it takes to list (GET). | 14:46 |
| zaitcev | Until then, vacuuming will not reduce the size. | 14:47 |
| zaitcev | I mean, manual vacuuming. | 14:47 |
| Nicolas | Ok I see, thanks Zaitvec ! | 15:25 |
| Nicolas | zaitcev sorry :) | 15:25 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!