Limiting the size of local buckets on the core

Question

I'm running a core. It's generating a lot of files in its internal bucket (not the history bucket): files that contain the recent-ish history of the blockchain.

Is there a way to limit this amount? Do I need to keep increasing my HD ad-infinitum?

We're still not sure why it's growing. Our graphs show that all cores behave about the same, in terms of free space (in other words, it isn't an isolated node). In addition, it's obvious that while the general direction is to use more disc space, occasionally the amount of free space increases (across all nodes) by as much as 2GB. This might reflect a larger volume of transactions on the blockchain, and consequently larger files on the HD, until they are published to the history bucket (on S3).

Answer 1

Author of the bucket subsystem here.

The local buckets are a replica (with a different and unfortunately very necessary structure) of the contents of the active SQL database. They exist to enable constant-time hashing of the state of the ledger, as well as fast delta-based catchup. In general, since they mirror the contents of the SQL database, they will grow at about the same rate as the SQL database, which is not super fast -- they do not store transaction history, just the current state of the ledger -- but over time yes they will continue to grow as the ledger gains more accounts.

If the bucket directory's stable state (absent the temporary size spikes) is not roughly similar to the size of the SQL database, it's possible there's some sort of error in history publication that's causing a backlog of un-garbage-collected buckets. Usually obsolete buckets get deleted. It might be helpful to see a stellar-core.cfg file in that case.

The temporary spikes you see in space usage are just that: temporary. Usually they will last minutes-to-hours at most. That is by design. The buckets are maintained in a log-structured merge tree, and that requires periodically merging two of the bucket files, on a background thread, to form a third (and then deleting the inputs). When such a merge happens, disk space will temporarily grow by as much as twice the largest bucket size. Bucket sizes are distributed exponentially, with the largest bucket 4x as big as the next-largest. So yes, seeing periodic +/- 2GB size spikes in the bucket directory (given the bucket directory is a few GB itself) is expected, but those spikes are not going to grow orders of magnitude larger than the overall size of the directory.

Finally: in protocol 11 we made some significant changes to the bucket compaction code, so once the network transitions to protocol 11 the buckets will gradually get a fair bit smaller for several months until they're all properly compacted. But this should be understood as a one-time win: the ledger is likely to continue to (slowly) grow over time and the buckets storing it will grow accordingly.