4

I'm running a core. It's generating a lot of files in its internal bucket (not the history bucket): files that contain the recent-ish history of the blockchain.

Is there a way to limit this amount? Do I need to keep increasing my HD ad-infinitum?

We're still not sure why it's growing. Our graphs show that all cores behave about the same, in terms of free space (in other words, it isn't an isolated node). In addition, it's obvious that while the general direction is to use more disc space, occasionally the amount of free space increases (across all nodes) by as much as 2GB. This might reflect a larger volume of transactions on the blockchain, and consequently larger files on the HD, until they are published to the history bucket (on S3).

  • not sure about which bucket you are referring to. if it is the one containing something like "bucket-31021af70d48df39ff770b8dfa48e62e5b1610f7d121c62c7c0b083573d3a23b.xdr.gz", it should not be enlarging. – cesarm Jan 22 at 2:22
  • yup, thats the one - and it is growing in size - a few G's over the last 60 days. wondering if its just local history that didnt get published to the history bucket. – FuzzyAmi Jan 22 at 8:45
  • for me, there're only three things (.xdr.gz, .xdr & tmp/) in this directory...the gzip & a small folder, which are in total around 1MB...so i suspect your claim correct – cesarm Jan 23 at 2:44
  • could you post your stellar-core.cfg to see if some settings going strange? – cesarm Jan 23 at 2:58
3

Author of the bucket subsystem here.

The local buckets are a replica (with a different and unfortunately very necessary structure) of the contents of the active SQL database. They exist to enable constant-time hashing of the state of the ledger, as well as fast delta-based catchup. In general, since they mirror the contents of the SQL database, they will grow at about the same rate as the SQL database, which is not super fast -- they do not store transaction history, just the current state of the ledger -- but over time yes they will continue to grow as the ledger gains more accounts.

If the bucket directory's stable state (absent the temporary size spikes) is not roughly similar to the size of the SQL database, it's possible there's some sort of error in history publication that's causing a backlog of un-garbage-collected buckets. Usually obsolete buckets get deleted. It might be helpful to see a stellar-core.cfg file in that case.

The temporary spikes you see in space usage are just that: temporary. Usually they will last minutes-to-hours at most. That is by design. The buckets are maintained in a log-structured merge tree, and that requires periodically merging two of the bucket files, on a background thread, to form a third (and then deleting the inputs). When such a merge happens, disk space will temporarily grow by as much as twice the largest bucket size. Bucket sizes are distributed exponentially, with the largest bucket 4x as big as the next-largest. So yes, seeing periodic +/- 2GB size spikes in the bucket directory (given the bucket directory is a few GB itself) is expected, but those spikes are not going to grow orders of magnitude larger than the overall size of the directory.

Finally: in protocol 11 we made some significant changes to the bucket compaction code, so once the network transitions to protocol 11 the buckets will gradually get a fair bit smaller for several months until they're all properly compacted. But this should be understood as a one-time win: the ledger is likely to continue to (slowly) grow over time and the buckets storing it will grow accordingly.

  • Superb answer. Thank you! Back when I asked about this issue I had a limited understanding of the buckets but have since learned a thing or two (from you :]). At the time we had encountered an issue where buckets were not publishing which (as you mentioned) can cause the size of the buckets to grow significantly. – FuzzyAmi May 12 at 5:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.