We want to setup Stellar Full Archive with Horizon
Using the https://github.com/satoshipay/stellar-core-parallel-catchup we managed to synchronized Stellar Core DB and build full archive.
After lunching new instance, from the gathered data we lunched Stellar Horizon. But Horizon was only able to serve only new leadgers (those, which one wasn't been in snapshot).
So after some googling we run
stellar-horizon db reingest start end, in parallel way - 4 concurrent threads, chunk size 10k, but the performance is really bad. After one day of work, we estimate that the whole process will take about 100 days. Which is not acceptable for us.
Could anyone kindly advice which parameters should by tunned (for horizon, postgresql or even OS)? CPU is utillized in about 30%, %iowait 6% to 24%, Memory usage 3GB of 16GB.
Is it possible, to reingest horizon on separate environment in parallel way and merge data to the production environment (something simillar like for Stellar Core DB and History Archive)? Which tables need to be coopied?