I will try to answer with what I understand from setting up a production ready private network:
- Make sure you setup your history so that it is available to the local core and other cores in you network. From here:
A common configuration is for each peer in a group to have a single history archive that it knows how to write to, while also knowing how to read from all archives in the group.
I translated this to mean that each core in my network should put and get locally. That history archive is then also served over HTTP so it is available to other core's. Since all of the core's in our private network are full validators this makes sense to me and provides plenty of redundancy.
- I am planning on using the outputs of
stellar-core http-command info (in particular the state field) and stellar-core http-command quorum to ensure everything is in order. These are also available using curl from localhost (or elsewhere on your network if you have set the configuration option PUBLIC_HTTP_PORT=true
3 & 4. I believe this is the same problem I had when configuring and testing a private network. You probably haven't setup and/or persisted (you did not mention if core is installed on bare metal/VM/container) the buckets correctly. See Graydon's answer here.
If you are using Docker for example you need to persist the following:
- Database
- History Archive
- BUCKET_DIR_PATH
If you do this on each core and also serve the History Archive to the other cores in the validating group you should not lose any data.
