How Connections (Overlay) Works Today
Here's how the overlay works today:
- When first starting stellar-core, it stores all nodes in
KNOWN_PEERS in the peer database as not preferred (false), and all nodes in
PREFERRED_PEERS in the peer database as preferred (true).
- When starting to make connections, a stellar-core node always attempts to connect to its preferred peers in the peer database first. If it hits the
TARGET_PEER_CONNECTIONS (includes both inbound and outbound connections), it doesn't attempt to make any more connections.
- If the number of connections (inbound & outbound) is <
TARGET_PEER_CONNECTIONS, then it starts making connections to other known nodes in the DB. This can be from pre-populating this persistent DB using
KNOWN_PEERS, but it adds more peers via messages from other nodes (that say their list of known peers).
- It keeps doing this every few seconds to keep active connections open to other stellar-core nodes.
There are two other aspects to this:
PREFERRED_PEER_KEYS - whenever you come across a node to connect to, it will check if it is preferred. If there isn't info on whether its IP is already preferred, and the key (node ID) matches one of the keys in
PREFERRED_PEER_KEYS, the node will then be marked as preferred. This is useful if you don't know the exact IP address of a given node, but you still want to mark it as preferred.
PREFERRED_ONLY - this only allows you to connect to nodes you've marked as preferred - any connections made to non-preferred nodes will drop and back-off.
In Summary/Admin's Perspective
- Put node ID's/public keys of nodes in your quorum set in
- Put IP addresses of nodes you know and trust, or simply want to have priority in connections (potentially duplicating the keys you've already specified in
- Put any other node you know of in
KNOWN_PEERS that shouldn't get special treatment - this is most important when first starting stellar-core to bootstrap the connection process.
- If you know of nodes you already trust and want to have a preference to connect to them (say nodes in your quorum set), you should put them in
- For bootstrapping stellar-core for the first time, if you have additional nodes that have been forwarded to you but you don't want them to have preferential treatment, you add them to
KNOWN_PEERS. As you gain more peers in the database via messages, this configuration point becomes less important - its primary purpose is to bootstrap stellar-core when it's first connecting to the network.
- For nodes in your quorum set that you have the keys for, you should put them in
PREFERRED_PEERS_KEYS. This gives you a level of redundancy if you have already put these nodes in
PREFERRED_PEERS, because if the IP address ever changes you can still find those nodes, have them marked as preferred, and not break consensus because you can't find them via their new IP.
PREFERRED_ONLY should only be used if you wish to restrict your connections to those you've explicitly specified as preferred, and to ignore messages from nodes about other nodes to connect to.
Explicit Answers To Questions
Should I use KNOWN_PEERS or PREFERRED_PEERS? The docs mention both.
You should use both -
KNOWN_PEERS for bootstrapping for the first time,
PREFERRED_PEERS for establishing preferential status for nodes (which is important for nodes in your quorum set).
What exactly constitutes a dependency?
A dependency in this context means a node in your quorum set that you're dependent on to reach consensus.
Why should I use PREFERRED_PEER_KEYS instead of PREFERRED_PEERS for quorum sets? Wouldn't PREFERRED_PEERS be better because it already contains ip:port so my node wouldn't need to resolve public key to ip:port?
PREFERRED_PEERS is marginally faster, but having the keys as well will make your node more robust to IP address changes. We recommend using both corresponding to the same nodes when possible.
The connection strategy is currently being overhauled. Of note is that incoming and outgoing connections will be counted separately, where
TARGET_PEER_CONNECTIONS will limit outbound connections, but currently
MAX_PENDING_CONNECTIONS will limit inbound connections. When this gets merged in, we'll update the docs accordingly (otherwise, do ping us about it!).