3

I am trying to set up a private stellar cluster with at least 3 nodes. I am able to setup a 2-nodes cluster but when it comes to add another node, the cluster is broken and only one node is synced.

Here is how I configured my 2-nodes cluster.

I started a docker container on each node by running :

docker run --rm -d -p "80:80" -p "8000:8000" -p "11625:11625" -p "11626:11626" -v "/home/ubuntu/stellar:/opt/stellar" --name private-cluster stellar/quickstart --standalone

I logged into the node on 101, stop stellar-core, recreated core database and updated stellar-core.cfg with :

HTTP_PORT=11626
PUBLIC_HTTP_PORT=true

NETWORK_PASSPHRASE="MyStandalone Network ; December 2018"

NODE_SEED="SAOQ42SRGOLSVRGJI2BCX4F5PEI3QA3IWN55GCFHQ5PIMJ2CJQTBNCGQ"
NODE_IS_VALIDATOR=true
PREFERRED_PEERS=["192.168.8.102"]

DATABASE="postgresql://dbname=core host=localhost user=stellar password=mypassword"

COMMANDS=["ll?level=debug"]

FAILURE_SAFETY=0
UNSAFE_QUORUM=true
[QUORUM_SET]
THRESHOLD_PERCENT=50
VALIDATORS=["GCYJMPMSMOINIXGLYA7ID45GJ2N46U7LMHFOTYKWHHNBLAA4K6X4FYGE", "GDMEEIAG6KJQAN5RRQHRAB6BDRDJKKPTTTJDQ4WSKEZITQ7S63QGELN7"]

[HISTORY.vs]
get="cp /opt/stellar/core/history/vs/{0} {1}"
put="./cp-and-set-permissions.sh {0} /opt/stellar/core/history/vs/{1}"
mkdir="./mkdir-and-set-permissions.sh -p /opt/stellar/core/history/vs/{0}"

Then I cleaned stellar db :

sudo -u stellar stellar-core --newdb --conf etc/stellar-core.cfg

Moreover I created the history with :

sudo -u stellar stellar-core --newhist vs --conf etc/stellar-core.cfg

I ran :

sudo -u stellar stellar-core --forcescp --conf etc/stellar-core.cfg

Then started stellar-core :

supervisorctl start stellar-core

For the node on 102, I stopped stellar-core, recreated the core database and edited the configuration file like this :

HTTP_PORT=11626
PUBLIC_HTTP_PORT=true

NETWORK_PASSPHRASE="MyStandalone Network ; December 2018"

NODE_SEED="SB6O2IMZJH63X4XUHBKUECMBFYB4HAT32G4ALBGNSVFWJLICGHPRQWQD"
NODE_IS_VALIDATOR=true
PREFERRED_PEERS=["192.168.8.101"]

DATABASE="postgresql://dbname=core host=localhost user=stellar password=mypassword"

COMMANDS=["ll?level=debug"]

FAILURE_SAFETY=0
UNSAFE_QUORUM=true
[QUORUM_SET]
THRESHOLD_PERCENT=50
VALIDATORS=["GCYJMPMSMOINIXGLYA7ID45GJ2N46U7LMHFOTYKWHHNBLAA4K6X4FYGE", "GDMEEIAG6KJQAN5RRQHRAB6BDRDJKKPTTTJDQ4WSKEZITQ7S63QGELN7"]

[HISTORY.vs]
get="cp /opt/stellar/core/history/vs/{0} {1}"
put="./cp-and-set-permissions.sh {0} /opt/stellar/core/history/vs/{1}"
mkdir="./mkdir-and-set-permissions.sh -p /opt/stellar/core/history/vs/{0}"

[HISTORY.h1]
get="curl -sf http://192.168.8.101/{0} -o {1}"

I didn't forcescp on 102 as I wanted it to join the cluster on 101. I just cleaned stellar-core db, created my history archive and started stellar-core.

The node on 101 get synced automatically, whereas the one on 102 went to "Joining SCP", then "Catching Up", then "Getting history" and finally "Synced". I did some transfers and when I checked the root account balance it was updated on both nodes. Thus, I had a successful running 2-nodes cluster.

I tried to add one more node in this cluster following the same pattern but whatever configuration I use nodes are not all synced or neither share the same root account balance.

Any help would be welcome.

7
  • what do you mean by broken? what is the state of the 2 other cores? run curl localhost:11626/info on the cores to get the state
    – FuzzyAmi
    Dec 23, 2018 at 7:54
  • What I mean by broken is the others nodes are stuck in "Joining SCP" state
    – Parker
    Dec 23, 2018 at 19:25
  • What do you mean by "other nodes"? Could you list the states of 101, 102, 103 respectively? Both before and after the changes you made, and also what changes you have made.
    – cesarm
    Dec 24, 2018 at 1:55
  • I updated my question with steps I went through when I was setting up my 2-nodes cluster.
    – Parker
    Dec 26, 2018 at 18:21
  • THRESHOLD_PERCENT=50 is not a good config. You need something >50%, e.g. 51%.
    – cesarm
    Dec 27, 2018 at 2:44

1 Answer 1

0

You can use this docker-compose.yml. Its basically fork of stellar-quickstart adjusted to work on private clusters. It works in ephemeral mode, you don't have to mount any volume, you can shut it down and start again and it will start fresh each time. Additionally, it supports confd configuration, so you can just pass stellar-core.cfg parameters using environment variables.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.