4

I am seeing plenty of these warnings while my node is catching up with CATCHUP_COMPLETE.

2018-10-27T09:32:08.197 GADLA [Herder INFO] Quorum information for 20686064 : {"agree":18,"disagree":0,"fail_at":3,"hash":"0c093d","missing":6,"phase":"EXTERNALIZE","validated":false}

2018-10-27T09:32:08.197 GADLA [Herder WARNING] Ledger 20686066 (8b2723) closed and could NOT be fully validated by validator
2018-10-27T09:32:08.216 GADLA [Ledger INFO] Got consensus: [seq=20686066, prev=15a7dd, tx_count=10, sv: [  txH: 8b2723, ct: 1540632097, upgrades: [ ] ]]

Why is my node not validating?

I guess it's obviously because I am still catching up and it did not get to the most recent ledgers yet.

2018-10-27T11:51:57.502 GADLA [History INFO] Catching up: applying checkpoint 27578/207756 (13%)

Will validation begin once I reach synced state? It will still take a few days to catch up and I am curious to find out if there's a different problem in the meanwhile.

Hope someone more experienced can chime in here.

Below is my stellar-core.cfg

NODE_SEED="SXXX"
NODE_IS_VALIDATOR=true
CATCHUP_COMPLETE=true
NETWORK_PASSPHRASE="Public Global Stellar Network ; September 2015"
LOG_FILE_PATH="/data/log"
BUCKET_DIR_PATH="/data/buckets"
DATABASE="postgresql://dbname=core host=dockerhost user=XXX password=XXX"
FAILURE_SAFETY=1
NODE_NAMES=[

    "GCGB2S2KGYARPVIA37HYZXVRM2YZUEXA6S33ZU5BUDC6THSB62LZSTYH sdf1",
    "GCM6QMP3DLRPTAZW2UZPCPX2LF3SXWXKPMP3GKFZBDSF3QZGV2G5QSTK sdf2",
    "GABMKJM6I25XI4K7U6XWMULOUQIQ27BCTMLS6BYYSOWKTBUXVRJSXHYQ sdf3",

    "GADLA6BJK6VK33EM2IDQM37L5KGVCY5MSHSHVJA4SCNGNUIEOTCR6J5T coinqvest-fi",
    "GD6SZQV3WEJUH352NTVLKEV2JM2RH266VPEM7EH5QLLI7ZZAALMLNUVN coinqvest-de",

    "GBX7KXY43NOGLCV7TMFFM7UPNM5VGQ52W4IAZNTVMXZ44BHYFESA67N3 cryptomover1",
    "GC7MH45NSXXPBLQJRSEVF2DFUVLGGYOJER5FRUNVCYVMXJYJT5LLQJW5 cryptomover2",
    "GCX7S2QY2VXRFDDVVGKRVSMIVGQZQ4NEDYZ3WB7ZUYIVJKMQ4FVVHVR6 cryptomover3",

    "GC5SXLNAM3C4NMGK2PXK4R34B5GNZ47FYQ24ZIBFDFOCU6D4KBN4POAE satoshipay1",
    "GBJQUIXUO4XSNPAUT6ODLZUJRV2NPXYASKUBY4G5MYP3M47PCVI55MNT satoshipay2",
    "GAK6Z5UVGUVSEK6PEOCAYJISTT5EJBB34PN3NOLEQG2SUKXRVV2F6HZY satoshipay3",

    "GDIQKLQVOCD5UD6MUI5D5PTPVX7WTP5TAPP5OBMOLENBBD5KG434KYQ2 stronghold1",
    "GA7MREQ7673YDVANF4WBPN7LBQM4BSH4BQUFUTC4YLSSQCQUQTXRVBZN stronghold2",
    "GDHV7FL7JP6LUEWWYUOA4C3QAFQ4LZ6OHFBWFIL6IFIS3AOBI2DHV62F stronghold3",
    "GBGDBLFKR3LORWOI65LVC7ES5OGZ4GHILEHCBVKPW2PMP2OL43F6B2JJ stronghold4",
    "GCBXBCIKCCVUIHAZ5QFWK6CKSX4AESOJ33IQNUE42BP7J66X23TM6WPF stronghold5",

    "GCKWUQGSVO45ZV3QK7POYL7HMFWDKWJVMFVEGUJKCAEVUITUCTQWFSM6 ibm-au",
    "GBUJA3Z5TLAKLI5MEH4TETLXJBQVSVW74MNEKP5UUHTP3IMLNSUPOTVA ibm-br",
    "GB2HF2NHRKKFZYFDGD7MUENOYROOEK7SWYV2APYOODP6P7BUJTLILKIL ibm-ca",
    "GBJ7T3BTLX2BP3T5Q4256PUF7JMDAB35LLO32QRDYE67TDDMN7H33GGE ibm-hk",
    "GCH3O5PTCZVR4G65W3B4XDKWI5V677HQB3QO7CW4YPVYDDFBE2GE7G6V ibm-in",
    "GAEEH4TBR7YQQWKJ2FIT57HXZZTMK2BX5LY4POJUYFSEZ7Y2ONHPPTES ibm-it",
    "GDRA72H7JWXAXWJKOONQOPH3JKNSH5MQ6BO5K74C3X6FO2G3OG464BPU ibm-no",
    "GAENPO2XRTTMAJXDWM3E3GAALNLG4HVMKJ4QF525TR25RI42YPEDULOW ibm-uk",
    "GARBCBH4YSHUJLYEPKEPMVYZIJ3ZSQR3QCJ245CWGY64X72JLN4A6RSG ibm-us",

    "GCBAQCWXKIIOQOEDIW6EB7GY6UREHPTKK26NZFLVK63ZTRFSXDVX2AHV smartlands",
    "GCJCSMSPIWKKPR7WEPIQG63PDF7JGGEENRC33OKVBSPUDIRL6ZZ5M7OO tempo",
    "GB7H5CNUNVCM6KGG6P2LAQE4YZP4D6CHFJRSSS34VNEPDDVIFAWRJ7ZA mobius",

    "GCFONE23AB7Y6C5YZOMKUKGETPIAJA4QOYLS5VNS4JHBGKRZCPYHDLW7 lobstr1",
    "GDXQB3OMMQ6MGG43PWFBZWBFKBBDUZIVSUDAZZTRAWQZKES2CDSE5HKJ lobstr2",

    "GASN57EFNZWME73BJXYZUTCD34EPX4KIIZQTQDTMBWWVH6JIZJUCBGQX goodx"


]

KNOWN_PEERS=[

    "core-live-a.stellar.org:11625",
    "core-live-b.stellar.org:11625",
    "core-live-c.stellar.org:11625",

    "finland.stellar.coinqvest.com:11625",
    "germany.stellar.coinqvest.com:11625",

    "cryptodealer.hk:11625",
    "cryptomover.org:11625",
    "cryptomover.net:11625",

    "stellar1.satoshipay.io:11625",
    "stellar2.satoshipay.io:11625",
    "stellar3.satoshipay.io:11625",

    "validator1.stellar.stronghold.co:11625",
    "validator2.stellar.stronghold.co:11625",
    "validator3.stellar.stronghold.co:11625",
    "validator4.stellar.stronghold.co:11625",
    "validator5.stellar.stronghold.co:11625",

    "au.stellar.ibm.com:11625",
    "br.stellar.ibm.com:11625",
    "ca.stellar.ibm.com:11625",
    "hk.stellar.ibm.com:11625",
    "in.stellar.ibm.com:11625",
    "it.stellar.ibm.com:11625",
    "no.stellar.ibm.com:11625",
    "uk.stellar.ibm.com:11625",
    "us.stellar.ibm.com:11625",

    "stellar.smartlands.io:11625",
    "stellar1.tempo.eu.com:11625",
    "us.stellar.mobius.network:11625",

    "v1.stellar.lobstr.co:11625",
    "v2.stellar.lobstr.co:11625",
    "de.stellar.goodx.network:11625"

]

[QUORUM_SET]
THRESHOLD_PERCENT=51
VALIDATORS=[
    "$sdf1",
    "$sdf2",
    "$sdf3"
]

[QUORUM_SET.1]
THRESHOLD_PERCENT=51
VALIDATORS=[
    "$coinqvest-fi",
    "$stronghold4",
    "$ibm-uk",
    "$satoshipay1",
    "$mobius",
    "$cryptomover2",
    "$ibm-no"
]

[QUORUM_SET.2]
THRESHOLD_PERCENT=51
VALIDATORS=[
    "$tempo",
    "$ibm-hk",
    "$stronghold2",
    "$ibm-br",
    "$cryptomover3",
    "$satoshipay3",
    "$stronghold3"
]

[QUORUM_SET.3]
THRESHOLD_PERCENT=51
VALIDATORS=[
    "$coinqvest-de",
    "$stronghold5",
    "$ibm-ca",
    "$satoshipay2",
    "$ibm-us",
    "$smartlands",
    "$cryptomover1"
]

[HISTORY.local]
get="cp /data/history/vs/{0} {1}"
put="cp {0} /data/history/vs/{1}"
mkdir="mkdir -p /data/history/vs/{0}"

# Stellar.org history store
[HISTORY.sdf1]
get="curl -sf http://history.stellar.org/prd/core-live/core_live_001/{0} -o {1}"

[HISTORY.sdf2]
get="curl -sf http://history.stellar.org/prd/core-live/core_live_002/{0} -o {1}"

[HISTORY.sdf3]
get="curl -sf http://history.stellar.org/prd/core-live/core_live_003/{0} -o {1}"

stellar-core --c info

{
   "info" : {
      "build" : "v10.0.0",
      "history" : {
         "local" : {
            "failure" : 0,
            "success" : 2236684
         },
         "sdf1" : {
            "failure" : 0,
            "success" : 215807
         },
         "sdf2" : {
            "failure" : 0,
            "success" : 215481
         },
         "sdf3" : {
            "failure" : 0,
            "success" : 215411
         }
      },
      "ledger" : {
         "age" : 47915002,
         "baseFee" : 100,
         "baseReserve" : 100000000,
         "closeTime" : 1492904919,
         "hash" : "2d23d2270126ad6252324973e13dc77e1a4970ab674a1946820ebe4b671ea71a",
         "maxTxSetSize" : 50,
         "num" : 10361495,
         "version" : 6
      },
      "network" : "Public Global Stellar Network ; September 2015",
      "peers" : {
         "authenticated_count" : 8,
         "pending_count" : 0
      },
      "protocol_version" : 10,
      "quorum" : {
         "20719315" : {
            "agree" : 14,
            "disagree" : 0,
            "fail_at" : 2,
            "hash" : "0c093d",
            "missing" : 10,
            "phase" : "EXTERNALIZE",
            "validated" : false
         }
      },
      "startedOn" : "2018-10-27T21:10:48Z",
      "state" : "Joining SCP",
      "status" : [
         "Catching up: applying checkpoint 161899/323349 (50%)",
         "Publishing 1346 queued checkpoints [10274239-10361471]: Retrying in 0 sec: prepare-snapshot"
      ]
   }
}
Improve this question
0

2 Answers 2

Reset to default
5

Yes you are correct: when a validator is only observing consensus it will emit this warning.

In your case the reason it’s only observing consensus is that your node is still catching up to the network. As it doesn’t have the current version of the ledger it cannot validate transactions that it sees.

As soon as the validator is fully caught up this should go away.

Improve this answer
0
0

The node is declared be validating (NODE_IS_VALIDATOR=true), but it is not doing work as it is not included inaccording to any of your [QUORUM_SET].

i.e. You have missed your own node "$self" in the [QUORUM_SET].

*

A direct solution could be changing your line

from NODE_SEED="SXXX" to NODE_SEED="SXXX my_node"

and then include "$my_node" in any one of [QUORUM_SET] / [QUORUM_SET.1] / [QUORUM_SET.2] / [QUORUM_SET.3].

Afterwards, stop the running Core, and then use commands like stellar-core --conf my_stellar.cfg --forcescp and then stellar-core --conf my_stellar.cfg to restart it, and it should be validating.

*

Effect (1) - NODE_SEED SXXXX is labeled as my_node

Effect (2) - $my_node represents the public key of SXXXX at [QUORUM_SET]

*

*

Explain here:

from the example config

The key for "self" is implicitely added at the top level, so the effective quorum set is [t:2, self, QUORUM_SET]. Note that "self" is always agreeing with the instance (if QUORUM_SET includes it)

*

Example Line 178 - making reference for long long key as "self"

NODE_SEED="SBI3CZU7XZEWVXU7OZLW5MMUQAP334JFOPXSLTPOH43IRTEQ2QYXU5RG self"

*

Example Line 418 - dereference "$self" as a long long key

[QUORUM_SET.1]
THRESHOLD_PERCENT=67
VALIDATORS=[
    "$self", # 'D' from above is this node
    "GDXJAZZJ3H5MJGR6PDQX3JHRREAVYNCVM7FJYGLZJKEHQV2ZXEUO5SX2 E_from_above",
    "GB6GK3WWTZYY2JXWM6C5LRKLQ2X7INQ7IYTSECCG3SMZFYOZNEZR4SO5 F_from_above"
]

Don't forget the dollar sign "$" at QUORUM_SET, as a deference sign.

Improve this answer
4
  • Thank you for looking into this! My node is actually one of the nodes listed in NODE_NAMES. I use the variable assigned in NODE_NAMES to include it in one of the quorum sets. I did not explicitly assign a variable in NODE_SEED. Do I have to do that? My node is still in catching up state: "Catching up: applying checkpoint 213896/323349 (66%)". Could it have something to do with my history not having caught up with the current and most recent ledgers yet? --c info shows ledger 13689309 but the network is already at 20744095. Should I just wait until catching up is complete and I have sync?
    – marcinx
    Commented Oct 31, 2018 at 0:46
  • The focus is [QUORUM_SET]..... You can either label your Secret Key SXXX under NODE_NAMES or NODE_SEED, or simply include the long long Public Key (corresponding to your SXXXX) under [QUORUM_SET]..... If [QUORUM_SET] has no information related to your node's identity (SEC key=SXXXX), your own node is not involved in doing actual validating work.
    – cesarm
    Commented Nov 1, 2018 at 9:23
  • Yeah, that makes sense. Thanks. Since my node is listed in the quorum set I think it has to do with the catching up state. Unfortunately it takes about 10 days to catch up and I hoped someone on here could provide some clarity instead of waiting that long :) We never ran a full history archiver before...
    – marcinx
    Commented Nov 1, 2018 at 9:55
  • I guess that taking over 1 week make sense, as it takes a few hours even for catching up to my 1-week private network. You could scale things up and imagine, as the public net has been there for years
    – cesarm
    Commented Nov 2, 2018 at 1:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.