Timeline for Stellar-core catch up failed
Current License: CC BY-SA 4.0
10 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 9, 2020 at 11:56 | comment | added | alexqrid | @taherehfiroozian Yes, it seems that the bug was with the network in my DC. For more information refer to the issue on github. | |
| Jun 8, 2020 at 18:49 | comment | added | taraf | Did you manage to fix this issue? I'm facing the same problem with my stellar core. If yes could you share the solution with me? @alexqrid | |
| Jan 17, 2020 at 22:54 | comment | added | alexqrid | By the way it is also doubtful that the bug is in stellar-core because I was able to run the node in "synced" state for several months and didn't face with any problems. I am going to dig into the network issues and look for solutions. | |
| Jan 17, 2020 at 22:46 | comment | added | alexqrid | Thanks. As I am running stellar-core on Digital Ocean's VPS, it is highly improbable that I have a faulty disk,may be some network problems. | |
| Jan 17, 2020 at 18:29 | comment | added | Graydon Hoare | I've checked multiple archives and they all appear to have dd408c8785919ab2a6f774c1b81763f6bcf3f32446ffcc6bdce4bb28fed85adf as the hash for ledger 27757241 -- so given multiple other nodes are regularly doing full network-validated catchups from those archives, the data is probably correct in the archives and you're dealing with either corruption on your end or a bug in core. | |
| Jan 17, 2020 at 17:57 | comment | added | Graydon Hoare | (I'll check on the archives and verify the hash you're seeing) | |
| Jan 17, 2020 at 17:56 | comment | added | Graydon Hoare |
Interesting! this presents 3 possibilities (listed in what I think are the order of likelihood): 1. Your node's local disk or network corrupted some bits in transit. Possibly a faulty disk, network controller or such. Can you run disk diagnostics? For example on ubuntu, apt install smartmontools and run smartctl -a /dev/<the-disk> and look for the SMART overall-health self-assessment test result? 2. The history archives you're fetching from have corrupt data. 3. Stellar-core has a bug in its data validation routines and is failing to validate actually-valid data.
|
|
| Jan 16, 2020 at 12:01 | comment | added | alexqrid | As I got the history archives from SDF nodes that log message made me think that something happened with SDF nodes. So I added additional nodes to fetch history archives from, but after a couple of days I met this problem again. | |
| Jan 16, 2020 at 11:53 | comment | added | alexqrid |
Thank you for the answer. I am using the latest version of stellar-core stellar-core 12.2.0 (3f8a5356f3948eff57d17d76f6341ece4d47e2ff) on Ubuntu 18.04. Can't imagine why does it happen. Node never crashes. But this problems started occuring from 31.12.2019. You said that "Typically this is the result of some sort of data corruption on the node's local storage system". But the log is surprising me One or more of history archives may be corrupted. Update HISTORY configuration entry to only contain valid ones [VerifyLedgerChainWork.cpp:339] And I can not find what configs should I update.
|
|
| Jan 15, 2020 at 18:04 | history | answered | Graydon Hoare | CC BY-SA 4.0 |