How can I unpack Stellar's XDR history files?

Question

Inside the Stellar core history archive there are these folders:

bucket  history  ledger  results  scp  transactions

So inside of the transaction folder:

└── transactions
└── 00
    └── 72
        ├── 69
        │   ├── transactions-007269bf.xdr.gz

I can now unzip this file and get "transactions-007269bf.xdr"

My question is how do I unpack this file?

I managed to use Stellar-archivist's command 'dumpxdr' that unpacks the XDR file to JSON format.

How can I do it programmatically? (preferably python, but I'm open to suggestions) I can execute the stellar-archivist program from code, but I want to do it without being dependent on an external program.

Answer 1

Those files are streams of XDR objects, you can simply pick up any of the "base" SDK packages that include the generated code to (un)marshal objects (all generated from the .x files that are the specification).

A stream is written as a concatenation of pairs (XDR encoded size, XDR encoded object). The XDR encoded size is ORed with 0x80000000 and written byte by byte with the high order byte first. You can mimic what archivist XDRStream does to decode.

Most languages are supported, for example the javascript one is js-stellar-base

You can find a list of some of the existing SDKs on the developers reference page