2

During a manual stress test with js-stellar-sdk, some transactions may be responded slower (e.g. 10 sec). At high TPS, asynch programming possibly causes server.loadAccount() to get the same tx_seq, thus the tx_bad_seq error.

As I would like to test 500TPS, are there suggestions to prevent this? I know building up channels helps, but setting up 5,000 channel accounts might not be the ideal solution...

p.s. I have tried to use 500 channel accounts to test for 200TPS, and this issue persists. My JS testing script distributes the transaction to each channel, so every channel should have around 2.5sec to submit a tx, which seems not so sufficient to prevent the problem...

Improve this question

1 Answer 1

Reset to default
1

You will keep getting the same error even with lower TPS, until you change the approach. Just use a queue and operation grouping. 500 channel accounts are capable to deliver ~1000 operations per second (50 tx per ledger, 100 operations each).

Queue

Setup a queue for your operations. In the simplest form, it's an array, used as a buffer layer for transactions. Many languages (C#, Java, even JavaScript) provide a ready-to-use Queue class implementation, it's one of the basic programming primitives. JavaScript allows to implement a queue as an Array, methods .push() and .shift() allow to add an element and retrieve it accordingly.

When you want to initiate a Stellar transaction, you put it into the queue. Do not start building a transaction too early, operate with Operation class on this stage.

There are a lot of enterprise grade messaging queue implementations that provide high throughput and may ensure delivery even in case of power failure. Take a a look at RabbitMQ or Kafka.

Batching

Once one of your channel accounts is available (transaction submitted successfully, or maybe it was in the idle state), you start building a transaction.

Mark the account as busy, fetch the sequence, and create a new tx. Then take 100 oldest entries from the queue and add them to the tx. Sign the tx with all required signatures and submit it to the network. Once the confirmation from the Network is received, mark the channel account as idle, now it's ready to process the next batch.

Improve this answer
5
  • Thanks @Orbit Lens. But batching may make some tx be delayed?
    – cesarm
    Commented Apr 13, 2018 at 1:18
  • While tx_bad_seq bothers, during testing it often pops out "Error: timeout of 60000ms exceeded". I suspect it is [stellar-sdk.js] and there is certain unescaped/unhandled request/event...
    – cesarm
    Commented Apr 13, 2018 at 1:35
  • 1
    Of course some transactions may be delayed during the load peaks, but it will be the rear case. In this way you can guarantee that all your transactions are applied, and their application order. For your test setup (500 channel accounts) you may experience some delays only after 1000 ops/second, which is more than enough for the most projects. If you are planning more than 1000 ops/s, you'll have to use more channel accounts.
    – Orbit Lens
    Commented Apr 13, 2018 at 2:20
  • Can't say anything about the timeout error. Maybe you are making too many Horizon API calls. If so, the error may be caused by the rate limiting.
    – Orbit Lens
    Commented Apr 13, 2018 at 2:24
  • 1
    That shouldn't be my case... At my cores, I used 'upgrades?maxtxsize=10000'; at Horizon, I set "--per-hour-rate-limit 1440000000", so the rate limiting at both servers should be ok...
    – cesarm
    Commented Apr 13, 2018 at 3:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.