How does stellar-notifier work?

Question

I am checking the following repository - https://github.com/orbitlens/stellar-notifier

Can anyone explain how this works end-to-end?

Few queries:

1. I want a notification on whenever a payment is received (in the network - not a particular account). Is that possible?

2. What is mongodb used for?

3. What does maxActiveSubscriptions means ?

After starting the notifier in ubuntu system and setting up mongo db, I tried to check mongo db, but I'm getting the following error:

> show dbs

local             0.000GB

stellar-notifier  0.000GB

> use stellar-notifier

switched to db stellar-notifier

> db.stellar-notifier.find()

2019-03-04T04:50:56.260+0000 E QUERY    [thread1] ReferenceError: notifier is not defined :
@(shell):1:1

Note: I have followed all the steps described in the Installation section.

Answer 1

if you just want all payments from the network, then you don't need stellar-notifier. Just use stellar-sdk.

Here's example in javascript

const StellarSdk = require('stellar-sdk');
StellarSdk.Network.usePublicNetwork();
const server = new StellarSdk.Server('https://horizon.stellar.org');
server.payments()
    .stream({
        onmessage: msg => {
          console.log(msg);
        },
    );

Now to your questions

1. StellarSdk allows to subscribe only to all payments or payments for 1 account. stellar-notifier is useful when you want to monitor multiple accounts, but not everything. You subscribe to every account that you're interested in and receive only related events.
2. to persist the state (users, subscriptions, notifications). Basically to make sure that even after restart of stellar-notifier it will remember accounts you subscribed to
3. It's a limit to make sure that stellar-notifier performance doesn't degrade with number of subscriptions. Up to 10k it's fast and good. If you need more - consider multiple instances.

Answer 2

I absolutely agree with umbrel, you are better off with a simple stream subscription script using stellar sdk.

Nevertheless let me answer the question with regard to stellar-notifier and mongodb.

About stellar-notifier:

I assume stellar-notifier does exact that: it subscribes to a horizons transactions stream (umbrels example subscribes to payments since you don't care about other operations), checks if any incoming transaction matches one of its subscriptions and calls the notification url.

As far as I can see it has only a HTTP-API interface, to create a subscription you have to POST to its /api/subscription endpoint. This subscription will call https://example.com/notify.php for any operation affecting GA63ODNJMEGIOKZ4X6ZZT4MOH5V3FTIPUOEATDZDBRM662V4EX7FORK4:

 'curl -d "reaction_url=https://example.com/notify.php&account=GA63ODNJMEGIOKZ4X6ZZT4MOH5V3FTIPUOEATDZDBRM662V4EX7FORK4" -X POST http://localhost:4021/api/subscription'

About mongodb:

A mongo database contains collections (which is similar to sql tables) that you can query like this:

> show databases
admin             0.000GB
config            0.000GB
local             0.000GB
stellar-notifier  0.046GB

> use stellar-notifier
switched to db stellar-notifier

> show collections
notifications
subscriptions
txingestioncursors
users

> db.getCollection('subscriptions').find({})
{ "_id" : ObjectId("5c7d052ab2275c3506d0698e"), "status" : 0, "operation_types" : [ ], "delivery_failures" : 0, "sent" : 0, "user" : null, "reaction_url" : "https://stellar.sui.li/callback.php", "account" : "GA63ODNJMEGIOKZ4X6ZZT4MOH5V3FTIPUOEATDZDBRM662V4EX7FORK4", "created" : ISODate("2019-03-04T10:59:54.766Z"), "updated" : ISODate("2019-03-04T10:59:54.766Z"), "__v" : 0 }

Of course it's more comfortable to use an admin frontend to access data. You can look for "Robo 3T" if you want to dive into mongo.

Answer 3

Most of your questions are covered in repository readme. It's always advisable to read the docs before trying to run any app.

As @umbrel and @sui mentioned, you can utilize standard SDK payments streaming for the simplest case. However, you'll have to write tons of code if you need guaranteed event notifications.

For example, let's consider that you are building a payment processor or an anchor. Probably you need to track transaction memo values and maybe some other operations (like trades) for your account. It's not possible with /payments endpoint. You'll end up tracking transactions and parsing raw XDR to retrieve operations. And what if someone sent you a payment when you were rolling an update? Or maybe your server suddenly restarted. You'll miss the payment notification, and a user will be frustrated. Of course, it's possible to avoid this. You just need to store the cursor somewhere, bulk load transactions on application start, and switch to live stream tracking once you processed the history.

I've implemented the same code for different projects more than once, and it was almost identical. Therefore, I decided to create a unified solution that works as a micro-service and allows to forget about all the complexity of reliable notifications streaming.

All you have to do is to deploy it, start the app, and subscribe to the desired notifications. The interaction with the notifier is handled by HTTP API. Once the subscription is created, it will automatically send POST requests for all operations that match your criteria to the endpoint you specified.

Instead of coding each case by yourself, you can create a page or HTTP handler that will receive the notifications. The subscription filter parameters allow you to build intricate filters conditions, like "notify me when account GABU...KPH7 receives a payment in MOBI tokens with memo 34567", or simply grab operations specific operations by type.

With the notifier you can relax and upgrade your application whenever you like. Once it is up, the notifier will continue to send the notifications from that exact point and strictly adhere to the operations order. Restarted the notifier server itself? No problem, you won't miss an operation in any case.

The database is used internally to store cursors and subscription parameters. Users are not supposed to query it directly. To make it work, you need to register register new subscription, either from your application or simply from the console. For example, to track PAYMENT, PATH_PAYMENT, and CREATE_ACCOUNT operations

curl -X POST -d '{"operation_types":[0,1,2], "reaction_url": "http://my-server/payments-handler"}' -H "Content-Type: application/json" http://notifier-server-ip/api/subscription

And answers to your questions:

I want a notification on whenever a payment is received (in the network - not a particular account). Is that possible?

Yes, just specify what type of operations you need without filtering by account. By the way, an account can receive funds as a result of a PAYMENT, PATH_PAYMENT, CREATE_ACCOUNT, and MERGE_ACCOUNT operations.

What is mongodb used for?

To store auxiliary data and ensure that temporary service (the notifier, subscriber, or both) disruptions won't affect the notifications.

What does maxActiveSubscriptions means?

The maxActiveSubscriptions parameter specifies how many subscriptions you want to be handled by a single Notifier server. You can increase this parameter if you are running it on the adequate hardware. I tested the service with as much as 10,000 concurrent subscriptions.