The memo is associated with the transaction, so you'll have to actually pull the transaction for each payment.
Here is some code for how I've handled it in the past with the JS SDK:
function watchForPayments(publicKey, { onPayment, cursor }) {
const paymentsCallBuilder = server.payments().forAccount(publicKey);
if (cursor) {
paymentsCallBuilder.cursor(cursor);
}
return paymentsCallBuilder.stream({
onmessage: async record => {
const transaction = await record.transaction();
onPayment({
memo: transaction.memo,
amount: record.amount,
createdAt: new Date(record.created_at),
asset: record.asset_type,
cursor: record.paging_token,
to: record.to,
from: record.from,
hash: record.transaction_hash
});
}
});
}
The key part is const transaction = await record.transaction();
, which makes another request to get the transaction (which contains the memo) associated with the operation.