Manage a peak inflow of data

I have an external payment processor that sends a webhook whenever a payment is collected. The webhook currently triggers a Backendless method.

As the payments have increased, there are a lot of webhooks triggered on the 1st of each month, causing a rate limit on Backendles.

The payment collected data in each webhook payload is very simple, an object with 2 strings. I am trying to get around this by storing the ‘payment collected’ data before processing it gradually with a timer, so as to avoid the peak.

Is there a way I can add a backendless table URL directly in the webhook URL field, collect the data and then run a timer on the table? Otherwise I will need to use Zapier on an external service with no rate limits to collect the data in an external DB, then run a Backedless timer on the data.

Any other suggestions?

Hello @Andreas_Marinopoulos

I have a big wish to help you, but unfortunately, I can’t understand what is your problem.
I will try to suggest:

For now, you have a service for payments, that sends a request for one of your API Services to save in the Backendless Table some information about payment. After that, you get this information in a timer and process it. What you want to improve - is get rid of this API service that saves information in the table.

Is it correct? If not, please provide a step-by-step guide on what you do now, and what you want to achieve. This information will help me to invite colleagues with needed expertise if needed.

Regards, Dima.

Hi @Dima and sorry for being unclear. Your description is correct but the issue is that the data coming into the api service comes in a large volume on a specific date. This creates 3000 concurrent requests and overloads my Backendless app. I am looking for the best way to smoothen the flow. But I cannot smoothen the source, I will get over 3000 requests again on November 1.

I could offer you to use the REST API for saving data to the table.
https://backendless.com/docs/rest/data_single_object_create.html

It will decrease your API calls x2 if you store them from API service before, but it still doesn’t solve the root issue. If all requests come at the same time, I couldn’t see how we could delay it from our side.

Which service you are using for payments?