Suggestion: API request cap to protect against unexpectedly high app costs

Suggestions/Ideas
1

In a separate thread about where to host app data to minimize API requests,
I came up with this question:

Is there a safety feature I can enable to deny requests beyond a certain tier?
By “deny” I mean - throttle, queue, or stagger timing of API requests,
in order to safeguard against unexpectedly high fees.

Without such a feature I think it will be difficult to anticipate how high the cost of running the app will be.

This question arose because on the page Introducing the Scale Plan I did not see details about how to prevent costs from getting out of control.
However, if such a feature already exists and I’ve missed it, feel free to disregard this feature request.

:warning: Update: Before you read further - it turns out this feature exists! :warning:

So this post of mine here is mostly redundant.
I recommend the majority of people skip reading this post. It will probably bore and confuse you.
Only keep reading if you like to geek out on theoretical pricing plans / features.
There are some differences in my suggestion compared to the current implementation.
I hope the Backendless Team will still give my suggestion consideration.
But for the typical Backendless user - just know that we are already able to set an API request cap.
It is called a Scale Plan Tier Threshold, and you can find it in your Billing settings.

For those of you still curious, I will let the post text below remain as it is.
↓

Let’s examine a couple of scenarios to understand why such a feature would be valuable.

Example scenario #1 (with safety cap)

Let’s say for example my budget allows for up to Tier 4, based on the following calculation:
Tier 4 costs up to $2.15 on any day that it peaks at 21-50 requests per min.,
meaning that if it peaks at that level every day (unlikely, but let’s say for the sake of simplicity),
then I would be paying $64.50 ($2.15 * 30 days) plus the Scale Plan fee of $15.00,
totaling $79.5/month at maximum.
This would be the most expensive scenario if there is a safety mechanism.
It’s predictable, and not too scary.

Example scenario #2 (without safety cap)

Let’s say I am expecting typical traffic to be in the Tier 3 range. In a simple hypothetical example,
this is $1.75/day if traffic consistently peaks at 11-20 requests.
(And let’s say I anticipate there may be busier days when usage reaches Tier 4.)

But then something unexpected happens…

  • Maybe the app gets written about on a popular blog, and sees a rapid influx of new users.
  • Or maybe hackers decide it would be fun to carry out a DDOS attack on my app.

The daily price would spike to $230.00/day and could remain there for a number of days if unnoticed. Even if noticed, what sort of action could be taken?
As far as I know, a developer’s only recourse would be to shut the app down completely if there’s no way to throttle the requests. That would lock out users, giving a horrible customer experience. And a lot of money would be owed.

If unusually high traffic continued unnoticed and unmitigated for a number of days, the Scale Plan could easily exceed a cost of $1000 in a single month.
For a large company this might not be such a big deal, but for small business owners, independent developers, and entrepreneurs trying to get their fledgling startups going this would have significant impact.

I’m not trying to be pessimistic, or get too far ahead of myself in potential scenarios.
However, one has to consider the possible risks before stepping into a business decision.

So please let me know if such a safety mechanism exists for mitigating costs by limiting API requests.

A few more thoughts

In the case where there is a way to limit API requests, if halfway through the month app traffic has only gone up to Tier 3 when Tier 4 was anticipated, the limit could be raised to accommodate a peak of Tier 5. In other words, with the leftover budget can be reinvested to raise the ceiling in case of a spike later in the month. This is a bit of an over-simplification, but it’s one possibility to consider.

Another (better?) way to think about budgeting is to have a separate “heavy traffic budget” set aside.
For example, an app developer expecting typical usage to be Tier 3 can specify that they want to:

  • Allow traffic up to Tier 3 any day of the month
  • Allow x days of Tier 4 traffic
  • Allow y days of Tier 5 traffic
  • Cap any requests above Tier 5.

I’ll try to illustrate with a table:

A hypothetical app’s traffic for one month - with caps in place

  • We can see from the data that our hypothetical app has increases in user activity on weekends.

  • In this example a maximum of 2 days were allowed to go as high as Tier 5.

  • On 4/20 (Sat) traffic tried to go up to Tier 6 but was throttled.
    This was not allowed because 0 days were allocated/budgeted for Tier 6 by the developer.

  • On 4/27 (Sat) traffic tried to go up to Tier 5 but was throttled.
    This is because the 2 allocated Tier 5 days were already used up on 4/6 and 4/20.

  • On 4/30 (Tue) an unexpected spike tried to reach Tier 5 but was throttled.
    This was not allowed because the 2 allocated Tier 5 days had been used (as mentioned).
    It then tried to use a Tier 4 day. However the 10 allocated Tier 4 days were used up on April 5, 7, 12, 13, 14, 19, 21, 26, 27, 28.
    It finally settled on allowing a Tier 3 volume of traffic, as the developer had no cap set for Tier 3.
    (The developer anticipates that Tier 3 is typical usage for their app.)

April 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Weekday Mon Tue Wed Thu Fri Sat Sun Mon Tue Wed Thu Fri Sat Sun Mon Tue Wed Thu Fri Sat Sun Mon Tue Wed Thu Fri Sat Sun Mon Tue
Tier 6+ - - - - - - - - - - - - - - - - - - - X - - - - - - - - - -
Tier 5 - - - - - █ - - - - - - - - - - - - - █ - - - - - - X - - X
Tier 4 - - - - █ █ █ - - - - █ █ █ - - - - █ █ █ - - - - █ █ █ - X
Tier 3 - █ - █ █ █ █ - - - - █ █ █ - - - █ █ █ █ - - - - █ █ █ - █
Tier 2 █ █ █ █ █ █ █ - █ - █ █ █ █ █ █ █ █ █ █ █ - █ █ █ █ █ █ █ █
Tier 1 █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █ █

X = throttled because cap reached (tried to use the next tier down)

There are ways this system can be varied with different rules in order to be even more flexible (setting tier caps for specific days, for example).
And I don’t assume this idea to be complete or perfect by any means. But hopefully the goal of managing exposure to unexpected fees can be achieved through some sort of system.

As a developer I must also consider: At times when API requests are capped, it’s best that an app can continue to function in some capacity with graceful degradation. I’ll try not to get sidetracked on that point, but we will need to get creative with how we build our apps so that the user is not left staring at spinning progress indicators. And inevitably devs will still need to invest in the higher tiers as user bases grow. But equipping devs with a means to release their new apps without needing to worry about disproportionately high fees from the very beginning, would be a positive thing.

The reason I spent time making this example and proposing the idea is that I found myself having a bit of anxiety over the possibility of surprise fees, and thought it would afford developers peace of mind to know they won’t encounter a financial crisis because of how their customers choose to use their app.

As someone with the intention of becoming a paying customer, it’s in my interest that the billing plan isn’t likely to introduce scary surprises.

When developers and their app’s users are having a good experience, that’s good for Backendless too.

Please let me know what you think of this idea.
And thanks for reading all the way to the end of this epic novel. :wink: