Problem with data retrieval paging

General
1

When my app loads It needs to retrieve a large number of objects (currently around 2k but will grow n future) which includes for instance geocodes in order to show markers on the map or some analysis and show the results. It means It definitely needs to retrieve all the data objects at once. With the paging limit to 100 and needing to loop and making so many api calls the loading time of the app has dramatically increased. I was wondering whether there is any workaround for this problem? Is there any possibility increase the paging limit by purchasing any feature in the marketplace?

2

Hi @Majid_Sepandarand

There is no such possibility for Cloud version. However you can try Pro or Managed versions which allow you to configure data retrieval page size and remove the limits.
More about plans and pricing here:

Also, personally I would recommend to review the code. It’s a bad practice to load such a large number of objects at once.

Best Regards,
Maksym

3

@Majid_Sepandarand, I am curious, when your data table grows to be 100,000 objects? half a million, million, will you still be retrieving all data from it?

If not and your data is of a static size, you need to come up with a mechanism to cache it outside of the database.

Regards,
Mark

4

Is it possible to cache my database in backendless cache on the server side using timers for instance every two hours and users retrieving the data from the cache instead of the database? Can I secure my cache data to be only read only as well?

5

Hello @Majid_Sepandarand

You can use Backendless Cache, which you can read more about - Backendless Cache API Overview. I want to note that there is also a limit on the size of the value - before an object is placed in cache, it is serialized as a byte array. The size of the serialized object cannot exceed 10240 bytes.

I do not know what logic is hidden behind the fact that you need to load all objects at once.
Perhaps pagination on your side will help you and load only the data that is needed at the moment.
Perhaps it will help you with subsequent calls to receive only those objects that have been created or updated since your last call, they can be filtered by the appropriate created and updated fields.