Backendless Support
 

Data Paging or How to efficiently load large data sets in a mobile app

Data paging is the process of breaking up a larger set of objects into smaller chunks, commonly referred to as pages. Typically, paging applies to the results of search queries when the server may return too many objects at once. Since smaller pages of data can be returned to the client much faster, the user of the client application does not need to wait. As a result, the user experience of the app is significantly improved, which is the primary advantage of paging.

Backendless automatically breaks up results of search queries into pages. 

The examples below demonstrate the usage of the API and various techniques for paged data retrieval.

All the examples in this article rely on the code obtained through Backendless code generator. 

For details see how to generate client-side code for your mobile application with Backendless. To setup your backend in order to run these examples, you should:

  1. Register/login to Backendless Console and create an app.
  2. Download the following file: https://backendless.com/documentation/samples/restaraunt-app-tables-with-data.zip
  3. Configure your backends storage by following the instructions on how to import schema and data from a backup file to a Backendless app. You will need to use the file from step 2.
  4. Generate client-side source code using Backendless code generator.

In the previous post I showed how to load data from Backendless in the most basic API. The code In that post uses the basic find API to fetch a collection of the restaurant objects. Since Backendless automatically pages data, it will return only the first page. The default page size is 10 objects, however, that value can be changed using the following API (maximum allowed page size is 100):

                              FOR JavaScript implementations CLICK HERE

JAVA

Asynchronous API:
AsyncCallback<BackendlessCollection<Restaurant>> callback=new AsyncCallback<BackendlessCollection<Restaurant>>()
{ /* skipped for brevity */ };
int PAGESIZE = 80;
BackendlessDataQuery dataQuery = new BackendlessDataQuery();
dataQuery.setPageSize(PAGESIZE);
Backendless.Data.of(Restaurant.class).find( dataQuery, callback );
Synchronous API:
int PAGESIZE = 80;
BackendlessDataQuery dataQuery = new BackendlessDataQuery();
dataQuery.setPageSize( PAGESIZE );
BackendlessCollection<Restaurant> restaurants = Backendless.Data.of(Restaurant.class).find( dataQuery );

If a developer masters the skill how to know how to fetch the first page, the question is how to load subsequent pages. Before running any of the paging examples below, it would be worth-while to populate the backend storage with additional objects. Without them, you would not be able to see the paging in action - all the objects would be returned in the first page. Consider the following code which adds 300 restaurant objects to the backend storage:

JAVA

private static void addRestaurants()
{
 IDataStore<Restaurant> dataStore = Backendless.Data.of( Restaurant.class );
 for( int i = 0; i < 300; i++ )
 {
 Restaurant r = new Restaurant();
 r.setName( "TastyBaaS " + i );
 r.setCuisine( "mBaaS" );
 r = dataStore.save( r );
 System.out.println( "Saved " + r.getName() );
 }
}

If a developer runs the method above once, there should be 304 objects in the backend table (4 objects came from the data import and 300 created by the code).

Now the backend is ready to show the power of paging. The following code  demonstrates just that:

JAVA

Asynchronous API (applies to Android and plain Java):

private static void basicPagingAsync() throws InterruptedException
{
 long startTime = System.currentTimeMillis();
 final CountDownLatch latch = new CountDownLatch( 1 );
 final AsyncCallback<BackendlessCollection<Restaurant>> callback=new AsyncCallback<BackendlessCollection<Restaurant>>()
 {
 private boolean firstResponse=true;
 public void handleResponse( BackendlessCollection<Restaurant> restaurants )
 {
 if( firstResponse )
 {
 System.out.println( "Total restaurants - " + restaurants.getTotalObjects() );
 firstResponse=false;
 }
 int size = restaurants.getCurrentPage().size();
 System.out.println( "Loaded " + size + " restaurants in the current page" );
 if( size > 0 )
 restaurants.nextPage( this );
 else
 latch.countDown();
 }
 @Override
 public void handleFault( BackendlessFault backendlessFault )
 {
 }
 };
 Backendless.Data.of( Restaurant.class ).find( callback );
 latch.await();
 System.out.println( "Total time (ms) - " + (System.currentTimeMillis() - startTime ));
}
Synchronous API (Applies only to Java):
private static void basicPaging()
{
 long startTime = System.currentTimeMillis();
 BackendlessCollection<Restaurant> restaurants = Backendless.Data.of(Restaurant.class).find();
 System.out.println( "Total restaurants - " + restaurants.getTotalObjects() );
 while (restaurants.getCurrentPage().size() > 0)
 {
 int size = restaurants.getCurrentPage().size();
 System.out.println( "Loaded " + size + " restaurants in the current page" );
 restaurants = restaurants.nextPage();
 }
 System.out.println( "Total time (ms) - " + (System.currentTimeMillis() - startTime ));
}

In both synchronous and asynchronous examples of the API, the principle for loading the next page of data is very simple - the code calls the nextPage() method which returns exactly what the name says - the next page of data.

Also notice the getCurrentPage() method which returns a collection of data objects for the page from the most recent call of nextPage(). When the retrieval process gets to the end of the result set, it will return an empty collection - which is an indication that there are no more objects available for retrieval.

The nextPage() API is very convenient, it uses the page size you specify with the very first call and automatically instructs the backend to locate only the records for any subsequent page of data. However, there are scenarios when the client needs to load a page out of order. For example, the app UI may offer paging navigation with clickable numbers for the individual pages. In this case. Backendless provides an advanced mechanism for pageable data retrieval API - the getPage() call. The following example demonstrates the usage of the advanced paging - a mechanism which lets you download any block of data (or page) within the result set:

JAVA

Asynchronous API (applies to Android and plain Java):

private static void advancedPagingAsync() throws InterruptedException
{
 long startTime = System.currentTimeMillis();
 final CountDownLatch latch = new CountDownLatch( 1 );
 final int PAGESIZE = 100;
 final AsyncCallback<BackendlessCollection<Restaurant>> callback = new AsyncCallback<BackendlessCollection<Restaurant>>()
 {
 private int offset = 0;
 private boolean firstResponse = true;
 @Override
 public void handleResponse(BackendlessCollection<Restaurant> restaurants )
 {
 if( firstResponse )
 {
 System.out.println( "Total restaurants - " + restaurants.getTotalObjects() );
 firstResponse = false;
 }
 int size = restaurants.getCurrentPage().size();
 System.out.println( "Loaded " + size + " restaurants in the current page" );
 if( size > 0 )
 {
 offset+=restaurants.getCurrentPage().size();
 restaurants.getPage( PAGESIZE, offset, this );
 }
 else
 {
 latch.countDown();
 }
 }
 @Override
 public void handleFault(BackendlessFault backendlessFault)
 {
 }
 };
 BackendlessDataQuery dataQuery = new BackendlessDataQuery();
 dataQuery.setPageSize(PAGESIZE);
 Backendless.Data.of(Restaurant.class).find( dataQuery, callback );
 latch.await();
 System.out.println( "Total time (ms) - " + (System.currentTimeMillis() - startTime ));
}
Synchronous API (Applies only to Java):
private static void advancedPaging()
{
 long startTime = System.currentTimeMillis();
 int PAGESIZE = 100;
 int offset = 0;
 BackendlessDataQuery dataQuery = new BackendlessDataQuery();
 dataQuery.setPageSize( PAGESIZE );
 BackendlessCollection<Restaurant> restaurants = Backendless.Data.of(Restaurant.class).find( dataQuery );
 System.out.println( "Total restaurants - " + restaurants.getTotalObjects() );
 int size = restaurants.getCurrentPage().size();
 System.out.println( "Loaded " + size + " restaurants in the current page" );
 while( size > 0 )
 {
 offset += size;
 restaurants = restaurants.getPage( PAGESIZE, offset );
 size = restaurants.getCurrentPage().size();
 System.out.println( "Loaded " + size + " restaurants in the current page" );
 }
 System.out.println( "Total time (ms) - " + (System.currentTimeMillis() - startTime ));
}

The only difference between getPage and nextPage is the former must provide the size of the page (which is how many objects to return) and the offset, which is a sequential index from which to load the objects.

Review related topics:

Is article helpful?