Import data to backend programmatically

General
1

Hi,

I need to import data on Backendless database programmatically, and, in this file:
https://backendless.com/docs/rest/mgmt_import.html

I read:

Importing data into a Backendless backend can be done either with the Backendless console or using the Management REST API

but I’m not able to find the documentation on this API!

How can I access the Management REST API programmatically using JS code?

Thank you,
Elena.

Backendless Version: 6.6.5
Client SDK: JS
Application ID: 853476F7-1565-CBA3-FF64-81D8A9303300

2

Hello @Elena_Aralla!

Unfortunately, there is no dock yet, but you can use this excellent guide and implement this functionality based on it:
https://support.backendless.com/docs?topic=15102

And here are the methods that will allow you to import data:

    ...

    const FormData = require('form-data')

    ...

    const csv = // your csv file

    const formData = new FormData()
    formData.append('file', csv)

    const response = await apiClient.transfer.importDataServiceFiles(appId, formData, 'step1')

    const { tables, views } = response
    
    return await apiClient.transfer.importDataServiceFiles(appId, { tables, views }, 'step2')

Regards,
Alexander

3

Hi @Alexander_Pavelko,

thank you for the link and example code;

I tried to import a simple csv file, but this error occurs:

12:12:37.075 [master] Error: An error occurred. Invalid file.
    at new ResponseError (/Users/elena/projects/LiciLab/node_modules/backendless-console-sdk/node_modules/backendless-request/lib/index.js:42:116)
    at checkStatus (/Users/elena/projects/LiciLab/node_modules/backendless-console-sdk/node_modules/backendless-request/lib/index.js:244:9)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

I used a file formatted as in @mark-piller coockboock: How to import relational data into Backendless with CSV files
which name is customers.csv (the same name of the destination table)

Firstname,"objectId({""type"":""STRING_ID""})"
"Joe","765EF32B-A715-4C54-84D6-EB1E8B622121"
"Bob","8D79BA7D-ACE3-47DC-9D90-B5985CDDD363"
"Jack","FC584FF8-E44B-408D-B9F4-F7CF387CD538"

this is my import function defined in TestService:

const FormData = require('form-data');
const consoleSDK = require('backendless-console-sdk')

...

/**
     * 
     * @param {String} login
     * @param {String} password
     * @return {Object} rc
     */  
  async importData(login, password) {

    const { appId } = this.request.context // More about invocation context here:
                                           // https://backendless.com/docs/bl-js/bl_invocation_context.html

    // Create api client using `createClient` method of console SDK.
    // It expects server url, which we can take from Backendless SDK’s ‘serverURL’
    const apiClient = consoleSDK.createClient(Backendless.serverURL);
    await apiClient.user.login(login, password);

    const csv = "https://eu.backendlessappcontent.com/853476F7-1565-CBA3-FF64-81D8A9303300/F114A852-5C10-4A87-9D04-E95F4989162C/files/import/Customers.csv";

    const formData = new FormData();
    formData.append('file', csv);

    const response = await apiClient.transfer.importDataServiceFiles(appId, formData, 'step1');

    const { tables, views } = response;
    
    return await apiClient.transfer.importDataServiceFiles(appId, { tables, views }, 'step2');

  }
}

Backendless.ServerCode.addService( TestService, [{
  name       : 'login',
  type       : 'string',
  displayName: 'Login',
  required   : true,
  order      : 0
}, {
  name       : 'password',
  type       : 'string',
  displayName: 'Password',
  required   : true,
  order      : 1
}]);

the error occurs on the response of the step1 call.

I think the problem is the csv file, but I don’t understand why.

I’ve also these two question2:

  1. It’s necessary to specify the “objectId” field? What if I’m importing a new Customer that does’t exist on table?

  2. What happens if I’m importing a customer that already exists? Are Data overwritten?

4

Hello @Elena_Aralla!

This error is due to the fact that you need a cvs file, not a link to it.
You can use a request module to do this, for example:

const request = require('request')
...
const csv = request.get('https://eu.backendlessappcontent.com/853476F7-1565-CBA3-FF64-81D8A9303300/F114A852-5C10-4A87-9D04-E95F4989162C/files/import/Customers.csv')
...
  1. No, it is not necessary, you can do as in the example from the documentation:
  price,productName  
  599,iPhone  
  899,Surface Pro  
  399,XBOX One  
  1200,MacBook
  1. By default, it will not overwrite, but you can change the value of overrideExistingObjects and then the existing objects will be overwritten. For example, like this:
  ...
  const response = await apiClient.transfer.importDataServiceFiles(appId, formData, 'step1')
  const { tables, views } = response
  
  tables.forEach(table => table.overrideExistingObjects = true)
  ...

Regards,
Alexander

5

Hi @Alexander_Pavelko,

ops :slight_smile:

it works, thank you!

1 Like
6

Hi @Alexander_Pavelko,

one more question:
if I want import more than one csv, can I append it to the same form_data or have to exec import one file at time?

May I import zip file with all csv (one for each table)?

7

Yes, you can add multiple files at once:

 files.forEach(file => formData.append('file', file))

I think the zip file will need to be unzipped before adding it to the FormData.

Regards,
Alexander

8

Thank you @Alexander_Pavelko

I noticed that the import file generate a table which name is the same name of the file; is there a way to specify a name for table?

9

Hello @Elena_Aralla,

No, there’s no way to specify table name while importing except the file name.

Regards,
Olha