Pull data from an external data source using an HTTP request

Server Code Codeless
#1

Hi everyone,
I am new to this community and happy to be here. Here is my issue:

Backendless Version (6.1.10, Online )

Client SDK (REST)

Application ID 6FF3B177-880D-E896-FFB6-16A8815BE100

Expected Behavior

  1. Every 10 minutes, I want to use an HTTP request GET data from an external API containing data in a schema matching one of my data tables. (ie.https://api.close.com/api/v1/lead?query=(opportunity(status%3A"BG%20Enrolled%2FHired"%20pipeline%3A"Tutoring"))%20(lead_status%3A"Families%20(Tutoring)")) with username and password authorisation.
  2. Using business logic I want to format the received data.
  3. I want to create objects in my table(s) based on the data,

Actual Behavior

I am new to Backendless and I am using Codeless (don’t know how to code). I have a lot of data (customer and contact objects) in our CRM application and would actually need to keep my Backendless tables in sync with our CRM on an hourly frequency. I am currently using Parabolla to feed this data into Airtable/Google Sheets successfully.

I cannot find the right way to pull in data from an external data source using Codeless. I am trying to do it with a timer. Is there a best practice to do this or an online walkthrough or forum post? Thank you in advance!

#2

Hello @Andreas_Marinopoulos

Welcome to our community and thank you for trying out Backendless.

I am looking into it.

1 Like
#3

Hello @Andreas_Marinopoulos

We will be happy to assist you. I need to ask you a few more questions so I can understand the problem better.
Please write your appIDI.
At which step are you having problems?
How did you try to get data from an external resource?

Schematically what you want can be something like this:

TimerCodeless
TimerCodeless1104Ă—843 54.4 KB

1 Like
#4

Thanks so much for the prompt reply @Vladimir_Yalovy!

I am trying this now and will let you know asap.

#5

Hi @Vladimir_Yalovy, the timer seems to work. My only problem now is that the list I seek is under a nested key called “data”. I think I just need some time to get used to the Codeless user interface, but if you have a quick fix to access the array 1 level down from a nested key, I would be much obliged.

Here is what I have for the moment:

image
image1894Ă—859 107 KB

APP ID: 6FF3B177-880D-E896-FFB6-16A8815BE100

Best,
Andreas

#6

Hi Andreas,

Could you please show what the response from api.close.com looks like and what specific element you need to obtain?

Regards,
Mark

#7

Hi @Andreas_Marinopoulos

I’ve tried to run your timer logic but it gives me an error Unauthorized. Can you edit your logic to make a valid request?

Best Regards,
Maksym

#8

Thanks both of you @mark-piller and @Maksym_Khobotin2, indeed I am doing something wrong in calling the API.

First off, just to make sure I am debugging correctly, I am building the timer, deploying it, running it and checking real-time logging, correct?

image
image3324Ă—604 131 KB

This is what I am seeing in the log:
wrap messages
Light Theme
Basic timestamps
Type to search…
Filter
09:25:00 | SERVER_CODE | INFO | [12886] Building ServerCode Model for path (/opt/backendless/repo/6ff3b177-880d-e896-ffb6-16a8815be100/files/servercode/CODELESS/default/PRODUCTION)
09:25:00 | SERVER_CODE | INFO | [12886] ServerCode Model built in 5ms
09:25:00 | SERVER_CODE | INFO | [12886] [8AF24012-A02D-97D9-FFF6-4B887D288500] [INVOKE TIMER] test
09:25:01 | SERVER_CODE | ERROR | [12886] Error: [object Object] at new ResponseError (/usr/local/lib/node_modules/backendless-coderunner/node_modules/backendless-request/lib/index.js:42:116) at checkStatus (/usr/local/lib/node_modules/backendless-coderunner/node_modules/backendless-request/lib/index.js:237:9) at processTicksAndRejections (internal/process/task_queues.js:97:5)
09:25:01 | SERVER_CODE | INFO | [12886] Processing finished in 347.271ms

I have tweaked the logic slightly, please excuse the newbie issues, I think that I am not yet familiar enough with building business logic in Backendless as I am in other nocode tools. This is slightly outside my comfort zone.

image
image1949Ă—1065 204 KB

Here is what the GET request normally gives: https://www.loom.com/share/b113bdd142d64cb8a44780cf25b53096

#9

Hi Andreas,

Let’s make sure the HTTP/s block works as expected first. To do this, create a Codeless API service where the return connector will have the same HTTP/s block as you have shown above. Invoke the API service and make sure you’re actually seeing a response. This will be the first thing to be done. Working with an API service will be easier as you can actually see a response. Once you get it working (and we will help), then we’ll move on to adding the logic in a timer.

Regards,
Mark

#11

Hi @Andreas_Marinopoulos

The problem is you should pass the decoded query string. Because we encode it on our side.
So instead of

lead_status%20in%20%28%22Families%20%28Tutoring%29%22%29%20opportunity%28opportunity_status%20in%20%28%22stat_HHGfU5RWJczVPETBgenmIqNZtzubx8Dd9hu7iiaOMIh%22%2C%20%22stat_voeIU8KgGkuFlky0OJDXR2X4W5MyRTShOIEiwT5neBr%22%29%29%20

you should pass

lead_status in (“Families (Tutoring)”) opportunity(opportunity_status in (“stat_HHGfU5RWJczVPETBgenmIqNZtzubx8Dd9hu7iiaOMIh”, “stat_voeIU8KgGkuFlky0OJDXR2X4W5MyRTShOIEiwT5neBr”))

Or if you want to keep the encoded query, you can use the Decode URI block under Networking API to decode your query.

I updated your query, changed the _limit value from '1' to 1 because it requires the integer value and replaced Save Object codeless block with Bulk Create because the response is an array of objects.
I also created the test service method to check the response from HTTP\s block. You can check it by yourself under test API Service, method getParents. Invoke this method to see what HTTP\s request returns.

Good luck and happy codeless coding with Backendless!

Best Regards,
Maksym

1 Like
#12

Thanks @Maksym_Khobotin2, @mark-piller, amazing support guys, really thanks.

Will replicate both suggestions within the day and revert back to you. This is the key for me to do a successful codeless backend implementation.

I have already made my frontend (Bubble) communicate with Backendless, so if I can get Close.com syncing with my DB, I will be off to the races.

Will let you know when I have an update.

Best,
Andreas

1 Like
#13

Works perfectly!

#14

I’m a newbie to APIs and having an issue understanding how to use an external API with codeless. I would like to get data from an external API and display it, but there doesn’t seem to be any video or post anywhere I can find to help me with that. I wanted to start with a public API to not have to need an authentication token, but I just ended up completely lost on what to do with the information from the docs after MANY tries and fails. Is there any help that can be provided? Thank you in advance.

#15

Could you please share the details about the external API request you need to make?

#16

I chose a public API from the site dog.ceo and I wanted to display a random dog image in a block upon button click. The site contains a URL to display the random images, and what the JSON output is. I’m confused on how to use this information properly. I hope thats the details you need.

#17

What I’d like to see are the details that one needs to have in order to make a request from a system such as ours. Here’s an example of how these details are usually documented:
https://stripe.com/docs/api/charges/create

In there you will see the following:

  1. Request type ( POST, PUT, GET, DELETE, etc)
  2. Endpoint URL
  3. Request headers
  4. Request body

Without knowing these details you will not be able to make a request.

#18

In that case, the documentation is not good. The site only contains a URL and JSON code. See here - https://dog.ceo/dog-api/

#19

Their API documentation is actually here:

Which one did you need to use?

#20

I selected the random image option from this page.

#21

Here’s a demo of that API in action:
https://www.backendless.us/api/files/ui-builder/containers/support/index.html?page=externalapi

The button click has the following logic which uses the random image API:
UI Builder - ConsoleDemo - Backendless 2022-06-06 20-59-48

Hope this helps.

Mark