Filling the database in real time from several third-party api

Data Service
1

Hello, everyone! I am testing your platform. Very interested and appreciated the great work you have done.
I am interested in the organization of updating data in tables at the moment from 2 sources such as:

These are services that collect data from sensors in real time. They provide their own REST API. Is it possible to integrate them into your platform to display sensor statuses in real time? If so, how do you recommend doing it?

2

Hello @Sergiy_Homenko!

Welcome to our community!
Could you please clarify if I understand correctly that you’re planning to store and update this data in a database?
Or is the goal simply to retrieve the data and display it in the UI?

Regards,
Alexander

3

Hi Alexander,
I just need to receive data and display it in the user interface. I also need to do additional aggregations with data for users. Such as the average temperature per day and the like. But with aggregations, everything is clear, I just don’t understand how best to organize the actual data retrieval.

4

I need this data in order to :

  • make aggregations
  • show problematic sensors
  • show presence statuses for buildings (for example, presence in a hotel room)
    These are the tasks I plan to do. This is for a better understanding of the context.
5

If you’re planning to utilize Codeless for your logic, you can make use of this block to perform a GET request for retrieving data.

image
image4006×1288 391 KB

You can also run it at specific intervals.
It’s possible I might not have fully grasped the issue you’re facing, so please don’t hesitate to ask questions.
We’re here to help and assist you with pleasure!

Regards,
Alexander

6

Thank you very much for your answer. Okay, I will try this option. Maybe I will ask a lot of questions later. :sweat_smile:

7

Certainly, feel free to reach out. We’re always here and happy to assist! :slightly_smiling_face:

Regards,
Alexander

1 Like
8

Unfortunately, the previous solution does not solve the problem. I tried it and realized that if there are several users, there will be excessive use of service api and other problems.
I meant on the backendless server side, can we do something like a cron job or an example of how Apache Kafka works (collects data from different sources and writes it to the database).
We already have a project for a certain server architecture and the task of filling the database is performed by a node js service that updates tables with data received from situm and monnit every 10 seconds.
The most important question is whether we can either launch a node js service on your platform that will do this or maybe you already have some ready-made solutions for this?

9

Thank you for sharing more detailed information.
We do have a scheduler, and you can find more information about it here.
Additionally, I’ve reached out to my colleagues who have extensive expertise in this area to assist in addressing your questions.

Regards,
Alexander

10

Okay, thank you Alexander.
I’ll try to figure out how to do it with the scheduler.