Hello. I have a question about data aggregation. Or maybe you can offer some other solution.
I have a graph in which I display the temperature fluctuations of the sensor for a certain period of time (24 hours, 3 days, 7 days and dynamic user selection). The problem is that the sensor generates a lot of records and, for example, in a day it can generate several thousand records, and as you can imagine, in a month it will be a significant amount that cannot be obtained on the frontend.
I initially thought about creating such a service:
class ChartSensorService {
async getChartData(sensorId, startDate, endDate) {
//startDate= "2023-09-04T00:00:00.000Z"
//endDate= "2023-09-05T00:00:00.000Z"
let pageSize = 100;
let offset = 0;
let aggregatedData = [];
let tempData = {};
while (true) {
const whereClause = `SensorID = ${sensorId} AND created >= '${startDate}' AND created <= '${endDate}'`;
const queryBuilder = Backendless.DataQueryBuilder.create()
.setWhereClause(whereClause)
.setPageSize(pageSize)
.setOffset(offset);
const messages = await Backendless.Data.of('messages').find(queryBuilder);
if (messages.length === 0) {
break;
}
messages.forEach(message => {
const date = new Date(message.created);
const key = `${date.getFullYear()}-${date.getMonth()}-${date.getDate()} ${Math.floor(date.getMinutes() / 10) * 10}`;
if (!tempData[key]) {
tempData[key] = [];
}
tempData[key].push(message.TemperatureData);
});
offset += pageSize;
}
for (const [key, values] of Object.entries(tempData)) {
const maxTemp = Math.max(...values);
aggregatedData.push({ date: key, maxTemp });
}
return aggregatedData;
}
}
Backendless.ServerCode.addService(ChartSensorService);
But it doesn’t seem to be a very good option either. Do you have any advice for this situation?