Timeout on File listing for large directory

File Service
1

Backendless Version

com.backendless:backendless:6.3.4
Cloud 9

Client SDK

Java

Application ID

7E2B6F4D-30C8-734C-FF85-4B835F84EB00

Expected Behavior

Please describe the expected behavior of the issue, starting from the first action.

  1. Calling List sessionsToProcess = Backendless.Files.listing(“sessions”, “*”, false, pageSize, offsetSessions);
  2. The call returns with a list of files

Actual Behavior

Please provide a description of what actually happens, working from the same starting point.

Be descriptive: “it doesn’t work” does not describe what the behavior actually is – instead, say “the request returns a 400 error with message XXX”. Copy and paste your logs, and include any URLs.

  1. Calling List sessionsToProcess = Backendless.Files.listing(“sessions”, “*”, false, pageSize, offsetSessions);
  2. getting com.backendless.exceptions.BackendlessException: Read timed out
    Caused by: java.net.SocketTimeoutException: Read timed out

Remarks

I suspect this might happen because there are too many files in the folder (roughly 17.000, 2.5GB). In the web UI, this takes about 10s to load. Please advise if i can speed up the call, or increase the call timeout somehow.

2

Hi, @Rene_Habermann

Please provide an example of your code, how do you get the directory listing?

Please, take a loot the docs below, to make sure that the request is correct: Directory Listing - Backendless SDK for Android/Java API Documentation

Regards,
Marina

3

Backendless.initApp(“7E2B6F4D-30C8-734C-FF85-4B835F84EB00”, “*removed”);
// timeout occurs here:
List sessionsToProcess = Backendless.Files.listing(“sessions”, "*", false, 100, 0);
for (FileInfo file : sessionsToProcess) {
// process the file
}

4

Hi, @Rene_Habermann

Thanks, an internal ticket was created - BKNDLSS-28737. We are looking into the issue.

Regards, Marina

5

Any update on this? Is there an estimate on when that might be resolved, or get a new status? I can’t access production data, so this is a critical issue for me.

6

It looks like i have no way to reasonably get access to the data, as both the client won’t work, but also zipping the folder as archive is not possible, seemingly because of storage limitations.

Backendless finished archiving of files for your application unsuccessfully for application

I’ll start migrating to GCP or Azure, sadly. All i need is a reliable file service. You can close this ticket.

7

Hello @Rene_Habermann,

We made a fix and released it literally just now.
Could you please check it and let us know if this is working for you?

We try very hard to provide the most reliable, affordable and high-quality service for all our customers. We are very sorry that you are leaving us. We hope we can change your mind about us.

Best Regards,
Stanislaw

8

I’m seeing this issue again. Probably too many files in there. I didn’t change any code, but it will be close to 30GB in that directory. How many files i cannot say, as i also run into that issue when trying to view it in the browser. Can you reopen the ticket please?

Is there any way i could download this data now?

There was an unexpected error (type=Internal Server Error, status=500).
Read timed out
BackendlessException{ code: ‘Internal client exception’, message: ‘Read timed out’, extendedData: ‘{}’, detail: 'java.net.SocketTimeoutException: Read timed out
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

9

Hello @Rene_Habermann,

we’re looking into this…

10

I have reopened the ticket.