Application Files Deleted Automatically from Repo on BL Pro

Hi,

I’m deploying an application to Backendless Pro and after coderunner-js works and the deployment is done (repo PRODUCTION folder is updated with the app), the application gets deleted after 1-2 minutes. So, browsing the PRODUCTION/app folder becomes empty.

I’ve checked the bl-server logs and coderunner-js logs … nothing there that might help.

My questions are:

  1. When do you guys delete applications from the repo folder?

  2. In Consul key/value settings, I’ve noticed that there’s a LIMIT-SERVICE which looks like a possible reason of what’s happening. I’ve added my app id in the execludedApps section. It didn’t help. I’ve also tried to increase the timeout for appDeleteDelay, appToBeDeletedNotificationDelay and checkInterval. Didn’t help also.

  3. I’ve noticed that I’m getting this error 10:30:28.388 [ERROR] c.b.l.HttpLicenseHelper - Error from license server with code 400 and body: every now and then. Not sure if this means that I have a problem with my license. I got the license from Vitaly and it worked just fine except for this log entry. Not sure if this might be the reason of the application being deleted

  4. When I’m deploying my app, the line 86:
    await request.post(this.appUrl + /servercode/publishcode/${lang}).form(formData)
    in node_modules/backendless-coderunner/lib/server-code/services/api-server.js
    takes more than 60 seconds to finish. So, I get an error:
    Error: Unable to publish model. socket hang up
    Do you think this might be causing the problem? like for instance marking the deployed application as invalid and fails due to that?

Last note, I’m using BL Pro setup 6.1.8.mysql8

Thanks for the help in advance

Hello @Bassam_Jarad,

We will be happy to assist you. I need to ask you a few more questions so I can understand the problem better.

When you say

I’m deploying an application

do you mean some business logic (BL) – API services, events, timers?

  1. We never delete from the client folder. For example, if you deploy from coderunner to default model we do the following:
  • we create tmp folder in servercode/JS/default/ folder
  • then if everything is ok we move old production logic to tmp old folder
  • then we move tmp to servercode/JS/default/PRODUCTION/
  • then we remove all from servercode/JS/default/DRAFT/
  • then copy all from servercode/JS/default/PRODUCTION/ to servercode/JS/default/DRAFT/
  1. limit service is of for PRO, you should ignore all settings for it

  2. what is the body of the error?

  3. It should not be the problem. Even if the client disconnected the server should finish the job. But to say for sure I need the code to try and investigate. Please attach the project and we will try to reproduce the issue

Hi Sergey,

Thank you so much for the answers. I’m still struggling with this.

Yes, when I’m deploying an application, I install services/timers … etc. Just to make sure my timers are not causing the problem, I’ve disabled them all from code. Nothing changed in the behavior.

  1. This is exactly what’s happening and I can see my application code in the PRODUCTION folder once the coderunner-js finishes analysing the code (I check it using docker logs -f CODERUNNER_CONTAINER_ID). tmp folder gets renamed, then a PRODUCTION folder comes up and populated with my application. If I keep checking the files inside the app folder & node_modules folder, they start disappearing one by one until they become empty. This usually happens within like 2 minutes.

  2. OK

  3. I don’t get a body. just the line I’ve sent.

  4. I see. I’ve noticed a weird thing in the Redis server though. I got keys for the application that ends with -status and they all are marked as “CANCELLED”. Do you think this might have to do with anything?

  1. The easiest way will be if you provide your project and some specific instructions how to build if you have one
  2. No, these keys are for bl-taskman

I see … thank you. The problem is that I don’t think I can share the project (company policy). I hope you understand.

Just a quick overview of my setup, I’m working on a Windows 10 machine with Docker Toolbox and instead of running ./backendless_start.sh script to start the services, I use this:

BL_MYSQL_PORT=3306 BL_MONGODB_PORT=27017 BL_PROPERTY_config_redis_bl_debug_port=6380 BL_PROPERTY_config_server_publicPort=9000 BL_PROPERTY_config_console_port=80 BL_PROPERTY_config_rt_server_socketServer_connection_port=5000 REGISTRY=backendless VERSION=6.1.8.mysql8 MOUNTS=/c/Users/bassam/Desktop/Backendless/setup_win/scripts/mounts docker stack deploy -c ./consul-compose.yml bl-swarm && \
BL_MYSQL_PORT=3306 BL_MONGODB_PORT=27017 BL_PROPERTY_config_redis_bl_debug_port=6380 BL_PROPERTY_config_server_publicPort=9000 BL_PROPERTY_config_console_port=80 BL_PROPERTY_config_rt_server_socketServer_connection_port=5000 REGISTRY=backendless VERSION=6.1.8.mysql8 MOUNTS=/c/Users/bassam/Desktop/Backendless/setup_win/scripts/mounts docker stack deploy -c ./pre-configuration-compose.yml bl-swarm && \
BL_MYSQL_PORT=3306 BL_MONGODB_PORT=27017 BL_PROPERTY_config_redis_bl_debug_port=6380 BL_PROPERTY_config_server_publicPort=9000 BL_PROPERTY_config_console_port=80 BL_PROPERTY_config_rt_server_socketServer_connection_port=5000 REGISTRY=backendless VERSION=6.1.8.mysql8 MOUNTS=/c/Users/bassam/Desktop/Backendless/setup_win/scripts/mounts docker stack deploy -c ./backendless-compose.yml bl-swarm && \
BL_MYSQL_PORT=3306 BL_MONGODB_PORT=27017 BL_PROPERTY_config_redis_bl_debug_port=6380 BL_PROPERTY_config_server_publicPort=9000 BL_PROPERTY_config_console_port=80 BL_PROPERTY_config_rt_server_socketServer_connection_port=5000 REGISTRY=backendless VERSION=6.1.8.mysql8 MOUNTS=/c/Users/bassam/Desktop/Backendless/setup_win/scripts/mounts docker stack deploy -c ./coderunner-js-compose.yml bl-swarm

ugly I know … but it’s working for me (for now at least). So, as you can see, I’m mounting the repo folder to my local path on the host machine. I found out, after some research, that this might be causing the slowliness. It’s reported a lot on Docker forums. you can check this thread if you like:

https://forums.docker.com/t/file-access-in-mounted-volumes-extremely-slow-cpu-bound/8076

So, I modified backendless-compose.yml to use a named volume for the repo (so it’s no longer mounted to the host). I also modified coderunner-js-compose.yml to use that volume (repo) as external volume.

After I done that, the slowliness issue is gone. So, now, I’m getting always:

16:14:30.043 Preparing app zip file for deployment..
16:14:38.424 2317 files added into deployment archive
16:14:39.139 Publishing Model to server
16:14:39.173 o296 > POST /servercode/publishcode/JS FormData {
  _overheadLength: 255,
  _valueLength: 39434446,
  _valuesToMeasure: [],
  writable: false,
  readable: true,
  dataSize: 0,
  maxDataSize: 2097152,
  pauseStreams: true,
  _released: false,
  _streams: [
    '----------------------------837419444683910176349632\r\n' +
      'Content-Disposition: form-data; name="model"\r\n' +
      '\r\n',
    'default',
    [Function: bound ],
    '----------------------------837419444683910176349632\r\n' +
      'Content-Disposition: form-data; name="code"; filename="code"\r\n' +
      'Content-Type: application/zip\r\n' +
      '\r\n',
    <Buffer 50 4b 03 04 0a 00 00 00 00 00 d3 81 8c 51 53 82 ab b5 6d b0 02 00 6d b0 02 00 0a 00 00 00 6d 6f 64 65 6c 2e 6a 73 6f 6e 7b 22 68 61 6e 64 6c 65 72 73 ... 39434389 more bytes>,
    [Function: bound ]
  ],
  _currentStream: null,
  _insideLoop: false,
  _pendingNext: false,
  _boundary: '--------------------------837419444683910176349632'
}
16:14:39.198 There will be no more data.
16:14:52.477 o296 < (1): 13301.648ms
16:14:52.518 PUSHING CODE: 13378.581ms
16:14:52.530 Successfully published
16:14:52.532 DONE!!
Done in 24.45s.

The total process is done in 24s instead of what used to be > 60s

After I’m done with that, I deployed my app and kept monitoring the mount repo folder on coderunner-js and noticed that my code was deployed to my PRODUCTION folder. Then, momentarily, it was GONE… then, it came back again and stayed there. So, not sure if the removal of code from PRODUCTION folder is part of the deployment process, but it was gone for seconds. may be this is supposed to happen, but before the workaround, it was not being put back again for some reason on the host machine.

Anyway, I think with this workaround, I should be able to start “development” on that project (and fast). I’ll keep monitoring and see if the problem comes back again.

Hi Bassam,

The deployment process removes all the code from the deployment model you’re deploying (see the bullet point in bold font here)

In our experience, the Windows environment is rather unstable and is not recommended. We never do any work with Docker Toolbox on Windows. We recommend going with either Linux or Mac OSX installation to avoid any potential problems.

Regards,
Mark

Hi Mark,

Thank you for the info. I know that deploying code will delete the old one (for the same model), but what was happening to me is that the new code in the PRODUCTION folder was being deleted and never come back again for some reason. I even suspected that my antivirus was the one removing it. I turned that off, but it didn’t help.

Also, back when I started installing Backendless Pro, I first tried on a Linux virtual machine (vagrant + virtualbox + docker) that will host all Backendless and followed your guidelines on how to install. That went fine, but for some reason, the mysql & redis services kept failing every now and then. Possibly whenever I suspended my pc or turn it off, the database files got corrupted for some reason. So, I had to install it too many times until I gave up and thought that I should follow a different approach.

When I started using Docker Toolbox, that problem was gone. But I faced the issue of slow deployments + deleting PRODUCTION folder content after deployment is done.

Anyway, with the current approach that I’ve described above, I think Backendless is now stable and fast. I was able to do some development today on it and my app was working just fine.

thanks for the help guys. I’ll let you know if something weird comes up again. hopefully, it won’t happen.

Hello @Bassam_Jarad

Thank you for contacting us. We are glad that you succeeded. Please let us know if you run into any other issues, we’re here to make sure you have a fantastic experience with Backendless.