All API Services wiped on deployment failure

[Grant_Kerr]

Backendless Version (6.2.6)

Hi, I attempted to deploy a change to an existing codeless service however now all my custom API Services have disappeared. On the first deployment attempt an error was reported (which said that it would notify Backendless, as I presume it was an unexpected error). It was on a second attempt to deploy that resulted in all of my services disappearing (it actually said the deployment was successful).

Now when calling any of these I now get the response:

{"code":14019,"message":"Service not found","errorData":{}}

I’m on the eu server and my applicationID is:

A41C5EA6-A21F-FC2D-FF8E-807FEE106100

As you can see there are no services at all

Any idea on how this has happened, and are you able to re-deploy my services?

Many thanks,
Grant.

[viktor.liablin]

Hello @Grant_Kerr

We are sorry for the inconvenience. I’ve created an internal ticket BKNDLSS-24200 to investigate your issue.
We can roll back your app to 21.02.2021 17:00 UTC. But data saved after this date will be deleted.
Do you agree to this?

Regard, Viktor

[Grant_Kerr]

Hi @viktor.liablin,

That’d be great!

Many thanks,
Grant.

[viktor.liablin]

@Grant_Kerr

We have roll back your app to 21.02.2021 17:00 UTC already. But services list still empty.
Do you remember the date, you had services? We have one backup for your app for each day until 08.02.2021

Regard, Viktor

[viktor.liablin]

@Grant_Kerr

We have fixed it. Can you check?

Regard, Viktor

[Grant_Kerr]

Hi @viktor.liablin ,

I’ve just refreshed and the services seem to be listed now. I’ve performed a quick check using my application and some are working but a few others seem to be in a strange state. The image shows one with code blocks floating where they previously weren’t and loop body (green block) containing no code where it previously did. The code in the window on the right reflects this.

It should actually look this this (i’m currently source-controlling server code via screenshots)

It’s almost as if the order in which these services being restored matters. The block within the loop was actually another custom service removeProject, so if removeArtist was restored before removeProject then it wouldn’t know how to construct it properly, as it would be missing that piece.

I can’t remember the exact time the wipe happened but it was between 17:00 and 19:00 UTC yesterday ( 21.02.2021).

If it helps, the modification I made was to add a pair of try/catch blocks using the codeless interface within one of my services (removeLayer), wrapping file system functions getting a directory listing. I’m not sure if anything of this nature could cause what happened but sometimes these things have strange origins.

If I spot any other unexpected behaviour i’ll keep you posted.

Many thanks,
Grant.

[viktor.liablin]

@Grant_Kerr

The backup was created around failure time. Perhaps that is why it may be incomplete. If you want, we can try to roll back for 20.02.2021 17:00 UTC if you have to restore a lot.

Regard, Viktor

[Grant_Kerr]

@viktor.liablin thanks but don’t worry about it. I’ll piece together and fix the issues I see and make sure i’m in a better position to restore from backup myself in future. I need to formalise a process of performing daily backups anyway tbh.

Thanks so much for your help in all of this.

Grant.

[viktor.liablin]

@Grant_Kerr

Once again, we apologize for the inconvenience.
Happy coding.

Regard, Viktor

[Grant_Kerr]

Hi Viktor,

I’m just replying on this older post as this issue has happened again. All my services have been wiped completely. I remember this happening around the same time of day as last time (possibly while your system backups are being made?). Is there any chance you could redeploy them please? They were wiped Sunday 4th April @ around 16:48 GMT.

Many thanks,
Grant.

[sergey.kuk (Sergey Kukurudzyak)]

Hello @Grant_Kerr ,
I am able to see service in two models publishing and published

[Grant_Kerr]

Hi @sergey.kuk,

I’ve seen the methods within the published model appearing now. I assume you took steps to restore them? Is this a manual process of restoration, because (like when they were previously restored) some of them don’t restore correctly. I believe this to be because some of the methods are used by other methods, and if the child methods aren’t restored first, then the methods using them can’t be rebuilt properly (as they don’t exist yet). I can manually rebuild these but wondered if this is either a bug in your restore process or something I should be more specific about if it happens again? (i.e. the order of restoration)

Many thanks,
Grant.

[sergey.kuk (Sergey Kukurudzyak)]

@Grant_Kerr I do not do anything, I have just connected to your app and make sure that the services are shown. Next time try to open Backendless Console without cache(or in incognito tab)

[Grant_Kerr]

Cheers I’ll try that if this happens again in future.