All API Services wiped on deployment failure

Grant_Kerr · February 22, 2021, 1:33pm

Backendless Version (6.2.6)

Hi, I attempted to deploy a change to an existing codeless service however now all my custom API Services have disappeared. On the first deployment attempt an error was reported (which said that it would notify Backendless, as I presume it was an unexpected error). It was on a second attempt to deploy that resulted in all of my services disappearing (it actually said the deployment was successful).

Now when calling any of these I now get the response:

{"code":14019,"message":"Service not found","errorData":{}}

I’m on the eu server and my applicationID is:

A41C5EA6-A21F-FC2D-FF8E-807FEE106100

As you can see there are no services at all

Any idea on how this has happened, and are you able to re-deploy my services?

Many thanks,
Grant.

viktor.liablin · February 22, 2021, 5:08pm

Hello @Grant_Kerr

We are sorry for the inconvenience. I’ve created an internal ticket BKNDLSS-24200 to investigate your issue.
We can roll back your app to 21.02.2021 17:00 UTC. But data saved after this date will be deleted.
Do you agree to this?

Regard, Viktor

Grant_Kerr · February 22, 2021, 5:17pm

Hi @viktor.liablin,

That’d be great!

Many thanks,
Grant.

viktor.liablin · February 22, 2021, 5:42pm

@Grant_Kerr

We have roll back your app to 21.02.2021 17:00 UTC already. But services list still empty.
Do you remember the date, you had services? We have one backup for your app for each day until 08.02.2021

Regard, Viktor

viktor.liablin · February 22, 2021, 5:48pm

@Grant_Kerr

We have fixed it. Can you check?

Regard, Viktor

Grant_Kerr · February 22, 2021, 6:06pm

Hi @viktor.liablin ,

I’ve just refreshed and the services seem to be listed now. I’ve performed a quick check using my application and some are working but a few others seem to be in a strange state. The image shows one with code blocks floating where they previously weren’t and loop body (green block) containing no code where it previously did. The code in the window on the right reflects this.

It should actually look this this (i’m currently source-controlling server code via screenshots)

It’s almost as if the order in which these services being restored matters. The block within the loop was actually another custom service removeProject, so if removeArtist was restored before removeProject then it wouldn’t know how to construct it properly, as it would be missing that piece.

I can’t remember the exact time the wipe happened but it was between 17:00 and 19:00 UTC yesterday ( 21.02.2021).

If it helps, the modification I made was to add a pair of try/catch blocks using the codeless interface within one of my services (removeLayer), wrapping file system functions getting a directory listing. I’m not sure if anything of this nature could cause what happened but sometimes these things have strange origins.

If I spot any other unexpected behaviour i’ll keep you posted.

Many thanks,
Grant.

viktor.liablin · February 22, 2021, 7:05pm

@Grant_Kerr

The backup was created around failure time. Perhaps that is why it may be incomplete. If you want, we can try to roll back for 20.02.2021 17:00 UTC if you have to restore a lot.

Regard, Viktor

Grant_Kerr · February 22, 2021, 7:22pm

@viktor.liablin thanks but don’t worry about it. I’ll piece together and fix the issues I see and make sure i’m in a better position to restore from backup myself in future. I need to formalise a process of performing daily backups anyway tbh.

Thanks so much for your help in all of this.

Grant.

viktor.liablin · February 22, 2021, 7:26pm

@Grant_Kerr

Once again, we apologize for the inconvenience.
Happy coding.

Regard, Viktor