[URGENT] Very slow response time today from the app, web, and console + Bad request errors

@Marian_Koreniuk the business logic seems dead

{
“code”: 15000,
“message”: “Custom business logic execution has been terminated because it did not complete in permitted time - 20 seconds”
}

this is the error we get often, we routinely experience 2 issues: 1) everything offline, we discussed this a few times which lead to the deletion of a large part of the database and a migration to a different infrastructure with more resources 2) business logic not working, this one we often do not notice since the service otherwise works, but then causes issues to users that are trying to purchase plans (since we need the business logic to work for that).

please let me know when this is resolved, thank you

@Alexander_Pavelko @stanislaw.grin @Marian_Koreniuk @Andriy_Konoz

Hello, @Alessandra_Saviotti

I checked that the business logic is working, but specify which method you are asking to get this error?
I need exactly the steps you take to reproduce this error.

it’s working now, to reproduce, normally you can just try a Get from the Users table, as explained above in this thread.

again the business logic is not working

@Alexander_Pavelko @stanislaw.grin @Marian_Koreniuk @Andriy_Konoz

{
“code”: 15000,
“message”: “Custom business logic execution has been terminated because it did not complete in permitted time - 20 seconds”
}

this happens on any request for tables that have business logic involved, even if the operation is minimal and normally takes no time, you can just run a GET on users to experience the same

please resolve ASAP, this is happening very frequently now and prevents users from buying the product (quite a problem)

please reply if you are fixing this, it has been more than a day already

Hello, @Alessandra_Saviotti

so it’s fixed.

I want to know if you receive messages by e-mail indicating that you are resting on the limit of 1800 requests per minute?

Because the rest of the incoming requests, you will receive exactly the service unavailability error. That is, some requests are processed and some are not.

this is not about the limit (I know what you mean, that’s just about the number of requests), here something else is broken (as it is often see also this: Urgent: bad request (in production, prevents users from purchasing our product) - #43 by Marian_Koreniuk )

please check, because nothing is fixed, it is not working and not a single request goes through when the business logic is involved (try a GET on Users to reproduce)

@Alessandra_Saviotti So I checked, I see that the request of the user table goes through. Please check again.

Yes, you are right, the limit does not affect it. But I just wanted to clarify that you receive messages about limit and that they do not end up in spam and you see them.

@Marian_Koreniuk , nothing is working here, please login to console, go to Users, and then do a Get, and you will receive the same message (“Custom business logic execution has been terminated because it did not complete in permitted time - 20 seconds”, for every single request). I see the same from our web frontend (bad request), we have been down for an entire day when it comes to the business logic.

now it is resolved, can you tell me exactly what happened and what you did to resolve? this way it might be quicker next time if I know how to point you in the right direction, thanks

By the way, JS CodeRunner was working, but somehow it got stuck and did not execute requests, I restarted CodeRunner, and after that it really fixed itself. It is the future to figure it out.

1 Like

Just woke up and we have been down for 2 hours (and are still down). Please check ASAP @Marian_Koreniuk

Hello, @Alessandra_Saviotti

So there was a problem, work has been resumed. I am dealing with the causes of the incident.

Could you check now if it’s okay on your app?

thank you, seems good

@Marian_Koreniuk please double-check, we are offline once again

maybe you need to restart coderunner again: is there something here that can be automated? it seems it often stops working and our customers cannot purchase the product

@Alexander_Pavelko @stanislaw.grin @Andriy_Konoz

whole system seems also offline frequently this morning

please help, it’s been several hours already

is there anyone here?

please reply @Marian_Koreniuk @mark-piller

Hello, @Alessandra_Saviotti
coderunner stuck again, now restarted. I am investigating the problem.

Should have everything working now.

thanks, please let’s resolve this