SSL certificate intermittent issues with Zapier

Ian_Rutner · October 27, 2022, 3:37pm

Backendless Version (3.x / 6.x, Online / Managed / Pro )

Cloud 99

Client SDK (REST / Android / Objective-C / Swift / JS )

REST

Application ID

86DF97EF-A838-427E-A069-5F4DC2B95285

Expected Behavior

Please describe the expected behavior of the issue, starting from the first action.

Zapier makes a POST request to Backendless cloud code endpoint https://bijouwash.backendless.app/api/services/Tracking/status
Code is executed

Actual Behavior

Zapier makes a POST request to Backendless cloud code endpoint https://bijouwash.backendless.app/api/services/Tracking/status
Zapier task has intermittent failure with the following error: Could not verify SSL certificate. Please check certificate installation for domain!
Many of the Zapier tasks make the POST request successfully, but some fail with the above error.

mark-piller · October 27, 2022, 3:42pm

Hello,

This is the information about the certificate we see. There is nothing here that would indicate the certificate is not valid:

Could you reach out to Zapier to find out what specific problem they are seeing?

Regards,
Mark

Ian_Rutner · October 27, 2022, 4:35pm

Hi Mark,

I reached out to Zapier and was able to get a bit more information. Here is the error in their logs:

SSLError ExternalHTTPSConnectionPool(host=‘bijouwash.backendless.app’, port=443): Max retries exceeded with url: /api/services/Tracking/status (Caused by SSLError(SSLError(“bad handshake: Error([(‘SSL routines’, ‘tls_process_server_certificate’, ‘certificate verify failed’)])”)))

Do you have any recommendations for how to proceed?

mark-piller · October 27, 2022, 4:44pm

Hi Ian,

Does your API service make a request to a third-party server?

Regards,
Mark

Ian_Rutner · October 27, 2022, 4:54pm

Hi Mark,

No, that endpoint just takes the POST request, loads some data tables from backendless, and updates the data in those tables depending on what is in the body.

mark-piller · October 27, 2022, 5:02pm

Hi Ian,

I added a test GET endpoint in your API service. Could you ask Zapier support to try fetching the following URL from their environment:

https://bijouwash.backendless.app/api/services/Tracking/SSLCertTest

There must be something going on on their network. We do not see any issues on our side. Here’s an independent report on the deployed SSL cert (feel free to share it with Zapier):
https://www.ssllabs.com/ssltest/analyze.html?d=bijouwash.backendless.app

Regards,
Mark

Ian_Rutner · November 1, 2022, 6:51pm

Hi Mark,

I finally heard back from Zapier and they suspect that the issue is due to provisioning inconsistencies in the server pool used by Backendless. They asked me to check with you to ensure that all of the servers in your server pool are provisioned correctly and that TLS support is consistent across the server pool.

Please let me know if there is anything else you would like me to ask for. I’m still unable to resolve this intermittent issue.

Thanks,
Ian

Marian_Koreniuk · November 2, 2022, 2:54pm

Hello, @Ian_Rutner
I checked from our side only one server is engaged in SSL handshake, there is also a spare server that is not used. I checked all the servers that can participate in this, everywhere the certificates respond equally correctly.
What exactly did Zapier’s support say about the verification from an independent source, which is available at this SSL Server Test: bijouwash.backendless.app (Powered by Qualys SSL Labs) link?

We checked several times and the answer of this service was always correct.

It is possible that some of the TLS modes are not supported bt Zapier, this needs to be clarified.

Ian_Rutner · November 3, 2022, 3:33pm

Hi Marian,

Zapier mentioned that different TLS checkers return different results for our domain.

They sent over these two examples:
https://geekflare.com/tools/tests/vjruv1nxe

Can you check the TLS 1.3 support on your servers to make sure it is configured correctly?
Zapier will get back to me if they have any new updates as they are continuing to investigate this.

Thanks,
Ian

Marian_Koreniuk · November 3, 2022, 4:57pm

Hello, @Ian_Rutner

I looked at the reports described. I see that they are the same, but I have one guess, that’s why I made some changes in the priority of certain encryption algorithms that are supported in TLS 1.3, I made changes. Could you check from your side or indicate how to check that your logic works?

Luc_Zentar · November 4, 2022, 12:00pm

I have started to experience the same problem, hopefully the information below will help:

I have 2 environments, one is a custom domain (production) and one is not custom domain (development).

I have just moved into a WeWork office, and the problem only exists when using their wifi.

The custom domain is not impacted, but the other environment shows this error only when connected to their wifi. If I tether to my phone network, the problem doesn’t occur.

I’ve raised with WeWork and am waiting for a response, but thought this may help.

Ian_Rutner · November 4, 2022, 3:23pm

Hey Luc,

We also saw a similar issue when connected to an office network, and it was due to the firewall injecting a different certificate. Seems like it is unrelated to the issue here, but that may be helpful for you in tracking down what is going on at your WeWork office.