Summary

A root certificate in the certificate bundle used by hosted sites expired unexpectedly on May 30, 2020.

API clients may have experienced issues connecting to hosted sites during the period from 10:48 UTC to circa 15:30 UTC.

Modern browsers bundled newer versions of the root certificate and had no problem connecting to hosted sites.

Cause

The AddTrust External CA Root certificate expired May 30, 2020 at 10:48 UTC. This certificate was used to cross-sign certificates for ftrackapp.com and ftrackapp.cn by our SSL provider.

We updated our certificates on Feb 13, 2020. The certificate bundle provided by our SSL provider included root certificates which were expiring soon, which we failed to identify.

Resolution

We updated affected services with new certificates which contained updated intermediate certificates.

New certificate chain:

1. *.ftrackapp.com
2. Sectigo RSA Domain Validation Secure Server CA
3. USERTrust RSA Certification Authority
4. AAA Certificate Services

The certificates should be widely trusted. If you are still seeing SSL errors from API or integrations, please see “Additional information for continued issues” below or contact support.

Post-mortem

We will investigate and take action to minimize the risk of SSL certificates expiring unexpectedly in the future.

We will also look over our internal processes and policies to ensure information about an outage is available to our customers on our status page (status.ftrack.com) as soon as possible during an outage.

Additional information for continued issues

A root SSL certificate (AddTrust External CA Root) which we used to sign our certificates with expired on May 30, 2020. We updated our certificates to use new root certificates which should be widely accepted, but in rare cases it is not included in the trusted certificate store.

When connecting to your workspaces, the different clients use different certificates as trusted root certificates.

• Browsers, such as Google Chrome, bundles a list of certificates which are trusted. Make sure your browser is up to date if you are having issues with SSL in the browser.
• The new Python API uses Python requests, which under hood relies on the certifi package for trusted root certificates. Update the certifi package if you have issues with SSL from the new Python API.
• Connect and the legacy Python API also uses urllib2 for communicating with the server. This relies on the default behavior of Python’s SSL module. Windows loads CA certs from the CA and ROOT system stores. On other systems it calls SSLContext.set_default_verify_paths().
  

Microsoft should handle updates to certificates through windows update, but there may be other means of updating them as well. If you want to download and install the root certificates required manually, you can find them here: https://support.sectigo.com/Com_KnowledgeDetailPage?Id=kA01N000000rfBO. SHA-2 Root : USERTrust RSA Certification Authority at the bottom is what you need.

Please contact support if you need any more assistance with this.

More information from our SSL Service provider available here: https://support.sectigo.com/articles/Knowledge/Sectigo-AddTrust-External-CA-Root-Expiring-May-30-2020