Degraded performance for clients in the Asia Pacific region
Incident Report for ftrack
Postmortem

Root Cause:

During routine maintenance, a configuration change was made to the Kubernetes clusters in Google Cloud, which inadvertently caused an internal service to stop responding. This caused a partial outage and affected the ability of users to access certain features.

Resolution:

Our team immediately identified the issue and worked to resolve it as quickly as possible. The configuration change was rolled back, which restored the service.

Impact:

During the partial outage, users may have experienced difficulty accessing certain features. We apologize for any inconvenience this may have caused.

Next Steps:

We will continue working closely with our cloud service providers to ensure our services remain reliable and available to our users. We will also thoroughly review our change management policies and procedures to prevent similar incidents in the future.

Posted Mar 08, 2023 - 09:42 CET

Resolved
This incident has been resolved.
Posted Mar 08, 2023 - 00:15 CET
Monitoring
A fix has been implemented and we are monitoring the results.
Posted Mar 08, 2023 - 00:14 CET
Identified
Due to a configuration change in the Asia Pacific region, ftrack workspaces hosted in the region are experiencing a partial outage.

We have identified the issue and are implementing a fix.
Posted Mar 07, 2023 - 23:30 CET
This incident affected: ftrack Services and Components (Hosted applications).