We've identified and resolved an issue that was preventing some project trajectory reversions from completing successfully. The problematic code change has been reverted and deployed, and our systems are now operating normally.
What happened
A code change deployed at 12:35 PM UTC introduced a compatibility issue that caused project reversions to fail. This affected approximately 92% of reversion requests during the incident window.
What we did
Our engineering team quickly identified the root cause, reverted the change, and completed the fix at approximately 12:57 PM UTC. The corrected code is now live.
Current status
Project reverts are working normally. If you attempted to reverse a project trajectory during the incident and encountered an error, please try again now.
We apologize for the disruption and appreciate your patience while we resolved this issue.
Resolved
We've identified and resolved an issue that was preventing some project trajectory reversions from completing successfully. The problematic code change has been reverted and deployed, and our systems are now operating normally.
What happened
A code change deployed at 12:35 PM UTC introduced a compatibility issue that caused project reversions to fail. This affected approximately 92% of reversion requests during the incident window.
What we did
Our engineering team quickly identified the root cause, reverted the change, and completed the fix at approximately 12:57 PM UTC. The corrected code is now live.
Current status
Project reverts are working normally. If you attempted to reverse a project trajectory during the incident and encountered an error, please try again now.
We apologize for the disruption and appreciate your patience while we resolved this issue.
Investigating
We are currently investigating an issue affecting the project revert functionality. A higher-than-normal error rate is occurring on requests to revert project trajectories.
What's happening
Our monitoring detected that requests to our project revert endpoint are failing with server errors. We've identified the root cause as a recent code change deployed at 12:35 PM UTC that introduced an incompatibility in our workflow system, causing the workflow payload to fail decoding.
What we're doing
Our engineering team is actively rolling back the problematic change to restore full functionality.
Impact
If you attempted to revert a project trajectory and encountered an error, please try again once we've completed the rollback. We apologize for any inconvenience this may have caused.
