Workflows triggered by Git commits are currently delayed

Incident Report for CloudBees platform

Resolved

CloudBees has resolved the issues affecting Workflow Execution and Feature Management flags. The incident was related to an automated process causing a resource bottleneck and preventing legitimate Git webhooks from being handled in a timely manner. The aforementioned automated process has been stopped and protections have been put in place to prevent it from being re-enabled. The backlog of workflows have processed successfully as of 22:15 UTC. All systems are now fully operational and working as expected.
Posted Dec 08, 2025 - 22:51 UTC

Identified

CloudBees has identified the root cause of the delayed execution and is working to address it.
Posted Dec 08, 2025 - 21:58 UTC

Update

As we continue to troubleshoot this issue, we have determined that creating and updating existing flags is delayed. The scope of impact has been isolated and CloudBees is attempting to execution some remediation steps to address the impact.
Posted Dec 08, 2025 - 19:36 UTC

Update

CloudBees has received some reports indicating manually executed runs are also intermittently failing to start. Investigation in to root cause is still ongoing and additional information will be shared as it becomes available.
Posted Dec 08, 2025 - 17:32 UTC

Investigating

CloudBees has discovered an issue where workflows initiated by Git commits are not currently executing as expected since ~15:10 UTC. Investigation in to the root cause is currently underway.
Posted Dec 08, 2025 - 16:12 UTC
This incident affected: Feature Management (Feature Management - Flag creation, Feature Management - Flag Evaluation) and Workflow Execution.