HomeAI

GitHub outage lasted 7 hours and 47 minutes — Copilot recovery was delayed by retry loops

GitHub outage lasted 7 hours and 47 minutes — Copilot recovery was delayed by retry loops

On August 17, GitHub experienced a major outage that lasted 7 hours and 47 minutes, disrupting github.com, authentication, GitHub Actions, APIs, pull requests, issues, and Copilot. The incident affected developers and organizations worldwide. This was the second significant incident in August, following an Actions failure on August 6.

According to GitHub's investigation, the outage began when traffic reached a new peak, and a critical infrastructure component in the company's Central US data center failed to scale accordingly. The resulting capacity pressure cascaded through the systems, causing authentication failures and disrupting multiple GitHub services.

Recovery required several coordinated actions. Teams rerouted traffic, isolated affected infrastructure, and restored services in stages. Most GitHub services recovered earlier that day, but Copilot took longer. Errors in Copilot services triggered a client-side retry loop that increased traffic during recovery. GitHub had to mitigate that behavior before safely restoring traffic. The full root cause analysis includes a detailed technical timeline.

Neither outage was caused by a code or configuration change. Both incidents were capacity failures at their core. Critical components were not scaled before demand exceeded their capacity. Since April, monthly commits have grown from 1.4 billion to new record highs, putting additional strain on the infrastructure.

GitHub acknowledged that while progress has been made on reliability improvements since March and April, these incidents make clear that the work must accelerate. The company plans to strengthen capacity testing, improve auto-scaling mechanisms, and implement stricter safeguards against client-side retry loops. The full postmortem is available on GitHub's engineering blog.

Tags: Copilot
Slate (Sl8) — the new social network. Post, grow your audience and earn — plus staking rewards that actually pay.
Invite codehXA6hX
Join Slate