Three overlapping infrastructure bugs silently degraded Claude API output quality for a month

Anthropic (Claude API, Bedrock, and Vertex AI users)·Claude API serving infrastructureFixed

A misrouting bug sent short-context Sonnet 4 requests to servers configured for the 1M-token context window, a TPU serving misconfiguration caused token-generation corruption (e.g. stray Thai/Chinese characters or syntax errors in English/code responses), and a separate XLA:TPU compiler bug hit token selection for Haiku 3.5 and other models.

Blast radius

At the worst hour on August 31, 2025, 16% of Sonnet 4 requests were misrouted; about 30% of Claude Code users hit at least one degraded response during the period. Output corruption affected Opus 4.1, Opus 4, and Sonnet 4 requests between August 25 and September 2.

Root cause

Three separate infrastructure changes (routing logic, a TPU serving misconfiguration, and a compiler-triggering optimization) overlapped in time, and sticky routing meant affected users kept hitting the same bad server, producing confusing, inconsistent reports.

How it was caught

User reports of degraded responses beginning in early August were initially indistinguishable from normal feedback variance; a formal investigation opened in late August traced them to three distinct bugs.

What changed

Routing logic was corrected and rolled out across platforms by September 18; the TPU misconfiguration was rolled back September 2 with new output-anomaly detection tests added; the compiler bug was mitigated for affected models.

Take-awayWhen you serve one model across multiple hardware backends, add automated output-quality canaries per backend, since sticky routing can hide a bug from most users while a minority get every response from the broken path.