Anthropic's New Tokenizer Generates Up to 73% More Tokens Than GPT-5.6, Blunting Advertised Savings

Anthropic's newest tokenizer, now used by Claude Sonnet 5, Opus 4.8 and Opus 4.7, encodes the same text into noticeably more tokens than the tokenizer it replaces, which raises effective costs even where list prices look flat or lower.

• Anthropic says the same input can map to roughly 1.0-1.35x more tokens depending on content type
• For a sample TypeScript file, the new tokenizer emitted 1.73x more tokens than GPT-5.6 Sol's o200k tokenizer and 1.32x more than Claude's prior tokenizer
• Other languages showed similar gaps: Rust 1.58x, JavaScript 1.52x, Python 1.50x more tokens than equivalent OpenAI tokenization
• Sonnet 5's introductory rate of $2/$10 per million input/output tokens holds through August 31, 2026, then rises to $3/$15

Because coding workloads are especially token-dense, developers comparing Claude against GPT-5.6 Sol on sticker price alone may find actual per-task spend closer than the published rate cards suggest once the tokenizer difference is factored in.