Blog

Notes from the arena

What the head-to-head runs actually show, how the methodology works, and takes you can disagree with by voting.

Release reports

Method notes

Takes & analysis

Grok 4.5 Shipped With No Benchmarks. GPT-5.6 Shipped With Gamed Ones.

· 7 min

Two flagship launches landed one day apart this week, and neither gave you a number you could trust from the vendor. That is not a coincidence, it is the state of the industry.

Two Ways to Lose Trust: Models You Can't Test, Leaderboards That Grade Their Investors

· 6 min

This week's news cycle has Mythos and GPT-5.6 locked behind government clearance while Arena's leaderboard takes money from the labs it ranks. Both problems point at the same gap: independent, blind, live testing.

GPT-5.6 vs Claude Fable 5: Which Wins on Coding?

· 5 min

GPT-5.6 vs Claude Fable 5: pricing, thinking-effort levels, token appetite and real cost-per-task — plus where to watch both flagships build from one prompt, live.

GPT-5.6 vs GPT-5.5: Is the Upgrade Worth It?

· 4 min

GPT-5.6 vs GPT-5.5: what the Sol, Terra and Luna tiers change, how $5/$30 Sol compares to the $1.25/$10 flagship it replaces, and whether it one-shots better code.

GPT-5.6 vs Grok 4.5: Coding Head-to-Head

· 4 min

GPT-5.6 vs Grok 4.5: OpenAI’s $5/$30 flagship against xAI’s $2/$6 price shot with 500k context. Pricing, speed and live one-shot coding outputs, community-voted.

GPT-5.6 vs Gemini 3: Same Prompt, Real Outputs

· 4 min

GPT-5.6 vs Gemini: OpenAI’s $30 reasoning flagship meets Google’s fast, cheap Gemini 3 Flash on identical one-shot coding prompts — live outputs, community votes.

GPT-5.6 vs DeepSeek V4: Closed vs Open Weights

· 4 min

GPT-5.6 vs DeepSeek V4: OpenAI’s $5/$30 flagship against an open-weight model at ~$0.14/$0.28. What a ~35x price gap buys on one-shot coding, judged live.

Grok 4.5 vs GPT-5.6: The Newest Flagships, Compared

· 4 min

Grok 4.5 vs GPT-5.6: xAI’s $2/$6 flagship with 500k context meets OpenAI’s $5/$30 Sol on identical one-shot coding prompts — live outputs, community votes decide.

Claude vs ChatGPT for Coding: The Honest Comparison

· 4 min

Claude vs ChatGPT for coding, compared honestly: same one-shot prompts, live outputs, blind community votes. Quality vs cost — and which to pick day-to-day.

Gemini vs ChatGPT for Coding: Speed vs Reasoning

· 4 min

Gemini vs ChatGPT for coding: Gemini 3 Flash’s speed vs GPT-5.5’s paid reasoning, run on identical one-shot prompts and judged live by blind community votes.

DeepSeek vs ChatGPT for Coding: Real Outputs Compared

· 4 min

DeepSeek vs ChatGPT for coding: DeepSeek V4 Flash and GPT-5.5 compared on price, open weights, and real one-shot builds you can watch live and judge yourself.

ChatGPT vs Grok for Coding: Which Ships Better Code?

· 4 min

ChatGPT vs Grok for coding in 2026: GPT-5.5 and Grok 4.5 compared on price, speed, and one-shot code quality, with live community-voted head-to-head results.

Gemini vs Claude for Coding: Fast vs Frontier

· 4 min

Gemini vs Claude for coding: Google’s near-free Gemini 3 Flash meets Anthropic’s $50-output flagship Claude Fable 5 on identical one-shot prompts, voted live.

DeepSeek vs Claude: Open Weights Against the Flagship

· 5 min

DeepSeek vs Claude for coding: V4 Flash’s open-weight $0.14/$0.28 pricing against Fable 5’s $10/$50 flagship — and where to watch both build the same prompt, live.

Qwen vs DeepSeek: The Open-Weights Coding Duel

· 4 min

Qwen vs DeepSeek: Qwen3.5 27B and DeepSeek V4 Flash go head-to-head in a live, community-voted coding arena. Open weights, FP8 serving, self-host value.

GPT vs Gemini for Coding: Which Should You Pick?

· 4 min

GPT vs Gemini for coding: we run GPT-5.5 and Gemini 3 Flash on identical one-shot prompts, live in the browser, and let blind community votes pick the winner.

Grok vs GPT-5 for Coding: Real-World Outputs

· 4 min

Grok vs GPT-5 for coding, settled with live one-shot outputs and blind community votes: Grok 4.5’s $2/$6 pricing and 500k context against GPT-5.5’s $1.25/$10.

Qwen vs Claude: Can Open Weights Match the Flagship?

· 4 min

Qwen vs Claude for coding: Alibaba's open-weight Qwen3.5 27B takes on Claude Fable 5 in live, one-shot arena matchups. Can a free download match a $50/M flagship?

Kimi vs Claude for Coding: Agentic Open Weights Tested

· 6 min

Kimi vs Claude for coding: Moonshot’s open-weight K2.7 Code faces Anthropic’s Fable 5 flagship on identical one-shot prompts, judged live by blind community votes.

DeepSeek vs Gemini for Coding: Value or Speed?

· 4 min

DeepSeek vs Gemini: open-weight DeepSeek V4 Flash ($0.14/$0.28) meets Google’s fast Gemini 3 Flash on identical one-shot coding prompts — live outputs, community votes.

Kimi vs ChatGPT for Coding: Open Agentic vs Closed

· 4 min

Kimi vs ChatGPT for coding: Moonshot’s open-weight agentic K2.7 Code takes on GPT-5.5’s closed API. Pricing, self-hosting and live one-shot builds, judged by votes.

Claude vs DeepSeek: What $50 per Million Tokens Buys

· 4 min

Claude vs DeepSeek on price: Fable 5’s $50-per-million output vs DeepSeek V4 Flash at $0.28 — a ~180x gap, what a finished one-shot app costs, live in the arena.

Grok 4.5 vs Claude Opus 4.8: Flagship Coding Test

· 4 min

Grok 4.5 vs Opus 4.8: xAI’s $2/$6 newcomer against Anthropic’s $25-output vote leader. Pricing, effort levels and live one-shot coding builds — community-voted.

GLM-5.2 vs DeepSeek V4: Open-Weights Coding Kings

· 4 min

GLM vs DeepSeek: GLM-5.2’s 744B MoE meets DeepSeek V4 Flash in a live, community-voted coding arena. Both open weights, both FP8 — self-host vs API economics.

Llama 4 vs GPT for Coding: Open Weights Against OpenAI

· 4 min

Llama vs GPT for coding: Llama 4 Maverick and its open MoE weights face OpenAI's closed GPT-5.5 in a live, community-voted one-shot arena. Specs and stakes.

MiniMax vs Qwen: Open-Weights Coding Compared

· 4 min

MiniMax vs Qwen: MiniMax M2.7 and Qwen3.5 27B face off in a live, community-voted coding arena. Open weights, MoE vs dense, FP8 serving, and the self-host math.

The Best AI for Coding in 2026, Judged Live in the Arena

· 9 min

Which AI is best for coding? Instead of quoting benchmarks, we give every model the same one-shot prompt, run the real outputs live, and let blind votes decide. Here is what the arena currently says — plus the best free options and picks for Python, vibe coding, and coding agents.

GLM-5.2 "beats" Claude Code. Same week, a different benchmark says Opus wins by 14 points

· 7 min

Three benchmark stories about GLM-5.2 broke the same week, and they don't agree with each other. The disagreement is the actual story: single leaderboard numbers move with the harness, not just the model.

Karp Is Right That Tokens Get Wasted. He's Fixing the Wrong Thing

· 6 min

Palantir's CEO says enterprises are paying for tokens that create no value, and he has a point. But the fix isn't hiding the meter behind credits, it's watching what a token spend actually produces.

Does thinking effort actually matter? Same model, low vs max

· 5 min

Opus 4.8 and Sonnet ship at up to six effort levels each. The arena lets you blind-compare a model against itself — and the output data says effort is not the dial you think it is.