Blog

Notes from the arena

What the head-to-head runs actually show, how the methodology works, and takes you can disagree with by voting.

Release reports

Method notes

Takes & analysis

GPT-5.6, Grok 4.5, Muse Spark: Three Efficiency Claims This Week, Zero Common Yardstick

· 6 min

OpenAI, xAI, and Meta each shipped a model this week with a headline efficiency number attached. None of the three numbers were measured the same way, which is exactly why blind arena votes matter more than vendor math.

The Week "Aggressive Pricing" Stopped Meaning Anything

· 5 min

Meta, xAI, and OpenAI all shipped new models in a 48-hour span and all three called their pricing the headline. The actual per-token numbers don't agree with each other or with the adjectives.

Grok 4.5 Shipped With No Benchmarks. GPT-5.6 Shipped With Gamed Ones.

· 7 min

Two flagship launches landed one day apart this week, and neither gave you a number you could trust from the vendor. That is not a coincidence, it is the state of the industry.

Two Ways to Lose Trust: Models You Can't Test, Leaderboards That Grade Their Investors

· 6 min

This week's news cycle has Mythos and GPT-5.6 locked behind government clearance while Arena's leaderboard takes money from the labs it ranks. Both problems point at the same gap: independent, blind, live testing.

GPT-5.6 vs Claude Fable 5: Which Wins on Coding?

· 5 min

GPT-5.6 vs Claude Fable 5: pricing, thinking-effort levels, token appetite and real cost-per-task — plus where to watch both flagships build from one prompt, live.

GPT-5.6 vs GPT-5.5: Is the Upgrade Worth It?

· 4 min

GPT-5.6 vs GPT-5.5: what the Sol, Terra and Luna tiers change, how $5/$30 Sol compares to the $1.25/$10 flagship it replaces, and whether it one-shots better code.

GPT-5.6 vs Grok 4.5: Coding Head-to-Head

· 4 min

GPT-5.6 vs Grok 4.5: OpenAI’s $5/$30 flagship against xAI’s $2/$6 price shot with 500k context. Pricing, speed and live one-shot coding outputs, community-voted.

GPT-5.6 vs Gemini 3: Same Prompt, Real Outputs

· 4 min

GPT-5.6 vs Gemini: OpenAI’s $30 reasoning flagship meets Google’s fast, cheap Gemini 3 Flash on identical one-shot coding prompts — live outputs, community votes.

GPT-5.6 vs DeepSeek V4: Closed vs Open Weights

· 4 min

GPT-5.6 vs DeepSeek V4: OpenAI’s $5/$30 flagship against an open-weight model at ~$0.14/$0.28. What a ~35x price gap buys on one-shot coding, judged live.

Grok 4.5 vs GPT-5.6: The Newest Flagships, Compared

· 4 min

Grok 4.5 vs GPT-5.6: xAI’s $2/$6 flagship with 500k context meets OpenAI’s $5/$30 Sol on identical one-shot coding prompts — live outputs, community votes decide.

Claude vs ChatGPT for Coding: The Honest Comparison

· 4 min

Claude vs ChatGPT for coding, compared honestly: same one-shot prompts, live outputs, blind community votes. Quality vs cost — and which to pick day-to-day.

Gemini vs ChatGPT for Coding: Speed vs Reasoning

· 4 min

Gemini vs ChatGPT for coding: Gemini 3 Flash’s speed vs GPT-5.5’s paid reasoning, run on identical one-shot prompts and judged live by blind community votes.

DeepSeek vs ChatGPT for Coding: Real Outputs Compared

· 4 min

DeepSeek vs ChatGPT for coding: DeepSeek V4 Flash and GPT-5.5 compared on price, open weights, and real one-shot builds you can watch live and judge yourself.

ChatGPT vs Grok for Coding: Which Ships Better Code?

· 4 min

ChatGPT vs Grok for coding in 2026: GPT-5.5 and Grok 4.5 compared on price, speed, and one-shot code quality, with live community-voted head-to-head results.

Gemini vs Claude for Coding: Fast vs Frontier

· 4 min

Gemini vs Claude for coding: Google’s near-free Gemini 3 Flash meets Anthropic’s $50-output flagship Claude Fable 5 on identical one-shot prompts, voted live.

DeepSeek vs Claude: Open Weights Against the Flagship

· 5 min

DeepSeek vs Claude for coding: V4 Flash’s open-weight $0.14/$0.28 pricing against Fable 5’s $10/$50 flagship — and where to watch both build the same prompt, live.

Qwen vs DeepSeek: The Open-Weights Coding Duel

· 4 min

Qwen vs DeepSeek: Qwen3.5 27B and DeepSeek V4 Flash go head-to-head in a live, community-voted coding arena. Open weights, FP8 serving, self-host value.

GPT vs Gemini for Coding: Which Should You Pick?

· 4 min

GPT vs Gemini for coding: we run GPT-5.5 and Gemini 3 Flash on identical one-shot prompts, live in the browser, and let blind community votes pick the winner.

Grok vs GPT-5 for Coding: Real-World Outputs

· 4 min

Grok vs GPT-5 for coding, settled with live one-shot outputs and blind community votes: Grok 4.5’s $2/$6 pricing and 500k context against GPT-5.5’s $1.25/$10.

Qwen vs Claude: Can Open Weights Match the Flagship?

· 4 min

Qwen vs Claude for coding: Alibaba's open-weight Qwen3.5 27B takes on Claude Fable 5 in live, one-shot arena matchups. Can a free download match a $50/M flagship?

Kimi vs Claude for Coding: Agentic Open Weights Tested

· 6 min

Kimi vs Claude for coding: Moonshot’s open-weight K2.7 Code faces Anthropic’s Fable 5 flagship on identical one-shot prompts, judged live by blind community votes.

DeepSeek vs Gemini for Coding: Value or Speed?

· 4 min

DeepSeek vs Gemini: open-weight DeepSeek V4 Flash ($0.14/$0.28) meets Google’s fast Gemini 3 Flash on identical one-shot coding prompts — live outputs, community votes.

Kimi vs ChatGPT for Coding: Open Agentic vs Closed

· 4 min

Kimi vs ChatGPT for coding: Moonshot’s open-weight agentic K2.7 Code takes on GPT-5.5’s closed API. Pricing, self-hosting and live one-shot builds, judged by votes.

Claude vs DeepSeek: What $50 per Million Tokens Buys

· 4 min

Claude vs DeepSeek on price: Fable 5’s $50-per-million output vs DeepSeek V4 Flash at $0.28 — a ~180x gap, what a finished one-shot app costs, live in the arena.

Grok 4.5 vs Claude Opus 4.8: Flagship Coding Test

· 4 min

Grok 4.5 vs Opus 4.8: xAI’s $2/$6 newcomer against Anthropic’s $25-output vote leader. Pricing, effort levels and live one-shot coding builds — community-voted.

GLM-5.2 vs DeepSeek V4: Open-Weights Coding Kings

· 4 min

GLM vs DeepSeek: GLM-5.2’s 744B MoE meets DeepSeek V4 Flash in a live, community-voted coding arena. Both open weights, both FP8 — self-host vs API economics.

Llama 4 vs GPT for Coding: Open Weights Against OpenAI

· 4 min

Llama vs GPT for coding: Llama 4 Maverick and its open MoE weights face OpenAI's closed GPT-5.5 in a live, community-voted one-shot arena. Specs and stakes.

MiniMax vs Qwen: Open-Weights Coding Compared

· 4 min

MiniMax vs Qwen: MiniMax M2.7 and Qwen3.5 27B face off in a live, community-voted coding arena. Open weights, MoE vs dense, FP8 serving, and the self-host math.

The Best AI for Coding in 2026, Judged Live in the Arena

· 9 min

Which AI is best for coding? Instead of quoting benchmarks, we give every model the same one-shot prompt, run the real outputs live, and let blind votes decide. Here is what the arena currently says — plus the best free options and picks for Python, vibe coding, and coding agents.

GLM-5.2 "beats" Claude Code. Same week, a different benchmark says Opus wins by 14 points

· 7 min

Three benchmark stories about GLM-5.2 broke the same week, and they don't agree with each other. The disagreement is the actual story: single leaderboard numbers move with the harness, not just the model.

Karp Is Right That Tokens Get Wasted. He's Fixing the Wrong Thing

· 6 min

Palantir's CEO says enterprises are paying for tokens that create no value, and he has a point. But the fix isn't hiding the meter behind credits, it's watching what a token spend actually produces.

Does thinking effort actually matter? Same model, low vs max

· 5 min

Opus 4.8 and Sonnet ship at up to six effort levels each. The arena lets you blind-compare a model against itself — and the output data says effort is not the dial you think it is.