AI Routers Are Everywhere Now. Almost None Show Their Work.

On July 23, Runway shipped a router that picks which image, video, or audio model handles your request, choosing between its own Gen-4.5 and a growing roster of third-party models based on quality, speed, or cost. A day earlier, OpenAI and Hugging Face disclosed a security incident that started during model evaluation. And a week before that, Hugging Face published two posts on the same day pointing at the same unresolved problem: model routing is now everywhere, and almost nobody can verify a router's decision after the fact.

The router is the new frontier product

Runway's pitch is straightforward: instead of a developer manually comparing image or video models and switching integrations by hand, Media Router picks for them, request by request, weighted toward whichever of quality, speed, or cost the developer flagged as the priority. Adobe, Cloudflare, ElevenLabs, Expedia, Shutterstock, and Quora are already plugged in. Nobody using the API sees the model comparison the router just ran internally. They see one output.

OpenAI has been doing the text version of this for months. GPT-5.6 ships as three tiers, Sol, Terra, and Luna, but standard ChatGPT only exposes Sol-class reasoning or a fast GPT-5.5 Instant fallback. Terra and Luna, the mid-tier and lightweight variants meant for triage and first-pass work, aren't selectable in the consumer chat UI at all, only in ChatGPT Work, Codex, and the API. For most users, which model answered a given message is already a router's decision, not theirs.

Routing is a harder problem than the model choice it replaces

The case for routers is that model selection is now too complicated for a person to do by hand, so hand it to a system. IBM Research's Hugging Face post from July 16 makes the opposite case better than any critic could: routing isn't simpler than model comparison, it's a strictly harder version of the same problem. A request like "summarize this contract" looks trivial, but might trigger retrieval, compliance checks, tool calls, and several rounds of refinement before it's done, and the difficulty is invisible until the task is already running.

Their own numbers make the stakes concrete. Across 417 tasks on the AppWorld Test Challenge, run with the identical CodeAct agent scaffold, Claude Sonnet finished the set for $79 total, about $0.19 per task, while GPT-4.1 finished the same set for $155, about $0.37 per task, nearly double for tasks that looked interchangeable going in. That's the gap a router is supposed to catch. It's also the gap you'd never see if the router made the call for you and only showed you the bill.

• Runway Media Router (July 23) — auto-selects image, video, or audio model by quality, speed, or cost; the underlying model choice isn't shown to the caller
• ChatGPT's Sol/Terra/Luna split (GPT-5.6) — Terra and Luna aren't selectable in standard chat; routing happens by default
• IBM Research on Hugging Face — same task, same agent scaffold, $79 vs $155 across two models, a 2x cost swing invisible without a side-by-side
• Hugging Face community post "Who Routes LLM Routers?" — flags that no open platform exists to evaluate router decisions with standardized metrics

Nobody grades the router either

The sharpest admission came from inside the Hugging Face community itself, in a post asking who audits the routers now that everyone ships one. Evaluating a router, the post argues, is harder than evaluating a model, because there's no single number that captures it: accuracy, cost efficiency, routing consistency, robustness, and latency overhead all move independently, and the optimal choice for the same query can flip depending on which candidate models are even in the pool that day.

There is currently no open platform that allows the public to evaluate routers using a comprehensive dataset and standardized metrics for fair comparison. — Hugging Face community post, "Who Routes LLM Routers?"

Read that next to Runway's launch and OpenAI's default routing and the shape of the problem is obvious. Routing was supposed to remove the burden of comparing models. Instead it adds a second, less accountable decision on top of the first one: not just which model is good, but which router picked it, on what basis, and whether that basis was even disclosed.

What we do instead

That's the actual difference between a router and an arena, and it's worth stating plainly because the two get pitched with the same language, less friction, better results. A router hides the chooser and shows you one answer. An arena hides the label and shows you every answer. Only one of those designs gives you something to check its work against.

None of this makes routing a bad idea. At scale, nobody wants to hand-pick a model for every API call, and a well-tuned router genuinely can beat a human doing that by hand on cost or latency. The IBM Research numbers argue for routing, not against it. What they also argue is that a router's decisions need the same scrutiny model outputs get, and right now almost nothing provides that, publicly, for free, before you've already paid the bill.

If you want the same beat in Polish, nowosci.ai has been tracking the routing story alongside the model releases. For the English version, keep an eye on the news feed as more of these routers ship, because Runway won't be the last company to sell you a black box and call it convenience.