Karp Is Right That Tokens Get Wasted. He's Fixing the Wrong Thing

Alex Karp went on CNBC this week and said the quiet part out loud: "something has gone completely wrong" with how OpenAI and Anthropic charge for their models, and enterprises are, in his words, paying for tokens that create no value. He's not wrong about the symptom. He's wrong about the cure.

The complaint, in Karp's own words

Karp's framing was blunt: enterprises are frustrated, costs are climbing, and the token meter runs whether or not the output is worth anything. Palantir's answer is to sell customers control over their own compute and models instead, an air-gapped stack built with Nvidia rather than a subscription to someone else's frontier API.

I'm not throwing shade at them, but something has gone completely wrong. The basic view among enterprises in this country is I'm going to chillax and waste my time with tokens. — Alex Karp, CNBC, July 1, 2026

It's a good soundbite because it matches a real feeling: teams watch a token counter climb during a long agent run and have no idea whether the last ten thousand tokens were reasoning, or padding, or the model re-explaining itself. That anxiety is legitimate. Accenture reportedly told staff outright to stop using AI for tasks that didn't need it, because token spend was escalating faster than anyone could justify.

The price fell 280-fold. The bills went up anyway

Here's the part Karp's framing skips past: per-token prices did not go up. Stanford's AI Index has tracked something close to a 280-fold drop in the cost of GPT-3.5-class inference between late 2022 and late 2024, from roughly $20 per million tokens down to a few cents. Frontier-adjacent pricing has kept falling since. If tokens were the whole story, enterprise AI bills should be collapsing too.

They aren't. Agentic workflows call a model dozens or hundreds of times per task, multi-step tool loops rerun the same context, and a coding agent set to a higher thinking-effort tier can burn far more tokens for a task that a lower tier finishes in a fraction of the length. We wrote about this directly in Does thinking effort actually matter?: more output tokens correlate with cost, not with quality. Karp's enterprises are living that finding without the vocabulary for it. The token is cheap. The number of tokens spent per unit of useful output is the variable nobody is watching.

Hiding the meter is not the same as fixing it

• Per-million-token pricing for GPT-3.5-class inference: ~$20 (late 2022) to a few cents (late 2024), a widely cited ~280x drop, per Stanford's AI Index
• Enterprise AI spend has kept climbing through the same window as agentic, multi-call workflows multiply per-task token counts
• Workday's Flex Credits and similar consumption-credit models convert opaque token counts into flat per-action prices, which improves predictability but not accountability for output quality
• Accenture reportedly instructed staff to stop reaching for AI on tasks that didn't warrant the token spend, a symptom of cost without a quality signal attached

Credit bundles and seat-plus-consumption hybrids are a reasonable business response to sticker shock. But converting "12,000 tokens" into "6 credits" just moves the abstraction one layer up. It still tells you nothing about whether the output was correct, whether a cheaper model or a lower effort tier would have done the same job, or whether the run that cost twice as much actually produced a twice-as-good answer. Karp wants control over the stack; most enterprises just want to know what they're buying.

The only real answer is watching the output, not the invoice

This is the whole reason a blind, live-output arena is worth running instead of trusting a spec sheet or a press release. You cannot tell whether tokens "create value" by staring at a bill. You can tell by putting the same prompt in front of several models at several effort levels, running the outputs live rather than showing screenshots, and letting people vote without knowing which model or which price tier produced which answer. That's the test Karp's complaint is actually asking for, just aimed at the API bill instead of the arena leaderboard.

It's also why the open-weight scramble happening in parallel matters more than the pricing op-eds. GLM-5.2 is being benchmarked hard against DeepSeek V4 and Kimi K2.6 on SWE-style coding tasks this week, and it's cheap enough that the token-cost argument mostly evaporates if the outputs hold up. Whether they actually hold up against Sonnet 5 or Opus on a real coding prompt isn't a question a press release can answer. It's a question you check by running the prompt and reading what comes back, which is exactly what the coding arena is for.

Karp's real target should be the missing feedback loop, not the token as a unit. A token is just a unit of metering. The fix for "we don't know what we're buying" was never going to be a different unit, whether that's a Flex Credit or Palantir's own custom-model contracts. It's watching, side by side, whether the expensive run actually beat the cheap one. Readers who follow this same pricing fight in Polish can find it covered at nowosci.ai, which tracks the same beat for a European audience.

None of this makes Karp's frustration fake. Enterprises really are burning budget on runs that produce nothing usable. But the lesson from watching models compete blind, prompt after prompt, is that the waste was never really about the price per token. It was about nobody checking the output against the alternative before the invoice arrived.