Two Ways to Lose Trust: Models You Can't Test, Leaderboards That Grade Their Investors

Two stories broke this week that look unrelated until you put them next to each other. First: the U.S. government is deciding, model by model and customer by customer, who gets to touch the most capable AI systems before the rest of us do. Second: the leaderboard most people cite as the neutral scoreboard for those same systems took a $150M round at a $1.7B valuation with OpenAI, Google, and Anthropic as strategic investors. One story is about access. The other is about adjudication. Together they explain why so much of what you read about frontier models right now is unverifiable, and why blind, live comparisons matter more in July 2026 than they did a year ago.

The models you can't test yet

Anthropic's Mythos 5 and OpenAI's GPT-5.6 have both spent the last several weeks behind a government checkpoint instead of a normal product launch. Mythos was pulled back after a U.S. official said the model found vulnerabilities in classified government systems within hours during a testing exercise. Commerce Secretary Howard Lutnick has since cleared it for redeployment, but only to a list of more than 100 approved U.S. companies and federal agencies, not the general public.

GPT-5.6 followed a similar script. The Office of the National Cyber Director and the Office of Science and Technology Policy asked OpenAI to stagger release of GPT-5.6 Sol, Terra, and Luna, with the Commerce Department approving access customer by customer during a preview window. That restriction is reportedly lifting this Thursday, July 9, but for the last two weeks the model generating the most headlines was one almost nobody outside a government-approved list could actually run.

• Mythos 5: export-blocked, then cleared for roughly 100+ named orgs and agencies, not open release
• GPT-5.6 Sol/Terra/Luna: customer-by-customer government approval during preview, wide rollout targeted for July 9
• Framework: Trump's executive order on "Promoting Advanced AI Innovation and Security" asks labs to voluntarily share frontier models with the government for up to a month of cybersecurity review before public launch
• Result: the models people are arguing about on social media this week are, for most of that argument, running only on the labs' own claims

None of this makes the review process wrong. A model that reportedly finds exploitable holes in classified systems in hours is a reasonable thing to slow down. But it does mean that every benchmark screenshot, every "we tested it internally" thread about Mythos or gated GPT-5.6 variants, is coming from someone on an approved list, not from an outside party who can be checked. We run a live arena specifically so readers aren't stuck taking a lab's word for it, and the honest caveat is that gated models sit outside that arena too, for now, exactly like they sit outside everyone else's.

The leaderboard that grades its own investors

The second story is about the tool people reach for when a model *is* available: Arena, the crowdsourced leaderboard many treat as the closest thing to a neutral scoreboard in the industry. Reporting this month put a number on something people had suspected for a while, that Arena's most recent funding round included OpenAI, Google, and Anthropic as strategic backers, alongside Andreessen Horowitz, Kleiner Perkins, and Lightspeed, at a $1.7B valuation.

Independence isn't just about preventing active corruption, it's about avoiding even the appearance of conflicts that could undermine trust. When the referee takes money from the teams, every close call looks suspicious and every methodology change gets scrutinized for which companies benefit. — analysis in TechBuzz.ai's coverage of Arena's funding

Arena's counter-argument is what one write-up called "structural neutrality": since OpenAI, Google, and Anthropic all invested, no single lab can lean on the platform without the others noticing. That argument has a real weakness. Critics have already drawn the obvious comparison to pre-2008 credit rating agencies, which were funded by the banks they rated and made the same balanced-incentive argument right up until it turned out that everyone being incentivized to inflate ratings together is not the same thing as nobody being incentivized to inflate them.

What this actually changes about how you should read benchmark claims

For the next few weeks, treat any strong claim about Mythos 5 or the early GPT-5.6 variants with the same skepticism you'd give a pre-release press kit, because functionally that's what it is: a small, non-random sample of testers with a relationship to the lab. Once wider rollout lands, that's the point to actually compare outputs, not before. And for models that are broadly available today, check whether the leaderboard doing the ranking has a reason to want a particular answer. We keep our own comparisons blind and run the actual outputs live rather than relying on labs' self-reported numbers, precisely because the two stories above show what happens when either access or incentives get compromised.

This isn't a uniquely American problem either. Chinese labs are undercutting GPT-5.5 pricing by a reported 97% this same week, which means the pricing and access questions are becoming global at the same time the trust questions are. Readers following the European and Polish-language side of this beat can track the same story at nowosci.ai.

The uncomfortable truth is that the models worth arguing about right now are split into two buckets: the ones you can test but shouldn't fully trust the scoreboard on, and the ones you can't test at all yet. Neither bucket is going away soon. Export-style review is now a standing part of how frontier models ship, and leaderboard consolidation makes financial entanglement more likely, not less, as the number of well-funded arenas shrinks. The only real fix on the reader's side is to keep asking, for any claim about a model's quality, who ran the test and whether they knew which model they were grading.