MatCo

Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead. You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement. I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible. But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier. Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want. Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well. So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it. If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.

Some thoughts on Dario’s post: 1. Dario does not actually address Gavin Baker’s account of what he said – something he could easily deny if it were inaccurate. 2. Dario claims his critics live in a “bubble” where all regulation equals regulatory capture. He calls this an overly simplified view and notes that “Many people outside this bubble think of regulation as something that constrains corporate power and benefits ordinary people.” This argument is a straw man. Of course treating all regulation as capture would be overly simplified – but almost no one holds that view. I have repeatedly argued for strong antitrust enforcement to keep industries competitive, especially Big Tech. If Anthropic continues toward monopoly or duopoly status, I would be among the first to demand those rules apply. 3. Regulatory capture is not vague or in the eye of the beholder. Nobel laureate George Stigler defined it as regulation acquired by an industry and designed and operated primarily for its benefit. Stigler challenged the traditional view that government regulation arises from a benevolent state protecting the public from market failures. Rather, industry groups have concentrated stakes and pour resources into influencing regulators, whereas the public’s stake is diffuse and unorganized. The revolving door between companies and the agencies that regulate them compounds the problem. Anthropic understands these dynamics: it has hired multiple senior Biden AI-policy officials and built a substantial government-affairs operation plus a network of aligned organizations to push its preferred frameworks at state and federal levels. 4. Dario has consistently pushed for a new federal agency to review and approve frontier models prior to release – a proposal framed variously as an “FDA for AI,” an “FAA for AI,” and most recently a “FINRA for AI.” I call it a “DMV for AI” because a review process modeled on the FAA or FDA (which takes years) or FINRA (which issues rules for a staid industry widely seen as protecting incumbents) will create long queues as AI models wait for testing and approval. This process will only become more labyrinthine as rules accumulate to prevent theoretical harms. This would handicap the U.S. relative to China, which will not adopt the same constraints. It would also undermine Anthropic’s own business model, whose pricing power depends on remaining ahead of open models. Whatever Dario states today, it is difficult to believe the company would simply accept outcomes that erase that advantage. 5. Anthropic is on track to become one of the most valuable companies in history, with the resources to navigate any approval process and shape the rules while competitors wait. Dario wants open models under heavier scrutiny – he has called them dangerous in Senate testimony, criticized them for not being centrally monitored or withdrawn, and linked them to IP theft. He says he has never sought a ban, but he could achieve a similar result by insisting that identical rules apply to both open and closed models. The U.S. risks becoming an island of costly closed models while the rest of the world races ahead with broader choice. 6. Dario acknowledges that AI is structurally centralizing but attributes this mainly to chips and scaling laws. Access to compute matters, but the deeper risk is who decides which capabilities are available to whom. His preferred pre-deployment testing and FAA/FINRA-style oversight would place that gatekeeping power in a federal bureaucracy working hand-in-glove with a small number of frontier labs – reinforcing centralization rather than countering it. 7. The second part of Dario’s post assumes we have amnesia about Anthropic’s well-orchestrated campaigns hyping AI fears. His May 2025 claim that AI would wipe out 50 percent of entry-level knowledge jobs within five years still lacks supporting evidence fifteen months later. Similarly Anthropic breathlessly promoted its heavily contrived “blackmail” study on 60 Minutes. Yet Dario blames public negativity on a long-standing loss of trust in institutions rather than his own messaging. 8. These narratives have done more than anything to shape public fear. People are left asking the same question Mark Zuckerberg posed: why race to build a future you describe in such negative terms? Thomas Sowell’s "The Vision of the Anointed" captures the mindset – elite intellectuals convinced that only they are enlightened enough to control the outcome. As Zuckerberg notes, concentrating power in the hands of an enlightened few has rarely produced the promised results; the practitioners turn out to be less enlightened in practice than in self-conception. 9. Gavin Baker summarized the disagreement cleanly on our pod: Dario believes frontier AI is too powerful to distribute; we believe it is too powerful to centralize. Dario appears to believe, sincerely, that safety and progress are best served by centralizing authority in a marriage of corporate and state power. The weight of human history gives us reason to fear that outcome.