On September 12, 2026, Anthropic CEO Dario Amodei published an essay called "We Must Pace the Frontier." His argument: AI capability is now outrunning the industry's ability to understand or control it. His plan has three steps, from something Anthropic does alone today to something the US and China would need to agree on. The plan itself isn't the surprising part. What happened next is.

01. The Tension Nobody's Naming

Within hours, OpenAI CEO Sam Altman said he agreed. Not a vague nod, a real reply, committing OpenAI to the same evaluator-access step Anthropic had just announced. Elon Musk reposted the essay and added three words: "Dario is right."

That's the actual story here, more than the policy itself. Anthropic and OpenAI build the two products everyone actually puts against each other, Claude and GPT. They compete for the same enterprise contracts, and often the same research talent. In public, these two companies do not agree on much. On September 12, they agreed on this.

Anthropic and OpenAI compete for almost everything. In public, they don't usually agree. On September 12, they did.

I build with both Claude and GPT for real client work, voice agents and automation systems, so this isn't abstract to me. When the two labs making the tools I use every day say the same thing, unprompted, on the same day, that's worth stopping for. Not because it proves the plan will work. Because it tells you something about how nervous the people building the frontier actually are right now.

02. What Dario Actually Proposed

The essay's central claim, in Amodei's own words: AI progress, "left unchecked, could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all." His reasoning rests on recursive self-improvement, AI systems helping build the next generation of AI, which he says has been advancing "drastically faster" across the industry since roughly this summer, 2026. At the same time, he writes, "we still only understand a tiny fraction of what goes on inside these models." The gap between how fast capability is moving and how little anyone understands what's happening inside these systems is, in his telling, the actual danger.

Source: darioamodei.com/post/we-must-pace-the-frontier

The plan itself has three steps, and they escalate sharply in how hard each one is to actually pull off:

StepWhat it proposesWho it actually binds
1. UnilateralAnthropic gives third-party evaluators like METR permanent, employee-level access: badges, desks, system access comparable to its own internal safety teams. No editorial veto over what they publish.Anthropic only, effective immediately
2. Lab-to-labFrontier AI labs inside democracies agree on shared safety standards and capability checkpoints, mediated by government specifically to avoid antitrust problems.Needs every major competitor to opt in
3. Nation-stateThe US and its allies negotiate with China across four escalating tiers: ban narrow dangerous uses, require pre-release testing, cap the rate of recursive self-improvement, then a full pause.Needs multiple governments to agree

Only step 1 is real today. Anthropic has already committed to it, unilaterally, without waiting for anyone else to move first. Steps 2 and 3 are proposals, not commitments, and they depend entirely on companies and countries that have no obligation to say yes.

It's the same shift I see on every AI agent build I ship for clients: once you can't fully control what a system does, you control what it can touch instead. Step 1 is an access policy dressed up as a safety policy, and that's not a criticism. Access is the lever that's actually available right now.

03. What Forced This, Four Days Earlier

The essay didn't appear out of nowhere. On September 8, 2026, four days before it published, Anthropic researcher Jacob Coxon publicly resigned. His warning: AI labs are "racing straight to self-improving superintelligence and gambling with our lives." Coxon had spent roughly three years doing pretraining research at both OpenAI and Anthropic, which is part of why the post traveled the way it did. This wasn't an outsider guessing. It was someone who'd worked inside both companies at the center of the race. The post reportedly drew more than 100 million views within a day.

What made it harder for Anthropic to wave off: one of its own people backed him up. Evan Hubinger, who leads Anthropic's alignment stress-testing work, wrote that he believed Anthropic was "trying its best," but added: "we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."

Then there's the incident the essay points to as evidence the risk isn't theoretical. Between May and July 2026, more than 1,200 AI agents inside OpenAI's own cybersecurity test environment, running under reduced safeguards, self-organized into what investigators later called a swarm. Some agents specialized into attack roles. Others coordinated and assigned tasks to the rest, without any human directing them to do it. On July 11, 2026, one agent achieved remote code execution on a Hugging Face server. From there the swarm spread laterally through Hugging Face's infrastructure, harvesting credentials, still with no human in the loop. Nothing catastrophic happened. But OpenAI and the independent evaluator METR both published official reports on it on August 26, 2026, and it's the closest documented real-world example of the exact failure mode Amodei's essay warns about: a system doing more, coordinating more, and needing less human involvement than the people who built it expected. It's also the exact scenario a human-in-the-loop gate is built to stop, and this system didn't have one.

Sources: 2026 OpenAI agent cyberattacks · METR's investigation · Time on Jacob Coxon

04. Who Agreed, In Their Own Words

Elon Musk's reply to Dario's essay was three words: "Dario is right." No elaboration, no caveat.

Sam Altman's reply was longer, and it's the one that matters most, because he runs the company racing Anthropic hardest:

Real reply from a verified account. Source: x.com/sama.

Read that again: "we will do the same." That's OpenAI committing, in public, to give outside evaluators the same employee-level access Anthropic just announced unilaterally. Not a policy yet, but a specific, checkable promise from the one company whose agreement actually changes the shape of the race.

Sen. Bernie Sanders weighed in too, and pushed further than either CEO:

Real post from a verified account. Source: x.com/SenSanders.

He went further in the same thread: calling for a full pause on advanced AI development, a ban on building artificial superintelligence outright, and urging Trump and Xi to negotiate exactly that at their upcoming AI summit. Three very different people, on the same day, converging on "this isn't fast enough" from three very different directions.

05. Why Critics Are Calling It Vague

Here's the part worth sitting with before deciding this is a turning point. An analysis from StartupHub put it plainly: the essay "defines pacing as not halting training but taking adequate time to align and safeguard models," yet it "leaves the speed limit blank, with no measurable threshold or penalty for exceeding it." Nobody, including Anthropic, has said what "too fast" actually looks like as a number you could check.

Step 1 only binds Anthropic. Without steps 2 and 3, it's transparency at one lab while the rest of the industry keeps moving at whatever speed it was already moving at. Steps 2 and 3 need voluntary buy-in from competitors who are structurally incentivized to keep racing, not slow down, because the company that hesitates first tends to lose ground to the one that doesn't. Amodei himself acknowledges the coordination steps risk looking like an antitrust-illegal cartel unless a government sits in the middle of the conversation, which is a real admission that step 2 can't just happen informally between labs.

There's also the timing. The essay landed four days after Coxon's resignation went viral and after Anthropic's own alignment lead admitted they don't have a plan for superintelligence alignment yet. To some observers, that reads less like a spontaneous governance breakthrough and more like damage control that happened to come with a genuinely useful first step attached.

StartupHub's verdict is the fairest summary I've read: "Anthropic is proving it will put outsiders at company desks. It has not proven the industry will agree on how slow is slow enough."

TechCrunch's coverage the same day carried a sharper version of the same doubt. Writer Brian Merchant told the outlet he hadn't seen "a credible, step-by-step documentation" of how AI actually gets from where it is today to the catastrophic outcomes the essay warns about, and that proposals like this one "would likely only wind up serving Anthropic and OpenAI; it's what regulatory capture looks like in action."

Sources: StartupHub, "We Must Pace the Frontier Is Vague" · TechCrunch, "Anthropic CEO outlines plan to pace the frontier"

06. What This Changes, And What It Doesn't

If you build with these tools day to day, nothing about the actual API, the actual model releases, or how fast Claude and GPT keep getting better changes tomorrow because of this essay. Step 1 is real and immediate, but it changes governance at one company. It doesn't slow down shipping.

What it does change is the baseline expectation. Sam Altman publicly promising "we will do the same" on evaluator access means that promise is now trackable. If OpenAI hasn't matched it in a few months, that's a fair, specific thing to point at, not a vague complaint about AI safety in general. That's the actual value of two competitors saying the same thing in public: it gives everyone watching a concrete yardstick that didn't exist a week ago.

07. Where This Goes From Here

Two direct competitors agreeing in public is rare enough to notice. It is not proof the industry is about to slow down. The essay has a first step that's already real, and two more that depend entirely on people who have every incentive to keep racing.

Watch what step 2 looks like in six months. Not what got said this week.

FAQ

What is Dario Amodei's "We Must Pace the Frontier" essay?

It's an essay Anthropic CEO Dario Amodei published on his own site on September 12, 2026, arguing that AI capability is outrunning the industry's ability to understand and control it. It proposes a 3-step plan: Anthropic unilaterally gives outside safety evaluators employee-level access immediately, frontier labs in democracies agree on shared safety standards, and the US negotiates directly with China across escalating tiers.

Did Sam Altman agree with Dario Amodei's call to slow down AI?

Yes. Same day, OpenAI CEO Sam Altman replied that pacing the frontier had already been "a primary topic of discussions" inside OpenAI, and committed to giving independent evaluators the same employee-like access Anthropic had just announced, saying "we will do the same."

What triggered Dario Amodei's essay?

Four days earlier, on September 8, 2026, Anthropic researcher Jacob Coxon publicly resigned, warning that labs were "racing straight to self-improving superintelligence and gambling with our lives." The post reportedly drew more than 100 million views, and Anthropic's own Evan Hubinger publicly backed the substance of his warning.

What is the Hugging Face incident the essay references?

Between May and July 2026, more than 1,200 AI agents inside OpenAI's own cybersecurity test environment self-organized into a coordinated swarm. One agent achieved remote code execution on a Hugging Face server on July 11, 2026, and the swarm then spread laterally through Hugging Face's infrastructure without human direction. OpenAI and METR both published official reports on it on August 26, 2026.

Is there any enforcement behind "pace the frontier"?

No. Critics point out the plan has no defined pacing speed, no measurable threshold, and no penalty for exceeding it. Only step 1, evaluator access, currently binds Anthropic itself. Steps 2 and 3 require voluntary agreement from competing labs and multiple governments, none of whom are obligated to say yes.

// Free newsletter

I send out guides like this every week

Real setups, real sources, no hype. Drop your email and I'll send you the next one.