AI Existential Risk or AI Fear Mongering? Inside Dario Amodei’s AI Safety Warning

Anthropic CEO Dario Amodei just published a new essay called We Must Pace the Frontier, and it’s landed right in the middle of a live argument about AI existential risk, Dario Amodei’s AI safety credibility, and whether the industry’s warnings amount to genuine caution or a very convenient kind of AI fear mongering. In it, he writes plainly: “We must slow the pace at which we improve the capabilities of AI models.” His argument is that AI is now being used to train the next generation of AI, and that left unchecked, this could eventually outpace both human comprehension and human safety regulation. Progress, he warns, will still feel fast even as the industry tries to slow it down.

The question worth asking isn’t whether Amodei believes what he’s writing. It’s whether decades of science-fiction imagery — machines turning on their creators, a countdown to a Skynet-style extinction event — have made it easier for the industry to package a genuine safety concern as a story we already know how to be afraid of.

A Trend That Set the Mood

Just before this debate reignited, social media had already split into two camps over something much smaller: a viral trend where people asked AI tools to turn their uploaded selfies into 1980s studio portraits — fluffy hair, neon lighting, colourful jackets. Millions joined in. It also triggered a very public backlash, with critics warning that the trend quietly hands over biometric data with no real way to verify it’s ever deleted, and burns real electricity and water at a scale no individual user ever agreed to.

That’s the backdrop this essay landed on: a public already primed to distrust AI companies, right as one of them published a warning that AI itself might be the real threat.

What Dario Amodei’s Essay Actually Argues

To be fair to Amodei, his essay is more measured than the headlines suggest. He frames the risk not as current models being dangerous, but as the rate of improvement changing — AI systems are increasingly being used to help build the next generation of AI, a dynamic researchers call recursive self-improvement. He proposes embedding permanent, employee-level third-party evaluators inside frontier labs to verify safety claims, rather than simply asking the public to take a company’s word for it. Anthropic has committed to this step unilaterally.

Whether that reads as sober risk management or as a pre-emptive PR move probably depends on how much trust you already had in the company writing it.

The Viral Warning: Jacob Coxon’s Resignation

Days before Amodei’s essay, an AI researcher who had worked at both Anthropic and OpenAI resigned and went public with his concerns. Jacob Coxon’s resignation thread reportedly drew a massive audience — estimates vary by outlet, from roughly 100 million views to over 130 million depending on the count and the platform.

Coxon accused both labs of “racing straight to self-improving superintelligence and gambling with our lives” without adequate safety precautions, warning that near-future systems could be capable of hacking, and of acquiring real-world power and resources on their own. Notably, Anthropic’s own alignment science lead publicly backed him up, saying he personally put the odds of AI causing human extinction within the next decade above 10%.

Coxon’s timing raised its own questions. He warned about recursive self-improvement before Amodei’s essay was published, and once his video went viral, he was asked directly about his financial stake in the companies he was criticizing. He confirmed he had given up his Anthropic shares, but that he still holds equity in OpenAI.

“It All Sounds the Same”: Timnit Gebru’s Long View

AI researcher Timnit Gebru offers a useful reality check. She describes a game she sometimes plays with people: reading them old quotes about artificial intelligence and asking them to guess which decade the quote is from. Her point, in short: “it all sounds the same” — many people can’t tell whether an AI warning is from this year or from thirty years ago, because the language of AI doom has barely changed even as the technology has.

Gebru has also argued that when something built by people fails or causes harm, the right question isn’t whether the technology itself was ethical or sentient — it’s who built it, and why it was built to fail in the first place. Applied here, her point is that treating AI as an autonomous, almost mystical threat conveniently shifts scrutiny away from the specific companies and people making the decisions.

Follow the Money: What OpenAI’s Leaked Financials Reveal

This debate isn’t happening in a vacuum — it’s happening right as OpenAI’s internal financials, shared privately with investors, became public. According to reporting based on those documents:

Neither Anthropic nor OpenAI has turned an annual profit yet.

How AI Startups Compare to Amazon, Google, and Spotify

People often reach for the “Amazon took years to turn a profit too” comparison to make these numbers feel less alarming. But the scale doesn’t really hold up:

  • Amazon lost roughly $3 billion total across its early years of unprofitability before turning a corner
  • Spotify lost around $3.2 billion over 18 years before becoming profitable
  • Microsoft was profitable within its first year
  • Google had only small early losses and was solidly profitable within three years
  • OpenAI alone lost close to $39 billion in a single fiscal year

AI labs don’t follow the same business model as those companies. Tech startups typically use venture capital to fund rapid scaling long before they’ve figured out how to make money — and once investors grow tired of funding losses at this scale, the pressure to pivot from building user value to extracting profit tends to arrive fast.

Who Benefits From the Fear?

As one social media user put it, a trillion dollars in market value is riding on convincing the public that something closer to Skynet already exists — when what’s actually been built is closer to a very capable autocomplete and search tool. The timing of this fear cycle, arriving just after OpenAI’s Q2 losses leaked, is hard to ignore.

None of this means the safety concerns raised by Amodei, Coxon, or Anthropic’s own researchers are fabricated. It means the incentive to be believed — by regulators, by investors, by the public — is not neutral. Framing AI as an almost godlike force on the verge of “deciding” to take over the world distracts from a much more mundane and answerable question: who built it, who is deploying it in contexts like warfare, and who benefits from the public being too afraid to ask.

We Can Still Unplug

In all of this fear mongering, it’s worth remembering something simple: these are machines running on infrastructure that companies own and control. We can unplug them any time we choose to.

Sources: Dario Amodei, “We Must Pace the Frontier”, The Jerusalem Post, Asia News Network, ChatGPT Is Eating The World, Newsweek, TipRanks, Fortune