Tech giants are attempting to put aside their rivalries and unite on AI safety, but they're clashing over how an actual slowdown would work

Anthropic CEO Dario Amodei, at right, has proposed slowing the rate of AI development while introducing new regulatory protocols. OpenAI's Sam Altman and SpaceX's Elon Musk agree.

Artificial-intelligence leaders rarely agree on anything.

As Anthropic, OpenAI, Alphabet's (GOOGL) (GOOG) Google and other AI labs compete in an expensive race to develop superintelligence, they've diverged on everything from open-source access to model architecture. It's led to fierce disputes that sometimes veer into name-calling.

So when Elon Musk, Sam Altman and Demis Hassabis - the heads of xAI parent SpaceX (SPCX), OpenAI and Google DeepMind - expressed support for Anthropic CEO Dario Amodei's calls to slow the development of AI, it seemed like the tech executives had put aside their differences to tackle serious technological risks. They agreed that the "frontier," an industry term for the most state-of-the-art AI capabilities, is progressing at a rate faster than humans can safely monitor and control.

Concerns about powerful AI systems causing societal harm through cyberattacks and economic disruption have flared after AI researcher Jacob Coxon announced his resignation from Anthropic earlier this month, saying in an X post that OpenAI and Anthropic are acting irresponsibly and "gambling with our lives." Anthropic's head of alignment science, Evan Hubinger, echoed that sentiment, posting on X that he sees a "greater than 10% chance" of AI eliminating humanity within the decade.

Amodei's essay "We Must Pace the Frontier" addressed those concerns by urging the AI industry to build new capabilities at a slower rate. More resources must be dedicated to model alignment, or ensuring that an AI model's behaviors match human values and intentions, he argued. Amodei also called for increased regulation of the frontier AI labs, with a focus on "transparency and third-party auditing."

But implementing guardrails for safe AI development is turning out to be far more complicated than simply voicing support on social media. New heated debates are flaring up across the industry as CEOs, researchers and policymakers grapple with what it means to pace the frontier.

First, not everyone agrees that AI safety is a collective-action problem.

The frontier labs have embraced working together to set standards. OpenAI global policy chief Chris Lehane said on Tuesday that the company had been collaborating on AI safety issues with Anthropic and Google DeepMind for several weeks.

OpenAI, Anthropic and Google DeepMind did not immediately respond to MarketWatch requests for comment.

On the other hand, Meta (META) CEO Mark Zuckerberg took a different stance, saying in a Tuesday X post that "every lab has the responsibility and incentive to move at the pace required to train its models safely, and the ability to take its own actions to ensure that happens." Zuckerberg highlighted that Meta delayed the release of its Muse AI agent for several months due to safety reasons. "We didn't call for everyone else to do this before we would," he added.

Meta did not immediately respond to a MarketWatch request for comment.

Ritwik Gupta, a computer science and AI professor at the University of Maryland, believes that pacing the frontier is contingent upon on widespread cooperation among labs. "That may not be possible in today's competitive free-market regime that we operate in," Gupta said. If only OpenAI and Anthropic agree to slow AI development, other players could seize the opportunity to catch up or set a new frontier, he pointed out.

Even if AI labs collaborate to slow development, Gupta sees potential for loopholes to emerge. Some researchers have proposed limitations on the computing power used to train AI. Today's frontier progress is increasingly driven by test-time workloads and software development surrounding AI rather than just raw model training, according to Gupta.

"I'm not fully convinced that pausing training alone is actually going to meaningfully change the rate of progress of the frontier," Gupta said.

The lack of shared definitions across a fast-moving industry is also stalling AI safety efforts.

A.J. Bhadelia, who oversees Americas public policy at Cohere, told MarketWatch that there remains "fundamental scientific disagreement in that space." In his opinion, some of the recent discussions about AI risk have approached the territory of "science fiction."

"That 10% number is not based on research...there's no scientific validity to that," Bhadelia said of the Anthropic researcher's doomsday prediction.

The AI industry needs to have an evidence-based discussion about the types and severity of AI risks before rushing to impose policies, he argued, as a consumer-facing AI tool could require a very different set of guardrails than an enterprise infrastructure provider.

While pacing the frontier sounds compelling in theory, some believe these ambiguous definitions and loopholes make a slowdown functionally impossible.

"In practice, there is no coherent, rational slowdown plan to be considered," Ben Goertzel, the founder of SingularityNET, said. A prominent AI researcher, Goertzel and DeepMind co-founder Shane Legg popularized the term "artificial general intelligence" in the early 2000s. As AI applications proliferate, Goertzel raised the question: "Where do you draw the line between AI development and software development?"

"There's a premise underlying all of this that you can control AI by controlling the biggest LLMs," Goertzel told MarketWatch.

Goertzel pointed to startups such as Advanced Machine Intelligence Labs, founded by Meta's former chief AI scientist Yann LeCun, and Discovery Loop, launched by ex-Google DeepMind chief scientist Jeff Dean. These companies are exploring experimental architectures that could challenge the industry's current training practices.

"I think the assumption that only a few companies can be at the frontier may not look so true a year from now," Goertzel said. "The smaller the cutting-edge things get, the harder it is to police."

One proposal that has picked up widespread traction is the use of independent evaluators within labs. In his essay, Amodei proposed placing "embedded evaluators" within frontier labs and giving these auditors employee-level access to monitor AI practices.

But again, the initial consensus gives way to a barrage of new questions. Who should do the auditing? Who will govern the auditors? And what types of evaluations should be administered?

Zuckerberg took a laissez-faire approach, writing in his Tuesday post that Meta Superintelligence Labs already engages independent evaluators and "other labs can just do this too."

A non-profit organization called Model Evaluation and Threat Research has grown in prominence recently after being mentioned in Amodei's essay as a potential third-party evaluator. But University of Maryland professor Gupta believes federally funded research and development centers (FFRDCs) would serve as more neutral third-party evaluators.

FFRDCs are government-funded, privately operated research centers that include private, non-profit think tanks and major research universities. The incentive structure for a congressionally funded FFRDC is less vague than philanthropy-backed AI safety groups, Gupta said.

On Friday, a group of independent AI evaluation organizations called the AI Evaluator Forum published a public letter outlining a general framework for embedded evaluators. With over 100 signatories, the letter called for diversity in evaluations and operational safeguards to help evaluators maintain independence.

"The independence question is very important," Nat Purser, director of U.S. policy at the AI Verification and Evaluation Research Institute, told MarketWatch. The institute is a signatory of the AI Evaluator Forum letter.

"People are right to think about conflicts of interest and not being involved in commercial entanglements with the labs," Purser said. Over the past few weeks she's received questions about whether auditors can speak freely without retaliation from the labs and if evaluators can gain sufficient access to the necessary materials needed to conduct an audit.

"A lot of this is going to have to be effectuated through policy if we want to see this standardized across the labs," Purser added. One proposal to promote evaluator independence would be for regulators to assign auditors from a licensed pool instead of having the labs hire their own auditors directly.

However, Purser cautioned that setting up an effective evaluator system won't be sufficient to catch all AI threats, as unforeseen risks will inevitably emerge.

"I'm not interested in pacing for pacing's sake," Purser said. "I'm more interested in implementing more processes that allow for meaningful oversight, and if that results in things slowing down, that'd be great."

-Christine Ji