A rogue state or terrorist organization has little reason to observe an AI slowdown

AI-company leaders are right to take risks seriously, and government has a legitimate role in safety. Caution should be aimed at speeding safe AI, not slowing the most advanced AI.

Some of the builders of the world's most powerful artificial-intelligence systems are warning that their creations could eventually threaten humanity.

These aren't science fiction writers. Their worries - AI systems deceiving their creators, bootstrapping their own code, or launching autonomous cyberattacks - show up in scientific literature.

Most of those worrisome capabilities do not exist today in anything approaching their most frightening forms. Some may never exist. But enough evidence is emerging that the warnings deserve attention.

Some leaders want to slow down until we better understand the risks. That's understandable. But slowing responsible developers does not necessarily slow everyone.

A rogue state, terrorist organization, or criminal enterprise will have little reason to observe a slowdown. They might see it as an opportunity. And if an AI system itself behaves in ways its developers did not expect, a rulebook won't make it behave.

There is a precedent for this problem. When malware spreads across the internet, the response isn't to slow technology, but to innovate faster, creating solutions that are technologically superior to the malicious software.

AI requires a similar strategy.

Developers should certainly work to make AI safe. But our best protection against dangerous AI may ultimately be other, more capable AI.

Consider cybersecurity. Researchers have already shown that AI agents can autonomously exploit software vulnerabilities; in one experiment, a GPT-4 agent successfully exploited 73% of vulnerabilities without being given prior knowledge. Yet defensive AI can spot anomalies, trick and defeat AI hackers, and report threats at speeds humans cannot match. When the contest is between offensive and defensive AI, the defense needs to be ready.

The same principle applies to some of the more disturbing scenarios. Suppose future AI agents learn how to specifically deceive you. Or, when caught, how to pretend to be shut down. You and I could be badly outmatched. But other AI systems could monitor those interactions, identify problems and intervene.

This changes the economics of AI safety. The issue isn't how much AI the world develops, but who has the strongest systems. If malicious actors are on the cutting edge of powerful AI, they will match or exceed the defense systems. We need the companies and countries most committed to using AI responsibly to be the AI leaders.

That presents a difficult trade-off. Developing more capable AI will create risks. Safety testing, security and careful deployment matter greatly. Negligent parties should be held responsible when carelessness causes harm, and governments have legitimate roles in protecting critical infrastructure and national security.

But deliberately slowing responsible developers creates risks. The knowledge behind artificial intelligence is widely dispersed. Powerful computing is spreading. Governments may restrict access to the most advanced AI models and chips, but that increases opportunities for rivals.

Rules followed by American companies are also unlikely to bind foreign rivals, a hostile government, or a clandestine organization. If slowdowns leave responsible developers with weaker technology, the result could be less safety, not more.

Washington should make defensive AI a national research priority: systems designed to identify dangerous capabilities in other models, monitor autonomous agents, detect deception, and defend networks. Developers can establish and publicize practices that give customers, investors, and business partners ways to distinguish responsible providers. Government can use its purchasing and research policies to reward strong security and safety practices without prescribing how a rapidly changing technology must develop.

Now read: As AI companies call for a slowdown, here's how Washington could pump the brakes

International cooperation can help. But verifying AI agreements may prove harder than verifying arms-control agreements. Nuclear programs leave physical signatures - materials, enrichment facilities and test sites. Important AI capabilities reside in software and computing systems whose activities are often harder for outsiders to observe.

The leaders of AI companies are right to take the risks they see seriously, and government has a legitimate role when developers can impose dangers on others. But caution should be aimed at speeding safe AI, not slowing the most advanced AI.

That means investing in systems that test, monitor, and, when necessary, counter other AI systems. It means holding developers accountable for preventable harms while preserving their ability to innovate. And it means recognizing that slowing America's most responsible developers will accomplish little if dangerous actors continue moving ahead.

The irony is, we need AI to protect us from AI. We should make sure the systems that defend us are the best in the world.

Mark Jamison is a nonresident senior fellow at the American Enterprise Institute and the director and Gunter Professor of the Public Utility Research Center at the University of Florida's Warrington College of Business.