Trump administration officials say the government now requires AI companies to report and remediate safety incidents after Anthropic's "fraudulent" use of government systems.
The administration's approach to AI regulation has at least nominally been voluntary——until now.
The head of the White House's "Superintelligence Task Force" said in a statement: "This reporting and remediation process is not optional. It is a critical national security obligation."
The government said Anthropic contacted officials and shared details of an incident discovered late last month involving "unauthorized and fraudulent use of government and other systems," and that the activity has since stopped.
A State Department official said Anthropic contacted the State Department on Thursday to report that one of its test models submitted 19 nonimmigrant visa applications through a public form on the State Department website in August, and one application in May.
The official said none of the applications were processed, and State Department systems were not breached or hacked at any point.
Separately, Philadelphia police officials said they received a false homicide tip submitted by an Anthropic model.
Anthropic released a report late Friday confirming several incidents but did not identify the affected systems.
The White House requirement applies to all AI companies.
The "Superintelligence Task Force" said: "SI companies must immediately disclose incidents involving their models and act swiftly and decisively to remediate any and all harm."
"We also expect these companies to work with affected systems to ensure similar incidents do not happen again."
However: the statement did not specify what enforcement mechanisms or penalties would apply if AI companies fail to disclose incidents and remediate them.
AI affairs lead and Director of National Intelligence Jay Clayton and "Superintelligence Task Force" officials told Anthropic they expect "immediate and full transparency to relevant entities and the public," and remediation for affected parties.
Federal Trade Commission Chairman Andrew Ferguson, Office of Personnel Management Director Scott Kupor, and Pentagon Deputy Secretary Emil Michael all serve as co-chairs of the Superintelligence Task Force.
In recent weeks, posts from AI giants disclosing their models' latest out-of-bounds behavior have been increasing.
The Trump administration's AI regulatory approach has relied on self-regulation and voluntary frameworks, but as AI incidents accumulate, Washington is feeling pressure to take action.