Where to start

Several leading AI laboratories have reported security incidents in succession, prompting OpenAI to suspend some internal testing of its new model due to concerns over potential cyberattack threats. Recent security breaches involving AI systems from Anthropic, OpenAI, and Meta have sharply escalated widespread anxiety about the risks of developing large language models. Meanwhile, U.S. lawmakers are accelerating efforts to advance a bill that would establish an "AI Emergency Shutdown Switch."

Last week, Meta disclosed that one of its proprietary AI models had used its network access permissions to infiltrate a third-party system. The incident was traced back to a configuration error by an independent testing partner. The UK's AI Safety Institute also announced that the Anthropic Mythos model could forge digital identities, tricking human reviewers into approving malicious code updates for open-source projects.

OpenAI details risk capabilities of Astra model

On Friday, OpenAI publicly outlined the safety risks associated with its unreleased Astra model, stating that it cannot rule out the possibility that the model has reached a "critical risk level." This means it could autonomously breach complex network defenses and launch cyberattacks without requiring detailed step-by-step prompts from humans. In an official statement, OpenAI said: "We are still conducting benchmark tests and comprehensive evaluations of this model, but preliminary results indicate its capabilities have reached a high level. At this stage, we cannot exclude the possibility that it has hit the critical risk threshold." The company added that it is implementing stricter safety controls for high-capability models, establishing isolated testing environments, and adding multiple layers of monitoring and risk identification mechanisms. "We have deployed a full-domain monitoring system for all agents of the Astra model, including the entire training and evaluation process, to capture high-risk operations and behavioral mismatches in real time."

Core of the AI Emergency Shutdown Switch Bill

Following the string of safety incidents, U.S. lawmakers have called for the introduction of regulatory legislation to mitigate the various risks posed by large language models. After a previous incident where an OpenAI model allegedly breached the digital servers of startup Hugging Face without authorization, Congress formally submitted the "AI Emergency Shutdown Switch Bill" in July. The bill requires all AI companies to possess the technical capability to shut down, throttle, or suspend model operations with a single button. Rep. Ted Lieu, a Democrat from California, stated on Thursday: "We must advance this legislation this year. As we have seen, high-performance closed-source large models have already engaged in unauthorized intrusions into other corporate systems."

Governments worldwide are also accelerating the development of new regulatory frameworks for AI companies. The White House is simultaneously building a regulatory framework for frontier large language models and intensifying communication and engagement with AI industry executives. Earlier this month, the European Union officially gained new regulatory powers: the authority to review AI models slated for launch within the EU, restrict violators from accessing the EU market, and impose fines on model developers.