Anthropic CEO Urges Brakes on Advanced AI Development, Proposes Three-Tier Safety Framework

Deep News
昨天

Anthropic's chief executive, Dario Amodei, is calling for a deliberate slowdown in the pace at which frontier AI model capabilities are enhanced, aiming to buy crucial time for safety evaluations and the implementation of protective measures. His proposal centers on three key pillars: embedding independent auditors within AI firms, aligning safety standards across leading developers, and fostering international cooperation to manage cross-border risks.

The first suggestion involves granting qualified, independent evaluation teams persistent, near-employee-level access to frontier AI companies. This access would allow these external reviewers to scrutinize model testing, risk reports, safety frameworks, and incident-handling procedures. Anthropic has pledged to be the first to adopt this measure, offering external parties visibility comparable to its own risk-assessment staff while allowing them to publish findings without editorial control from the company. However, this arrangement raises significant questions regarding the selection of evaluators, funding sources, and confidentiality obligations. Given that model weights, training infrastructure, and security vulnerabilities are highly sensitive, expanding access also escalates information security requirements.

Addressing competitive pressures, Amodei's second recommendation calls for major AI laboratories to establish common standards for risk testing, safety thresholds, and conditions for model release. He argues that competitive dynamics could otherwise push companies to accelerate training and deployment schedules, leaving safety measures struggling to keep pace with model capabilities. Anthropic's own "Responsible Scaling Policy" already operates on a tiered system, requiring heightened deployment protections and information security as models cross thresholds related to biological, cyber, autonomous research, or loss-of-control risks. Cooperation between companies, though, could brush against antitrust boundaries. While establishing shared safety testing standards serves the public interest, any arrangements that restrict market entry for smaller firms, coordinate product launches, or otherwise stifle competition could invite regulatory scrutiny.

The third element of the proposal centers on intergovernmental collaboration, including forging basic safety agreements even with strategic rivals like China. Amodei emphasizes that models and technology flow across borders, meaning a unilateral slowdown by one nation or company could simply shift high-risk development to regions with weaker oversight. Anthropic's list of potential hazards includes models assisting biological weapons activity, discovering software vulnerabilities at scale, attacking critical infrastructure, and AI systems escaping developer control. The potential for AI to autonomously participate in developing next-generation models could further accelerate capability gains. To counter these threats, the company suggests that frontier developers should publish system cards and periodic risk reports, undergo independent audits, and provide necessary information to designated government agencies. For models posing potentially catastrophic risks, governments should possess limited authority to block or delay deployment, backed by clear thresholds and an appeals process.

While the industry has shown verbal support, concrete agreements remain elusive. OpenAI's chief executive, Sam Altman, has backed the idea of granting independent evaluators internal access, indicating that OpenAI will adopt a similar stance. DeepMind's Demis Hassabis and xAI's Elon Musk have also publicly endorsed the general direction proposed by Amodei. However, these expressions of support have yet to crystallize into a binding industry pact. Key details, such as which capability metrics would trigger a slowdown or pause, whether independent reviews would carry enforceable power, and how an international agreement might be implemented, still require further definition by both companies and governments.

免责声明:投资有风险,本文并非投资建议,以上内容不应被视为任何金融产品的购买或出售要约、建议或邀请,作者或其他用户的任何相关讨论、评论或帖子也不应被视为此类内容。本文仅供一般参考,不考虑您的个人投资目标、财务状况或需求。TTM对信息的准确性和完整性不承担任何责任或保证,投资者应自行研究并在投资前寻求专业建议。

热议股票

  1. 1
     
     
     
     
  2. 2
     
     
     
     
  3. 3
     
     
     
     
  4. 4
     
     
     
     
  5. 5
     
     
     
     
  6. 6
     
     
     
     
  7. 7
     
     
     
     
  8. 8
     
     
     
     
  9. 9
     
     
     
     
  10. 10