Anthropic CEO Urges Brakes on Advanced AI Development, Proposes Three-Tier Safety Framework

Deep News
昨天

Anthropic's chief executive, Dario Amodei, is calling for a deliberate slowdown in the pace at which frontier AI model capabilities are enhanced, aiming to buy crucial time for safety evaluations and the implementation of protective measures. His proposal centers on three key pillars: embedding independent auditors within AI firms, aligning safety standards across leading developers, and fostering international cooperation to manage cross-border risks.

The first suggestion involves granting qualified, independent evaluation teams persistent, near-employee-level access to frontier AI companies. This access would allow these external reviewers to scrutinize model testing, risk reports, safety frameworks, and incident-handling procedures. Anthropic has pledged to be the first to adopt this measure, offering external parties visibility comparable to its own risk-assessment staff while allowing them to publish findings without editorial control from the company. However, this arrangement raises significant questions regarding the selection of evaluators, funding sources, and confidentiality obligations. Given that model weights, training infrastructure, and security vulnerabilities are highly sensitive, expanding access also escalates information security requirements.

Addressing competitive pressures, Amodei's second recommendation calls for major AI laboratories to establish common standards for risk testing, safety thresholds, and conditions for model release. He argues that competitive dynamics could otherwise push companies to accelerate training and deployment schedules, leaving safety measures struggling to keep pace with model capabilities. Anthropic's own "Responsible Scaling Policy" already operates on a tiered system, requiring heightened deployment protections and information security as models cross thresholds related to biological, cyber, autonomous research, or loss-of-control risks. Cooperation between companies, though, could brush against antitrust boundaries. While establishing shared safety testing standards serves the public interest, any arrangements that restrict market entry for smaller firms, coordinate product launches, or otherwise stifle competition could invite regulatory scrutiny.

The third element of the proposal centers on intergovernmental collaboration, including forging basic safety agreements even with strategic rivals like China. Amodei emphasizes that models and technology flow across borders, meaning a unilateral slowdown by one nation or company could simply shift high-risk development to regions with weaker oversight. Anthropic's list of potential hazards includes models assisting biological weapons activity, discovering software vulnerabilities at scale, attacking critical infrastructure, and AI systems escaping developer control. The potential for AI to autonomously participate in developing next-generation models could further accelerate capability gains. To counter these threats, the company suggests that frontier developers should publish system cards and periodic risk reports, undergo independent audits, and provide necessary information to designated government agencies. For models posing potentially catastrophic risks, governments should possess limited authority to block or delay deployment, backed by clear thresholds and an appeals process.

While the industry has shown verbal support, concrete agreements remain elusive. OpenAI's chief executive, Sam Altman, has backed the idea of granting independent evaluators internal access, indicating that OpenAI will adopt a similar stance. DeepMind's Demis Hassabis and xAI's Elon Musk have also publicly endorsed the general direction proposed by Amodei. However, these expressions of support have yet to crystallize into a binding industry pact. Key details, such as which capability metrics would trigger a slowdown or pause, whether independent reviews would carry enforceable power, and how an international agreement might be implemented, still require further definition by both companies and governments.

免責聲明:投資有風險,本文並非投資建議,以上內容不應被視為任何金融產品的購買或出售要約、建議或邀請,作者或其他用戶的任何相關討論、評論或帖子也不應被視為此類內容。本文僅供一般參考,不考慮您的個人投資目標、財務狀況或需求。TTM對信息的準確性和完整性不承擔任何責任或保證,投資者應自行研究並在投資前尋求專業建議。

熱議股票

  1. 1
     
     
     
     
  2. 2
     
     
     
     
  3. 3
     
     
     
     
  4. 4
     
     
     
     
  5. 5
     
     
     
     
  6. 6
     
     
     
     
  7. 7
     
     
     
     
  8. 8
     
     
     
     
  9. 9
     
     
     
     
  10. 10