Amid growing concerns from employees and outsiders about the potential dangers of artificial intelligence, OpenAI is revisiting a range of safety strategies. One possible approach, in fact, was already under discussion not long ago.
According to a person familiar with the negotiations, a detail not previously reported, OpenAI had been in talks with Anthropic about a legally binding agreement before a string of cybersecurity incidents tied to its own technology and severe warnings from industry professionals emerged. Under the plan, the two companies would have conducted stress tests on each other's large language models. This source indicated that earlier this year, both firms and their legal teams were working to finalize the terms of the arrangement, which involved running multiple types of tests on each other's models to identify vulnerabilities and hidden risks.
It remains unclear whether the deal was signed before OpenAI faced the incidents where unpublished models breached its own and other companies' systems. Spokespeople for both companies declined to comment.
This concept of cross-testing echoes an idea that SpaceX CEO Elon Musk proposed at the All-In Summit last week. Musk suggested that rival AI labs should examine each other's models for security flaws before any commercial release, creating a system of peer review. However, this line of thinking diverges from what OpenAI CEO Sam Altman and others have publicly discussed following the hacking events.
Altman stated that he agrees with Anthropic CEO Dario Amodei's proposal, which would allow independent third-party safety assessors to embed within AI companies. These assessors would have access rights similar to internal employees, enabling them to directly review models and safety procedures rather than only testing the finished products from an external perspective. Altman also supports establishing industry-wide safety standards for evaluating AI model risks, as well as a formal process for disclosing safety incidents to the public and governments.
A person familiar with OpenAI's strategy reported that the company has been examining various options for how companies and governments might collaborate on AI safety. According to this source, those discussions have recently expanded to cover new safety mechanisms that apply both before model training and prior to official release.
Musk's comments came after employees at both OpenAI and Anthropic raised alarms, arguing that current safety measures are insufficient against future hacker attacks or other more severe threats. Before all of this, the two companies, along with Google, had already started exploring the formation of an AI safety standards body responsible for testing and auditing models.
Some other players in the industry oppose these initiatives, arguing that such regulatory measures would slow down the mostly American-led progress of AI development. President Trump does not see AI as an existential threat, and executives at Microsoft and Nvidia stated this week that some of the safety risks raised by OpenAI and Anthropic stem from human error and imperfect engineering systems, not necessarily from the AI itself.
If OpenAI and Anthropic were to finalize this mutual testing agreement, it could also heighten worries about the two companies forming a de facto duopoly in frontier AI. Amodei has previously cautioned that leading firms joining forces to set AI standards could raise antitrust issues. Yet others in the industry argue that such collaboration between companies faces no legal barriers, likening it to how nuclear power firms and cybersecurity vendors cooperate on safety matters, including inspecting each other's products and facilities.
According to the person familiar with the negotiations, the proposed agreement specified that each company would only gain access to the other's application programming interfaces (APIs) for commercially available models, not unreleased ones. Both OpenAI and Anthropic also committed to not retaining any of the other's data during the testing process.