Nvidia Releases Software it Says Can Prevent AI Agents from Going Rogue

Dow Jones
Sep 28

Nvidia launched a software platform to help developers test and deploy artificial-intelligence agents and keep them in a secure environment, ramping up efforts to contain the technology following a string of rogue incidents in recent months.

The AI chip giant said its Nvidia Open Agent Safety Platform includes tools that let developers continuously monitor and govern AI behavior, ensuring agents follow the rules. The company said the software included technology that "can quarantine agents that attempt to move outside their boundaries in milliseconds."

The announcement comes after AI agents from companies including OpenAI, Anthropic, Meta Platforms and Alphabet's Google went rogue in recent months.

Last week, Australian Prime Minister Anthony Albanese said at the United Nations meeting in New York that an OpenAI agent had infiltrated a government-services website in Australia this summer. The company's agents also launched a highly disruptive hack of the company Hugging Face over the summer.

Rival Anthropic said in July that software it was testing got onto the internet and hacked unsuspecting companies without the AI-maker's knowledge in three separate incidents dating back to April.

Nvidia said the pattern across recent incidents was the same: agents were able to circumvent safety controls in sandboxes-environments in which developers test models-and complete their tasks.

"As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety," Nvidia's Chief Executive Jensen Huang said.

News Corp, owner of Dow Jones Newswires and The Wall Street Journal, has a content-licensing partnership with OpenAI.

 

At the request of the copyright holder, you need to log in to view this content

Disclaimer: Investing carries risk. This is not financial advice. The above content should not be regarded as an offer, recommendation, or solicitation on acquiring or disposing of any financial products, any associated discussions, comments, or posts by author or other users should not be considered as such either. It is solely for general information purpose only, which does not consider your own investment objectives, financial situations or needs. TTM assumes no responsibility or warranty for the accuracy and completeness of the information, investors should do their own research and may seek professional advice before investing.

Most Discussed

  1. 1
     
     
     
     
  2. 2
     
     
     
     
  3. 3
     
     
     
     
  4. 4
     
     
     
     
  5. 5
     
     
     
     
  6. 6
     
     
     
     
  7. 7
     
     
     
     
  8. 8
     
     
     
     
  9. 9
     
     
     
     
  10. 10