Biggest AI Rivals Agree They Need to Slow it Down

Dow Jones
4小時前

The leaders of three of the biggest AI companies agreed that they needed to slow development of the technology before their advances create a menace that can't be controlled.

It was a rare display of unity by Elon Musk, Sam Altman and Dario Amodei, who have been spending tens of billions of dollars to develop evermore powerful artificial intelligence models. Altman even suggested that his company, OpenAI, may need to delay its much-anticipated IPO to focus on safety.

Amodei, the CEO of Anthropic, kicked off the discussion Saturday with a blog post saying that the risks being posed by today's cutting edge tools were simply too great to continue development at the same breakneck pace. His warning came days after one of his employees stoked fears that there was a greater than 10% chance that AI systems could destroy civilization.

Two months ago, an AI swarm broke out of a lab and went on a hacking spree. Amodei said a swarm with greater capabilities and "a similar level of misalignment could have caused catastrophic damage." Such a rogue swarm could take over the internet in six to 12 months, he warned.

In a post to his personal blog, Amodei called for the AI industry to pace development of cutting-edge tools, suggesting new measures including more global coordination among democratic countries and the introduction of embedded third-party safety evaluators.

"I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong," he wrote in the post.

Amodei said AI's progress since the summer, driven by AI systems that can improve on their own without human intervention, "could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all."

In an interview with Fortune, Altman said OpenAI likely wouldn't go public this year given all the safety concerns. "I actually think that, given everything happening with safety, right now would be an ill-advised moment to go public, and we don't feel pressure on that," Altman told Fortune in the interview posted online Saturday.

"This week has obviously been a week where a lot more people are grappling with the issues in front of us," Altman told Fortune. "This is a conversation that the world is overdue for and we are clearly entering a realm of extremely capable models and a high stake and a need to get things right."

OpenAI, the maker of ChatGPT, filed paperwork for an IPO in June, a week after Anthropic filed for its own public listing. Anthropic, maker of Claude, could raise up to $100 billion and value the artificial-intelligence giant at around $2 trillion, the Journal has reported.

In a post on X, Amodei said Anthropic would provide third-party evaluators with "permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models' alignment during training."

Shortly after Amodei published his post, Musk, the CEO of SpaceX, said in an X post, "Dario is right." Musk was a co-founder of OpenAI and founded his own AI company, xAI, which is now part of SpaceX.

Altman said on X he agreed with Amodei about the need for pacing. "Committing to having independent evaluators with employee-like access is a great idea, and we will do the same," he wrote, promising to share more details soon.

Even some big AI investors agreed with Amodei. "While we must protect AI competition, this is an important step forward in finding the right balance between speed, self regulation & safety," hedge-fund manager and venture capitalist Brad Gerstner posted on X.

Amodei's pledge comes as some of his employees are wrestling with the potential existential risks of the technologies they are building. On Tuesday, Anthropic researcher Jacob Coxon resigned from the company, saying he didn't want to participate in an industry that he perceived as out-of-control.

Amodei said two factors prompted Saturday's post. First, the growing capabilities of AI systems, which are increasingly able to build faster versions of themselves-something known as recursive self-improvement.

The second factor was the July hack of the AI-software company Hugging Face by a swarm of as many as 1,200 agents that escaped from a test environment at OpenAI. That incident, and the investigations that it sparked, revealed that AI test systems had been engaged in a range of online activity that had gone unnoticed by the world's largest AI companies, including Anthropic.

On Friday, The Wall Street Journal reported that a May cybersecurity incident, known as GemStuffer, was actually carried out by OpenAI agents, unbeknown to investigators.

 

應版權方要求,你需要登入查看該內容

免責聲明:投資有風險,本文並非投資建議,以上內容不應被視為任何金融產品的購買或出售要約、建議或邀請,作者或其他用戶的任何相關討論、評論或帖子也不應被視為此類內容。本文僅供一般參考,不考慮您的個人投資目標、財務狀況或需求。TTM對信息的準確性和完整性不承擔任何責任或保證,投資者應自行研究並在投資前尋求專業建議。

熱議股票

  1. 1
     
     
     
     
  2. 2
     
     
     
     
  3. 3
     
     
     
     
  4. 4
     
     
     
     
  5. 5
     
     
     
     
  6. 6
     
     
     
     
  7. 7
     
     
     
     
  8. 8
     
     
     
     
  9. 9
     
     
     
     
  10. 10