英伟达提出AI内存全栈协同设计方案 Vera Rubin平台每GPU内存容量仅增约3% 推理吞吐量提升最高30倍

格隆汇
Nov 06
英伟达11月5日发布文章称,随着AI模型规模扩大、上下文窗口延长以及智能体应用普及,GPU内存压力持续上升,HBM供应趋紧、价格上涨。英伟达提出通过GPU、CPU、网络、存储及软件协同设计,提高内存使用效率,而非单纯增加GPU内存容量。 具体来看,英伟达通过NVFP4低精度格式压缩模型权重和KV Cache,将暂时不活跃的上下文转移至成本更低的存储层,并利用多GPU分布式部署释放HBM空间。 英伟达披露,在SemiAnalysis AgentX测试中,GB300 NVL72在特定工作负载下的单位成本推理吞吐量最高达到H200平台的10倍;Vera Rubin NVL72相较GB300 NVL72,单位成本推理吞吐量最高提升30倍,而单GPU HBM容量仅增加约3%。

Disclaimer: Investing carries risk. This is not financial advice. The above content should not be regarded as an offer, recommendation, or solicitation on acquiring or disposing of any financial products, any associated discussions, comments, or posts by author or other users should not be considered as such either. It is solely for general information purpose only, which does not consider your own investment objectives, financial situations or needs. TTM assumes no responsibility or warranty for the accuracy and completeness of the information, investors should do their own research and may seek professional advice before investing.

Most Discussed

  1. 1
     
     
     
     
  2. 2
     
     
     
     
  3. 3
     
     
     
     
  4. 4
     
     
     
     
  5. 5
     
     
     
     
  6. 6
     
     
     
     
  7. 7
     
     
     
     
  8. 8
     
     
     
     
  9. 9
     
     
     
     
  10. 10