Technology firm NVIDIA has introduced NVIDIA Cosmos 3, a foundational open-world model designed for physical artificial intelligence. This new model is built on a groundbreaking Mixture-of-Transformers architecture, integrating visual reasoning, world generation, and action prediction into a single system. It addresses a central challenge in physical AI: enabling robots, autonomous vehicles, or visual agents to generalize in the real world despite limited training data and fragmented simulation stacks.

The Cosmos 3 model is capable of natively understanding and generating text, images, video, ambient sound, and actions with leading physical accuracy. This advancement can reduce the training and evaluation cycles for physical AI from months down to just days. Its hybrid Transformer architecture combines reasoning Transformers with expert generative Transformers, allowing Cosmos 3 to analyze object interactions, motion, and spatiotemporal relationships before generating video and action trajectories.

In physical AI benchmark tests, the Cosmos 3 series models have achieved top results. It leads in world generation accuracy on benchmarks including Artificial Analysis, Physics-IQ, PAI-Bench, and R-Bench. It also excels in action policy for RoboLab and RoboArena, and in visual understanding on VANTAGE-Bench and the TAR leaderboard.

Trained on one of the largest multimodal physical AI datasets, which includes billions of text, image, video, sound, and action trajectory samples, the model provides developers with a powerful pre-trained foundation. This empowers them to build physical AI systems with less data and at a lower training cost.

NVIDIA founder and CEO Jensen Huang stated that the era of a physical AI explosion is imminent, thanks to multiple breakthroughs in multimodal reasoning for language, vision, and world models. He noted that the Cosmos 3 series opens up frontier, full-modality models, enabling developers to make a generational leap in building robots, autonomous vehicles, and visual AI capable of perceiving, reasoning, planning, and taking action in the physical world.