NVIDIA Releases Four Cosmos 3 Nano Checkpoints for Interactive World Simulation
NVIDIA open-sources specialized variants of Cosmos 3 Nano for autonomous driving and robotics, enabling multi-sensor synthetic data generation and physics-based video prediction.
Last verified:
Four Specialized Cosmos 3 Models for Simulation
According to the Hugging Face Blog, NVIDIA expanded its Cosmos world model platform with four new open-weight checkpoints designed for interactive simulation across robotics and autonomous driving. Each variant—Cosmos3-Nano-Transfer-Auto, Cosmos3-Nano-Sim-Auto, Cosmos3-Nano-Sim-Transfer, and Cosmos3-Nano-Sim-Bimanual—represents a different application of the base 16B-parameter Cosmos 3 Nano architecture, post-trained for specific sensor configurations and control interfaces.
Multi-Sensor Driving Scene Generation
Cosmos3-Nano-Transfer-Auto generates synchronized multi-view driving data across up to 11 camera perspectives, including fisheye lenses, with coordinated RGB and LiDAR output. The model accepts World Scenario Map (WSM) controls to specify scene structure and traffic dynamics, alongside text prompts for appearance variation—enabling researchers to systematically vary weather, lighting, and time-of-day conditions without collecting new real-world footage. Joint LiDAR generation uses aligned HD-map range-map controls to ensure geometric consistency between visual and 3D sensor modalities, addressing a fundamental challenge in autonomous vehicle development: ensuring appearance and geometry agreement across synchronized observations.
Streaming Generation and Robot Action Prediction
Cosmos3-Nano-Sim-Auto enables streaming generation of synchronized camera and LiDAR observations, connecting ongoing video prediction to interactive control loops. Cosmos3-Nano-Sim-Bimanual predicts visual consequences from robot or camera actions, providing a foundation for embodied AI research. The third variant, Cosmos3-Nano-Sim-Transfer, converts physics simulator controls and text prompts into photorealistic video—allowing researchers to adapt existing physics engines (often used for rapid iteration) without re-engineering perception pipelines.
Open Weights and Licensing Strategy
All four checkpoints are released under OpenMDW1.1 licensing, permitting both commercial and non-commercial deployment. The Hugging Face Blog emphasizes NVIDIA’s commitment to providing post-training recipes alongside the weights, enabling external researchers to adapt the models to custom sensor rigs, domains, and control interfaces—positioning Cosmos as a foundation platform rather than a finished product.
Why This Matters
The release democratizes access to high-quality synthetic world models for two high-stakes domains: autonomous vehicles and robotics. Teams evaluating policy robustness, stress-testing perception pipelines, or conducting domain adaptation now have open-weight alternatives to proprietary simulation platforms. The multi-sensor coordination (synchronized camera-LiDAR) is particularly significant for AV researchers, as it sidesteps the geometric inconsistency artifacts that plague single-modality synthetic data generation. For robotics, the explicit action-to-video prediction path (Sim-Bimanual) may accelerate research into visuomotor learning and model-based reinforcement learning—though the critical question remains whether model-generated rollouts support faithful policy learning, a claim the Hugging Face Blog explicitly flags as unresolved. Independent validation against real-world policy transfer will determine whether these checkpoints mature into production-grade tools or remain research artifacts.
Frequently Asked Questions
What is the primary use case for these Cosmos 3 Nano checkpoints?
The four checkpoints address robotics simulation (action-to-visual prediction) and autonomous driving (multi-sensor synthetic data generation with coordinated camera and LiDAR streams). They enable researchers to build interactive training and evaluation environments without collecting expensive real-world data.
Can these models generate synchronized camera and LiDAR data?
Yes. Cosmos3-Nano-Sim-Auto generates synchronized camera and LiDAR observations for driving simulation, while Cosmos3-Nano-Transfer-Auto supports joint RGB–LiDAR generation across up to 11 camera views guided by World Scenario Map controls and text prompts.
What licensing terms apply to these models?
All four checkpoints use OpenMDW1.1 licensing, which permits both commercial and non-commercial use under its terms.
How many parameters does each checkpoint have?
Each of the four post-trained variants has 16 billion trainable parameters.