NVIDIA Unveils Omniverse NuRec, Cosmos Models for Robotics Solutions Acceleration

来源：NVIDIA | 发布时间：2025-08-31

摘要：全新 NVIDIA Omniverse NuRec 3D Gaussian Splatting（3DGS）库支持大规模世界重建 • 全新 NVIDIA Cosmos 模型支持世界生成与空间推理 • 全新 NVIDIA RTX PRO Blackwell 服务器和 NVIDIA DGX Cloud 让开发者能够在任何地方运行需求严苛的仿真 • 物理 AI 领先者 Amazon Devices & Services、Boston Dynamics、Figure AI 和 Hexagon 采用仿真和合成数据生成技术

SIGGRAPH—NVIDIA today announced new NVIDIA Omniverse™ libraries and NVIDIA Cosmos™ world foundation models (WFMs) that accelerate the development and deployment of robotics solutions.

Powered by new NVIDIA RTX PRO™ Servers and NVIDIA DGX™ Cloud, the libraries and models let developers anywhere develop physically accurate digital twins, capture and reconstruct the real world in simulation, generate synthetic data for training physical AI models and build AI agents that understand the physical world.

“Computer graphics and AI are converging to fundamentally transform robotics,” said Rev Lebaredian, vice president of Omniverse and simulation technologies at NVIDIA. “By combining AI reasoning with scalable, physically accurate simulation, we’re enabling developers to build tomorrow’s robots and autonomous vehicles that will transform trillions of dollars in industries.”

New NVIDIA Omniverse Libraries Advance Applications for World Composition
New NVIDIA Omniverse software development kits (SDKs) and libraries are now available for building and deploying industrial AI and robotics simulation applications.

New Omniverse SDKs introduce data interoperability between MuJoCo (MJCF) and Universal Scene Description (OpenUSD), enabling over 250,000 MJCF robot learning developers to seamlessly simulate robots across platforms.
New Omniverse NuRec libraries and AI models introduce Omniverse RTX ray-traced 3D Gaussian splatting, a rendering technique that lets developers capture, reconstruct and simulate the real world in 3D using sensor data.
NVIDIA Isaac Sim™ 5.0 and NVIDIA Isaac Lab 2.2 open-source robot simulation and learning frameworks are now available on GitHub. Isaac Sim now includes NuRec neural rendering and new OpenUSD-based robot and sensor schemas that help robot developers close the simulation-to-reality gap.

Omniverse NuRec rendering is now integrated in CARLA, a leading open-source simulator used by over 150,000 developers. Autonomous vehicle (AV) toolchain leader Foretellix is integrating NuRec, NVIDIA Omniverse Sensor RTX™ and Cosmos Transfer to enhance its scalable synthetic data generation with physically accurate scenarios. Voxel51’s data engine for visual and multimodal AI, FiftyOne, supports NuRec to ease data preparation for reconstructions. FiftyOne is used by customers such as Ford and Porsche.

Boston Dynamics, Figure AI, Hexagon, RAI Institute, Lightwheel and Skild AI are adopting Omniverse libraries, Isaac Sim and Isaac Lab to accelerate their AI robotics development, while Amazon Devices & Services is using them to power a new manufacturing solution.

Cosmos Advances World Generation for Robotics
Cosmos WFMs, downloaded over 2 million times, let developers generate diverse data for training robots at scale using text, image and video prompts.

New models announced at SIGGRAPH deliver major advances in synthetic data generation speed, accuracy, language support and control:

Cosmos Transfer-2, coming soon, simplifies prompting and accelerates photorealistic synthetic data generation from ground-truth 3D simulation scenes or spatial control inputs like depth, segmentation, edges and high-definition maps.
A distilled version of Cosmos Transfer reduces the 70-step distillation process to one so developers can run the model on NVIDIA RTX PRO Servers at unprecedented speed.

Lightwheel, Moon Surgical and Skild AI are using Cosmos Transfer to accelerate physical AI training by simulating diverse conditions at scale.

Cosmos Reason Breaks Through World Understanding
Since the introduction of OpenAI’s CLIP model, vision language models (VLMs) have transformed computer-vision tasks like object and pattern recognition. However, they have not yet been able to solve multistep tasks nor handle ambiguity or novel experiences.

NVIDIA Cosmos Reason — a new open, customizable, 7-billion-parameter reasoning VLM for physical AI and robotics — lets robots and vision AI agents reason like humans, using prior knowledge, physics understanding and common sense to understand and act in the real world.

Cosmos Reason can be used for robotics and physical AI applications including:

Data curation and annotation, which enables developers to automate high-quality curation and annotation of massive, diverse training datasets.
Robot planning and reasoning, acting as the brain for deliberate, methodical decision-making in a robot vision language action (VLA) model. Cosmos Reason lets robots interpret environments and, given complex commands, break them down into tasks and execute them using common sense, even in unfamiliar environments.
Video analytics AI agents built on the NVIDIA Blueprint for video search and summarizationthat can extract valuable insights and perform root-cause analysis on massive volumes of video data.

NVIDIA’s robotics and NVIDIA DRIVE™ teams are using Cosmos Reason for data curation and filtering, annotation and VLA post-training. Uber is using it to annotate and caption AV training data.

Magna is developing with Cosmos Reason as part of its City Delivery platform — a fully autonomous, low-cost solution for instant delivery — to help vehicles adapt more quickly to new cities. Cosmos Reason adds world understanding to the vehicles’ long-term trajectory planner. VAST Data, Milestone Systems and Linker Vision are adopting Cosmos Reason to automate traffic monitoring, improve safety and enhance visual inspection in cities and industrial settings.

New NVIDIA AI Infrastructure Powers Robotics Workloads Anywhere
To enable developers to take full advantage of these advanced technologies and software libraries, NVIDIA announced AI infrastructure designed for the most demanding workloads.

NVIDIA RTX PRO Blackwell Servers offer a single architecture for every robot development workload across training, synthetic data generation, robot learning and simulation.
NVIDIA DGX Cloud, available on Microsoft Azure Marketplace, now offers Omniverse developers a fully managed platform to simplify streaming OpenUSD- and NVIDIA RTX™-based applications at scale from the cloud, minimizing infrastructure orchestration and management. Accenture and Hexagon are among the first industry leaders to adopt the platform.

Accelerating the Developer Ecosystem
To help robotics and physical AI developers advance 3D and simulation technology adoption, NVIDIA also announced:

OpenUSD Curriculum and Certification, which addresses demand for USD expertise, with support from AOUSD members Adobe, Amazon Robotics, Ansys — part of Synopsys, Autodesk, Pixar, PTC, Rockwell Automation, SideFX, Siemens, TCS and Trimble, as well as industry leaders such as Hexagon.
Open-source collaboration with Lightwheel to integrate robot policy training and evaluation frameworks into NVIDIA Isaac Lab, featuring parallel reinforcement learning training capabilities, benchmarks and simulation-ready assets for robot manipulation and locomotion.

NVIDIA 通过全新 Omniverse 库、Cosmos 物理 AI 模型及 AI 计算基础设施，为机器人领域开启新篇章

SIGGRAPH —— 2025 年 8 月11日 —— NVIDIA 宣布推出全新 NVIDIA Omniverse™ 库和 NVIDIA Cosmos™ 世界基础模型 (WFM)，加速机器人解决方案的开发和部署。

这些库和模型由全新 NVIDIA RTX PRO™ 服务器和 NVIDIA DGX™ Cloud提供支持，可帮助开发者在任何地方开发物理精确的数字孪生，在仿真中捕捉和重建现实世界，生成用于训练物理 AI 模型的合成数据，并构建理解物理世界的 AI 智能体。

NVIDIA Omniverse 和仿真技术副总裁 Rev Lebaredian 表示：“计算机图形与 AI 的融合，从根本上改变着机器人。通过将 AI 推理与物理精准的可扩展仿真相结合，我们正帮助开发者打造未来的机器人和智能汽车，这将为各行业带来价值数万亿美元的改变。”

全新 NVIDIA Omniverse 库助力世界构建应用

全新 NVIDIA Omniverse 软件开发套件 (SDK) 和库现已推出，可用于构建和部署工业 AI 和机器人开发仿真应用。

全新 Omniverse SDK 推出了 MuJoCo (MJCF) 与通用场景描述 (OpenUSD) 之间的数据互操作性，帮助超过 250,000 名 MJCF 机器人学习开发者无缝地模拟各平台的机器人。
全新 Omniverse NuRec 库和 AI 模型引入了 Omniverse RTX 光线追踪 3D Gaussian Splatting (3DGS) 技术，这种渲染技术可帮助开发者利用传感器数据，在 3D 中截取、重建和模拟现实世界。
NVIDIA Isaac Sim™ 5.0 和 NVIDIA Isaac Lab 2.2 开源机器人仿真和学习框架现已在 GitHub 上提供。Isaac Sim 现在包括 NuRec 神经渲染和基于 OpenUSD 的全新机器人及传感器模式，帮助机器人开发者缩小仿真与现实之间的差距。

Omniverse NuRec 渲染现已集成到超过 15 万名开发者使用的领先开源模拟器 CARLA 中。智能汽车 (AV) 工具链领先者 Foretellix 正在集成 NuRec, NVIDIA Omniverse Sensor RTX™ 和 Cosmos Transfer，以增强其可扩展的合成数据生成能力，提供物理精准的场景。 Voxel51 的视觉和多模态 AI 数据引擎 FiftyOne 支持 NuRec，帮助简化重建的数据准备工作。 FiftyOne 的客户包括福特和保时捷等。

Boston Dynamics、Figure AI、Hexagon、RAI Institute、Lightwheel 和 Skild AI 正在采用 Omniverse 库、Isaac Sim 和 Isaac Lab，加速其 AI 机器人开发。 Amazon Devices & Services 正在使用它们来支持新的制造解决方案。

Cosmos 推动机器人世界生成进步

Cosmos WFM 下载量超过 200 万次，可帮助开发者通过文本、图像和视频提示大规模生成各种用于训练机器人的数据。

NVIDIA 在 SIGGRAPH 大会上宣布的新模型在合成数据生成速度、模型精度、语言支持和控制方面取得了重大进步：

Cosmos Transfer-2 即将发布，它简化了提示并能从地面实况 3D 仿真场景或空间控制输入（如深度、分割、边缘和高清地图）中，快速生成逼真的合成数据。
Cosmos Transfer 的精简版本将 70 个步骤的蒸馏过程简化为一个步骤，开发者能够以前所未有的速度在 NVIDIA RTX PRO 服务器上运行模型。

Lightwheel、Moon Surgical 和 Skild AI 正在利用 Cosmos Transfer 大规模模拟各种条件，加速物理 AI 训练。

Cosmos Reason 突破对世界的理解

自 OpenAI 的 CLIP 模型推出以来，视觉语言模型 (VLM) 改变了物体和模式识别等计算机视觉任务。但是，它们尚未能够解决多步骤任务，也无法处理模糊或新颖的体验。

NVIDIA Cosmos Reason 是一款面向物理 AI 和机器人开发的全新开源、可定制的 70 亿参数推理 VLM，可让机器人和视觉 AI 智能体像人类一样推理，利用先验知识、物理理解和常识，理解现实世界并付诸行动。

Cosmos Reason 可用于机器人开发和物理 AI 应用，包括：

数据管理与注释，帮助开发者自动对海量、多样化的训练数据集进行高质量管理与注释。
机器人规划和推理，在机器人视觉语言行为 (VLA) 模型中，充当有意识、有条理决策的大脑。Cosmos Reason 让机器人能够解读环境，并在收到复杂指令时，将其分解为任务，并运用常识执行这些任务，即使是在不熟悉的环境中。
视频分析 AI 智能体基于可用于视频搜索和总结的 NVIDIA Blueprint 构建，能够从海量视频数据中提取有价值的洞察并进行根本原因分析。

NVIDIA 的机器人和 NVIDIA DRIVE™ 团队正在使用 Cosmos Reason 进行数据管理和筛选、注释和 VLA 后训练。Uber 正使用它为辅助驾驶训练数据添加注释和字幕。

Magna 正在其城市配送平台 (City Delivery platform) 中使用 Cosmos Reason 进行开发。该平台是一种完全自主、低成本的即时配送解决方案，旨在帮助车辆更快地适应新城市。Cosmos Reason 为车辆的长期轨迹规划器增加了对世界的理解能力。VAST Data、Milestone Systems 和 Linker Vision 正在采用 Cosmos Reason 实现城市和工业环境的自动化交通监控，提高安全性，并提升视觉检查能力。

全新 NVIDIA AI 基础设施为任何地方的机器人开发工作负载提供支持

为帮助开发者充分利用这些先进技术和软件库，NVIDIA 宣布推出专为极高要求工作负载而设计的 AI 基础设施。

NVIDIA RTX PRO Blackwell 服务器为训练、合成数据生成、机器人学习和仿真等所有机器人开发工作负载提供统一架构。
NVIDIA DGX Cloud 现已在 Microsoft Azure Marketplace 上提供，为 Omniverse 开发者提供一个完全托管的平台，简化从云端大规模流式传输基于 OpenUSD 和 NVIDIA RTX™ 的应用，从而最大限度地减少基础设施编排和管理。Accenture 和 Hexagon 是首批采用该平台的行业领先者。

加速开发者生态系统

为了帮助机器人开发者和物理 AI 开发者推动 3D 和仿真技术的采用，NVIDIA 还宣布：

OpenUSD 课程和认证：满足对通用场景描述专业知识的需求，由 OpenUSD 联盟成员 Adobe、Amazon Robotics、Ansys (Synopsys 旗下）、Autodesk、Pixar、PTC、Rockwell Automation、SideFX、Siemens、TCS 和 Trimble 以及 Hexagon 等行业领先者的支持。
与 Lightwheel 开源协作，将机器人策略训练和评估框架集成到 NVIDIA Isaac Lab 中，具备并行强化学习训练功能，以及机器人操作和运动的基准测试和仿真就绪资产。