CG-World: A Large-Scale World-State Dataset and Protocol for World Models
Quick Answer
CG-World is a comprehensive dataset designed for world models, featuring 850,000 temporally aligned segments that capture diverse states and events.
Quick Take
It supports intervention learning and counterfactual reasoning, facilitating advancements in geometry-conditioned video generation and action prediction. The dataset aims to enhance structured supervision for AI applications in embodied intelligence and .
Key Points
- CG-World contains approximately 850,000 segments, each 1-5 seconds long.
- The dataset records multimodal semantics, spatial structures, and various state parameters.
- It enables intervention learning with detailed tracking of factual and counterfactual branches.
- Evaluation shows effectiveness in controlled generation and action modeling tasks.
- Future expansions will focus on community collaboration for a shared data infrastructure.
DeepSignal Analysis
What happened
CG-World is a newly introduced dataset that contains approximately 850,000 temporally aligned segments, capturing various states and events relevant to world models. It is derived from industrial computer graphics production pipelines and includes detailed records of intermediate states and interventions. The dataset aims to support advancements in AI applications, particularly in embodied intelligence and Physical AI.
Key evidence
- CG-World v1 includes around 850,000 segments, each lasting 1-5 seconds, which are temporally aligned to capture diverse states and events.
- The dataset explicitly records various intermediate states, such as multimodal semantics, spatial structure, and physics caches, enhancing the learning process for AI models.
- CG-World is designed to facilitate intervention learning and counterfactual reasoning, with defined branch lineages that cover factual and alternative outcomes.
Why it matters
The introduction of CG-World addresses a significant gap in existing datasets, which often fail to capture the full dynamics of states, actions, and observations. By providing structured supervision, it enables more effective learning for AI systems, particularly in generating controlled outputs and modeling actions. This could lead to improved performance in applications requiring embodied intelligence and Physical AI, which are increasingly relevant in various industries.
Paper Resources
Source Excerpt
World models must learn the joint dynamics of states, actions, events, and observations, yet existing video, robotics, and simulation datasets usually capture only part of this structure. We introduce CG-World, a large-scale world-state dataset and protocol derived from industrial computer graphics production pipelines. CG-World explicitly records intermediate states, including multimodal semantics, spatial structure, skeletal and controller states, motion curves, camera and lighting parameters,
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.