Robot Data Scale Up

150. 对英伟达研究副总裁刘洺堉的4小时访谈：Cosmos 3、世界模型、武术、黄仁勋影响我的，和你不需要击败所有对手 adds Cosmos 3’s “better data” version through Liu Ming-Yu / 刘洺堉. Liu separates navigation data, which can be more shareable, from manipulation data, which is messier because hands, contact, object states, and body configuration matter more. The source says Cosmos 3 adds egocentric data, including human-eye viewpoints and hand-operation footage, to improve Physical AI generalization.

Robot data scale up is the bottleneck named by 沈宇军 in 147. 和蚂蚁灵波沈宇军聊：机器人原生基础模型、大脑和本体的关系、预训练与数据scale up、老师汤晓鸥. Shen says embodied intelligence has not yet reached a robot GPT-1 moment because the field lacks a data-scaling method comparable to internet-scale language data.

In the source, scaling does not mean simply recording more robot hours. 蚂蚁灵波 expands from about 20,000 to about 60,000 hours in its second generation, but Shen emphasizes that stricter cleaning, task choice, body coverage, camera positions, degrees of freedom, and data usability matter as much as volume.

Gig workers train humanoids on household chores adds a consumer-press example of this bottleneck through Household Robot Training Data. Joanna Stern describes startups paying people to film ordinary chores with head-mounted cameras so robot models can capture hand movement, task sequencing, and physical interaction before robots can reliably gather enough experience on their own.

Key Claims

• Real-machine data is valuable because it contains actual sensor noise, body constraints, execution latency, and physical failures.
• eGo-style first-person human data can scale some everyday scene and task coverage, but it may not contain enough hand-specific or contact-specific information for fine manipulation.
• Simulation is useful for evaluation and partial scenes, but the source treats it as insufficient for general flexible-object and complex manipulation training by itself.
• Cross-body data cleaning is hard because camera location, joint structure, hands, chassis, and degrees of freedom differ across embodiments.
• Task assignment is a data problem: a collection run has to cover head, waist, hands, base, wrist cameras, and scene changes rather than repeat one narrow behavior.
• Data quality and data modality can beat raw hours when model teams need action-relevant, physically grounded signal.
• The source’s “data native” next step means collecting data in ways that match what embodied models can actually use.
• Marketplace Tech adds that first-person human chore footage can help scale everyday household-task coverage, but it also raises labor, privacy, and task-quality questions.
• Cosmos 3 adds that egocentric data can support developer starting points, but data still has to be evaluated against task-specific customer pain rather than volume alone.

Connections

• Real Robot Data Strategy — existing page on target-domain and real-machine data.
• Embodied Data Pyramid — adjacent view that balances real robot data with simulation and human/internet data.
• Embodied Robot Data Paradigms — changing collection methods such as teleoperation, body-free capture, first-person video, motion capture, and dexterous-hand data.
• Robotics Simulation Evaluation and Sim2Real — evaluation and simulation context that the source qualifies.
• Robot Experience Data and Robot Reinforcement Learning — Physical Intelligence route where robot-owned attempts and corrections become data.
• Embodied Native Foundation Models and Vision Language Action Models — model routes whose progress depends on scalable embodied data.
• Household Robot Training Data, AI Trainer Labor, and Humanoid Robot Commercialization — paid human demonstration branch added by Marketplace Tech.
• Cosmos 3, World Foundation Models, Liu Ming-Yu / 刘洺堉, and Robot Generalization Performance Tradeoff — Nvidia data and generalization branch added by episode 150.