Embodied Data Pyramid

Embodied data pyramid is 谢晨’s frame for combining data sources in Embodied AI. In 134. 【数据的综述】和谢晨聊，新时代的石油、历史、版图、数据金字塔、定价与Recipe, the top layer is real robot teleoperation or body data, the middle layer is simulation data, and the bottom layer is internet-scale and human first-person data.

170: 【具身季报 26Q2】世界模型大风不停，和不想被贴标签的人 extends the top and bottom of this pyramid by naming concrete collection shifts: Aloha-style teleoperation, UMI-style body-free data, first-person video, whole-body motion capture, and dexterous-hand datasets. The source’s Embodied Robot Data Paradigms concept keeps the pyramid dynamic by asking which new data source unlocks which robot capability.

从会跳舞到有感知，触觉是机器人通往智能的门票吗？｜ S10E19 adds a tactile-data layer through Yimu Technology / 一目科技. Eric Li Zhiqiang / 李志强 says real tactile robot data is valuable but expensive, simulation data should be expanded with Optical Tactile Sensing, and large-scale video data can still help pretrain visual priors before touch and action are added.

Key Claims

• Real robot data is most physically accurate but expensive, hard to scale, and therefore too narrow to carry general robotics learning alone.
• Robotics Simulation Evaluation is the scalable middle layer because it can generate repeated tasks, failures, counterfactuals, and evaluation scenarios.
• Human first-person and internet data are larger and less body-specific, but can provide scene, object, task, and daily-life priors.
• The pyramid should be a loop: real-world and human data can be converted into simulation worlds, while simulation outputs must be checked against real-world results.
• This creates a tension with pure Real Robot Data Strategy: real data stays necessary, but the source argues it should not dominate the recipe by default.
• Hardware-specific dexterous-hand data complicates the pyramid because “real robot data” does not transfer cleanly when hand structure, sensors, or drive method change.
• Tactile Sensing creates a special data problem because it is high-frequency and continuous while also being closer to force and deformation ground truth than ordinary visual data.
• TouchNet is proposed as a field-level tactile dataset, but the source still places it inside a broader recipe of scarce real data, simulation, and video pretraining.

Connections

• Data As Education — broader metaphor behind the pyramid.
• Data Recipe Co-Creation — process for discovering how much of each layer improves a model.
• Physical World Data Flywheel — adjacent real-world data-loop concept from the Xinghaitu source.
• World Models and Vision Language Action Models — model routes that need physical-world data and evaluation.
• 光轮智能 — company building simulation-centered data infrastructure in the source.
• Embodied Robot Data Paradigms, Dexterous Manipulation, and Robot Teleoperation and Remote Takeover — collection-method updates from the LateTalk source.
• Tactile Sensing, Optical Tactile Sensing, TouchNet, and Tactile Transformer Encoder — tactile data, sensor route, dataset, and model-interface additions from the What’s Next source.