Agent Data

从蒸馏到合成数据到 RSI，模型竞争的下一个焦点是什么？｜对谈 Evolvent AI 联创孟繁青 adds a synthetic and RSI-oriented version. 孟繁青 distinguishes constructed Synthetic Agent Data from real human AI-use traces: synthetic trajectories can be cleaner and more trainable when the task, environment, and scoring are designed, while ordinary personal usage data is often too noisy and compliance-heavy to sell directly.

Agent data is Alexandr Wang’s term in Alexandr Wang on Scale and AI Data Infrastructure for data about how people complete tasks, not just what final answers look like. He describes the needed data as traces of thinking, information gathering, constraint checking, decision-making, and action.

The concept is tied to the shift from chatbots to agents. If AI systems move from talking to doing, then AI Data Infrastructure must capture how capable people actually perform work such as booking flights, reviewing contracts, building software features, or making product decisions.

Bytes: Week in Review - Apple’s new CEO, Meta’s latest AI play, and Roblox’s safety updates adds a contested employee-data version. The episode says Meta wants real examples of how people use computers so agents can complete everyday computer tasks, turning mouse movements, clicks, and keystrokes into possible Workplace Behavior Training Data. This reinforces the value of process data while making labor consent and Workplace AI Transparency central.

Key Claims

• Agent data is process data, not only input-output pairs.
• Many valuable workflows lack good training data because the reasoning, checking, and tool-use steps are not captured.
• Agent data depends on human experts because models still hallucinate, get stuck, and need guidance in real-world domains.
• Capturing agent data could make Data As Education more concrete by turning expert task performance into teachable sequences.
• The data is valuable only if privacy, permissions, task context, and evaluation are handled carefully.
• Workplace process traces may be valuable precisely because they show real tool use, but they also carry stronger employee surveillance risk than public examples.
• Synthetic agent data shifts value toward controllable environments, verifier design, and trace cleaning rather than raw user history.

Connections

• Synthetic Agent Data, Environment-Based Agent Benchmarks, and RSI Data — synthetic and model-improvement data branch.
• Alexandr Wang and Scale AI - source person and company.
• AI Data Infrastructure, Data As Education, and Data Engine Learning Loop - data concepts this extends.
• Agentic Workflow, Human-Agent Collaboration, and Human Judgment Under AI - workflow and oversight context.
• Context Engineering and Persistent Agent Memory - adjacent context layers agents need while acting.
• Meta, Reuters, Computer Use Agent, and Workplace Behavior Training Data - employee process-data branch added by Marketplace Tech Bytes.