Agent Evaluation Benchmarks

Definition

Agent evaluation benchmarks are repeatable tests of whether an agent can complete useful work across tools, state, time, and safety constraints, including cases where correct behavior is to refuse, pause, or avoid an action.

Current Synthesis

The sources move agent evaluation beyond static final-answer scoring. Kuse treats evaluation as product infrastructure for detecting behavior shifts across models, runtimes, multi-turn state, tool actions, security attacks, and situations where an agent should not act. Evolvent AI adds simulated work environments and multidimensional scoring, with the complication that benchmark trajectories may later feed post-training or synthetic-data loops.

Agent’s Last Exam extends the same architecture across professional and engineering domains. A valuable task combines real-work relevance with usable tools, an execution environment, and credible verification through program checks, state inspection, rubrics, specialist models, or humans. Leaderboard gains are useful evidence only when evaluation tasks remain held out and improvement transfers; narrow specialization, arbitrary aggregation, task defects, leakage, and saturation can make scores overstate general capability.

Key Claims

• Agent evaluation should cover trajectories, environment state, tools, and long-horizon behavior, not only final prose.
• Safety benchmarks need negative tests for refusal, disclosure, installation, spending, and other inappropriate actions.
• Authentic work relevance makes optimization more valuable but does not guarantee generalization.
• Multidimensional scoring is necessary when agents can reach a result through unsafe, wasteful, or invalid paths.
• Benchmark traces can become training data, increasing the need for explicit provenance and held-out separation.
• Human technical judgment remains necessary for new capabilities that fixed test suites do not yet represent.

Evidence

• Product regression and safety - 我们是如何定义 OpenClaw for Teams 新产品形态的｜对谈 Kuse&Junior 联创兼 CTO 宇豪 describes automated tests across model and runtime changes, state, tools, phishing, prompt injection, disclosure, and non-action.
• Environment and data loop - 从蒸馏到合成数据到 RSI，模型竞争的下一个焦点是什么？｜对谈 Evolvent AI 联创孟繁青 links simulated work, multiple scoring dimensions, trajectories, and post-training data production.
• Cross-domain real work - E253｜谁在给大模型出题、卖题、判卷？聊聊AI数据行业的野蛮生长 presents Agent’s Last Exam as a project for verifiable work beyond code.
• Score interpretation - E253｜谁在给大模型出题、卖题、判卷？聊聊AI数据行业的野蛮生长 distinguishes meaningful real-task optimization from leakage, specialization, arbitrary weights, flawed tests, and saturation.

Counterevidence & Qualifications

Benchmark performance is not deployment proof. Simulators can omit real permissions, latency, failure costs, users, and institutional constraints; public tests can leak into training; aggregate scores hide domain weights; and a benchmark near saturation may retain mostly ambiguous or defective cases. Online A/B tests and user preferences can also disagree with offline evaluation without providing a clean truth standard.

What Changed

• Added real-work correspondence and held-out integrity as tests for whether benchmark optimization is meaningful.
• Added benchmark saturation, weighting, and defective residual tasks as limits on score interpretation.
• Added expert-submission authenticity as an upstream benchmark-quality problem.

Related Concepts

• Environment-Based Agent Benchmarks - interactive task architecture inside the broader benchmark category.
• Benchmark–Training Data Separation / 评测集与训练数据隔离 - rule protecting measurement from training contamination.
• Agent’s Last Exam - cross-domain benchmark project used as the newest case.
• AI Verification - broader problem of establishing correctness and fitness for use.
• Agent Permission Boundaries - authority limits that safety benchmarks must exercise.
• Synthetic Agent Data - trajectory output that can support training but complicates provenance.
• Human Judgment Under AI - interpretation and novel-capability judgment not fully captured by fixed tests.