Benchmark–Training Data Separation / 评测集与训练数据隔离

Definition

Benchmark–training data separation is the rule that held-out evaluation tasks must not be disclosed, sold, or reused as model-training examples when the resulting score is meant to measure unseen capability.

Current Synthesis

Benchmarks and training data address related but different questions. A benchmark describes the gap between current performance and a desired capability; training data helps close known gaps. Because benchmark builders understand the task distribution, they may be well placed to create distinct training material, but authority over the exam creates a conflict if the exam itself becomes the lesson.

Leaderboard optimization is therefore not automatically illegitimate. It can represent useful progress when tasks correspond to real user work, evaluation items remain held out, and the learned capability improves several related tests or real outcomes. It becomes weak evidence when data leak, tasks are overly narrow, aggregate weights do not reflect users, tests reject valid solutions, or a nearly saturated benchmark leaves mostly broken and ambiguous items.

Key Claims

• Held-out evaluation tasks should not be sold or used as training examples for the same reported benchmark.
• Benchmark creators can build adjacent training data if the artifacts and governance remain separate.
• Real-work relevance makes score optimization more meaningful but does not remove leakage risk.
• Cross-benchmark and real-world transfer are stronger evidence than improvement on one leaderboard.
• Aggregate benchmark weights embed judgments about which capabilities matter.
• Saturation and defective residual tasks can reduce a benchmark’s discriminating value.

Evidence

• Role separation - E253｜谁在给大模型出题、卖题、判卷？聊聊AI数据行业的野蛮生长 contrasts forward-looking capability measurement with data used to improve present weaknesses.
• Integrity rule - E253｜谁在给大模型出题、卖题、判卷？聊聊AI数据行业的野蛮生长 records 孙一游’s position that the evaluation set itself cannot be sold as training data.
• Meaningful optimization test - E253｜谁在给大模型出题、卖题、判卷？聊聊AI数据行业的野蛮生长 ties useful benchmark gains to authentic tasks, held-out data, and generalized capability.
• Failure modes - E253｜谁在给大模型出题、卖题、判卷？聊聊AI数据行业的野蛮生长 uses weighting, specialized game solving, leakage, ambiguous requirements, strict tests, and saturation as reasons to qualify leaderboard inference.

Counterevidence & Qualifications

The source does not specify a formal contamination audit, disclosure standard, embargo design, or proof that a model has not encountered public benchmark material. Even without direct leakage, repeated leaderboard feedback can support overfitting. Conversely, poor transfer does not always mean cheating; it can reveal that the benchmark sampled a capability too narrowly or that deployment conditions differ from the test environment.

What Changed

• Added an explicit commercial boundary between benchmark authority and adjacent data sales.
• Distinguished legitimate real-work optimization from contamination and narrow leaderboard specialization.

Related Concepts

• Agent Evaluation Benchmarks - benchmark class whose scores depend on held-out integrity.
• Environment-Based Agent Benchmarks - interactive tests where tasks, tools, and verifiers can also generate tempting training traces.
• Agent’s Last Exam - central project case for separating evaluation tasks from training products.
• AI Benchmark Gaming - broader family of score optimization that may or may not reflect useful capability.
• AI Verification - need to verify both task completion and the integrity of the measurement process.
• Synthetic Agent Data - adjacent training material that must be checked for benchmark overlap and leakage.