Sequential Workplace Learning Evaluation

Definition

Sequential workplace learning evaluation tests whether an agent uses feedback, rules, and experience from earlier work episodes to improve its handling of later tasks in the same evolving organizational environment.

Current Synthesis

The source uses Apprentice Bench to distinguish two meanings of long-horizon intelligence. One is endurance within a single extended task; the other is development across many tasks, resembling how a novice employee becomes experienced. The second requires evaluators to preserve order, expose task-dependent feedback, and measure whether acquired knowledge transfers without assuming that one benchmark domain establishes general learning.

Key Claims

• Ordered tasks can test experience accumulation that independent benchmark items cannot observe.
• Useful evaluation should expose organizational conventions, workflow exceptions, and approval rules over time.
• Later performance should depend on appropriate use of earlier feedback rather than simple session length.
• The target is a learning meta-capability, not memorization of one fixed company’s accounting procedures.
• Hidden and harder test sets can limit direct benchmark tuning but cannot by themselves prove generalization.

Evidence

Ordered experience

• 【年终复盘②】RSI可能是NeoLab的机会、PA真的有用、创业要造灭火器还是砖头？｜对话谷雨 describes 100 sequential construction-accounting tasks whose later solutions can use earlier feedback and knowledge.

Employee-development analogy

• 【年终复盘②】RSI可能是NeoLab的机会、PA真的有用、创业要造灭火器还是砖头？｜对话谷雨 defines the desired long horizon as progression from organizational novice to experienced worker.

Anti-overfitting intent

• 【年终复盘②】RSI可能是NeoLab的机会、PA真的有用、创业要造灭火器还是砖头？｜对话谷雨 reports partial test secrecy and a planned harder version to preserve evaluation headroom.

Counterevidence & Qualifications

• Sequential improvement may reflect retrieval from context or notes rather than durable model learning; the distinction should be measured rather than assumed.
• A fixed sequence can reward benchmark-specific strategies, and hidden tasks do not eliminate contamination or domain overfitting.
• Construction accounting is one setting; transfer to other organizations, industries, tools, and time spans remains unproven in the supplied source.
• Scores can entangle base capability, memory, tool use, feedback quality, and evaluator design.

What Changed

• Established a distinct evaluation frame for learning across work episodes rather than only extending single-task runtime.

Related Concepts

• Continual Learning - underlying capability the evaluation seeks to measure.
• Apprentice Bench - concrete benchmark example in the source.
• Environment-Based Agent Benchmarks - broader interactive evaluation family.
• Enterprise Operational Memory - organizational knowledge that later tasks may require.
• Long-Horizon Agent Reliability / 长期智能体可靠性 - adjacent requirement for sustained correct behavior over time.
• Agent Evaluation Benchmarks - general benchmark-design context.