Comparative Judgment Assessment

Definition

Comparative judgment assessment measures extended work by aggregating repeated pairwise decisions about which of two responses is better.

Current Synthesis

In the episode, Daisy Christodoulou presents comparative judgment as a way to preserve holistic professional judgment while improving consistency and speed. A teacher need not translate every feature of an essay into a detailed rubric score; many simpler comparisons can be combined into a scale.

The method addresses one side of Exam Standardization-Validity Tradeoff by standardizing the aggregation process rather than fully standardizing the evaluator’s reasoning. Its value still depends on sampling, judge quality, task design, aggregation, and validation against the intended construct.

Key Claims

• Pairwise choices can be cognitively simpler than absolute rubric scoring.
• Many judgments can be aggregated into a ranked measurement scale.
• Holistic comparison may preserve qualities that detailed checklists fragment.
• Reliability and validity remain empirical properties, not automatic consequences of the method.

Evidence

• Procedure: 84. Exams describes teachers selecting the better of two pieces of writing across many comparisons.
• Claimed performance: 84. Exams reports Christodoulou’s view that the method is more reliable, valid, and quick than traditional rubric marking.

Counterevidence & Qualifications

The source does not provide study designs, effect sizes, failure cases, inter-rater composition, scale-construction details, or independent replication. Pairwise ranking can improve consistency without proving that the task measures the right knowledge or that all candidates receive equitable treatment.

What Changed

• Created a bounded account separating the method’s procedure from its source-reported benefits.

Related Concepts

• No More Marking - company presented as applying the method.
• Daisy Christodoulou - specialist explaining and advocating the approach.
• Exam Standardization-Validity Tradeoff - assessment problem the method attempts to rebalance.
• Examination Systems - wider selection infrastructure in which the method may operate.