Reward Prediction Error Learning

Definition

Reward prediction error learning is the wiki’s frame for updating value and future behavior when outcomes or successive predictions differ from what was expected.

Current Synthesis

Across the sources, reward prediction error connects dopamine to learning rather than only to pleasure. The simpler applied account compares an obtained outcome with an expected one: better-than-expected outcomes can strengthen future pursuit, worse-than-expected or omitted rewards can weaken it, and reliable intermediate cues can acquire motivational force before the endpoint.

How Dopamine & Serotonin Shape Decisions, Motivation & Learning | Dr. Reed Montague sharpens the computational account through temporal-difference learning. The relevant error can be the difference between one prediction and the next prediction, not only the gap between an initial expectation and a final reward. This allows value to propagate backward through a long sequence: weather cues can predict later rain, messages can update an uncertain relationship, and each step in foraging or work can alter expected value before a terminal outcome exists.

Goal and habit sources turn the model into behavior design. Intermediate milestones can make distant pursuit assessable, while preparation, effort, execution, and aftermath can all become learned parts of a task sequence. The condensed Essentials: How to Set & Achieve Goals edit repeats the earlier goal episode’s anticipated-, unexpected-, and omitted-reward account without adding an independent test. These are practical applications of an expectation-sensitive model, not proof that every stage produces one measurable dopamine signal or that every variable reward is beneficial.

The synthesis therefore nests rather than replaces the older explanations: endpoint comparison is a useful special case, while temporal-difference updating better explains learning across delay. Motivation remains historical, cue-sensitive, and state-dependent, and neither human learning nor reinforcement learning is reducible to dopamine alone.

Key Claims

• Prediction errors update value and future action rather than merely producing pleasure.
• Better-, worse-, omitted-, or differently timed outcomes can change later pursuit relative to expectation.
• Temporal-difference learning compares successive predictions, allowing learning before a final reward appears.
• Cues and intermediate states can acquire value as predictions propagate through a sequence.
• Motivation, craving, procrastination, and goal pursuit can therefore depend on learned context as well as conscious endpoint desire.
• Milestones and whole-task framing can create practical update points, but they do not establish a universal reward schedule.
• Dopamine is an important teaching signal in these sources, not a complete explanation of learning, valuation, motivation, or reinforcement learning.

Evidence

• Successive-prediction mechanism: How Dopamine & Serotonin Shape Decisions, Motivation & Learning | Dr. Reed Montague distinguishes temporal-difference learning from an endpoint-only comparison and uses the rain example to show expectations changing before an outcome.
• Anticipation and cue acquisition: Leverage Dopamine to Overcome Procrastination & Optimize Effort says dopamine can rise before reward and intermediate cues can enter learned pursuit.
• Outcome comparison: Leverage Dopamine to Overcome Procrastination & Optimize Effort and How to Increase Motivation & Drive describe better-, worse-, and omitted-reward updates relative to expectation.
• Goal-design application: The Science of Setting & Achieving Goals connects expectation-sensitive signals to milestone timing and progress assessment.
• Condensed goal provenance: Essentials: How to Set & Achieve Goals repeats that milestone and expectation account as an editorial derivative.
• Habit-sequence application: The Science of Making & Breaking Habits applies reward-prediction language to preparation, effort, execution, and aftermath.
• Anticipation and schedule boundary: How to Control Your Sense of Pain & Pleasure separates dopamine-linked pursuit from pleasure, while How to Increase Motivation & Drive connects novelty, surprise, omission, and intermittent outcomes to later motivation.

Counterevidence & Qualifications

These are podcast-level explanations rather than a derivation or validation of a full computational model. The supplied notes do not provide neural-recording methods, equations, regional specificity, effect sizes, or evidence for a literal individual subtraction assay, universal milestone schedule, task-envelope technique, or intermittent-reward protocol. Temporal-difference models can organize observations without proving that every relevant neuron or human decision follows one scalar signal. Variable rewards can also support gambling-like compulsion, manipulation, and unstable motivation, and dopamine does not exhaust motivation, learning, addiction, habit formation, or procrastination.

What Changed

• No current judgment changed; the new Essentials edit adds overlapping provenance for the applied endpoint-comparison account.

Related Concepts

• Dopamine Peak-Trough Baseline - baseline and trough context in which reward updates affect motivation.
• Dopamine Wanting Loop / 多巴胺渴爱循环 - wanting and pursuit branch shaped by learned cues.
• Motivation Reward-Effort Calculation - action-selection frame that uses expected reward and prior outcomes.
• Attention Capacity Selection - attention branch because salient cues compete for control.
• Effort As Reward - motivation-design branch where effort can become part of the learned reward signal.
• Habit Automaticity and Task Bracketing - habit-sequence branch to which the reward envelope is applied.