Replication Crisis

Definition

The replication crisis is the research credibility problem in which published findings fail under code reruns, data inspection, close experimental repetition, or reasonable alternative analyses.

Current Synthesis

The Planet Money source supplies a social-science auditing model: P-Hacking, Publication Bias, weak replication packages, and untested Robustness Checks can emerge from ordinary publication incentives without fraud. The Huberman-Bhattacharya interview adds a biomedical and laboratory layer, where small procedural differences, weak rewards for exact repetition, unpublished negative results, and volume-based careers make reliability difficult to assess. Together they shift repair from exhortation toward funded replication, inspectable materials, publication infrastructure, author response, and career credit.

Key Claims

• Replication failure can mean code does not run, data does not match, variables are undocumented, or conclusions disappear under reasonable alternative specifications.
• The crisis is not limited to psychology; the episode treats economics and wider social science as affected.
• Preregistration and data/code disclosure reduce some ambiguity but need actual inspection to matter.
• Post-publication discovery is useful but still signals that peer review and journal processes missed something important.
• The repair path combines Scientific Self-Correction with Research Integrity Incentives that change the expected behavior before publication.
• Biomedical replication may require close procedural knowledge and substantial funding rather than code reruns alone.
• Replication becomes more sustainable when it is a viable career activity and negative results remain searchable.

Evidence

• Social-science failure modes: Don’t hate the replicator, hate the game distinguishes rerunning code from testing robustness and documents missing variables, weak documentation, and specification disputes.
• Credible inspection: Don’t hate the replicator, hate the game argues that researchers change behavior when independent teams may actually inspect their work.
• Biomedical variation: Improving Science & Restoring Trust in Public Health | Dr. Jay Bhattacharya has Huberman and Bhattacharya discuss how small procedural differences and tacit lab details can make exact reproduction difficult.
• Institutional repair: Improving Science & Restoring Trust in Public Health | Dr. Jay Bhattacharya records Bhattacharya’s proposals for large replication grants, a searchable venue for replication and negative results, and credit for data sharing and cooperation.

Counterevidence & Qualifications

A failed replication does not automatically prove fraud or show that the original result is false; code errors, undocumented choices, population differences, procedural variation, statistical power, and legitimate specification disputes can produce different outcomes. The NIH reforms in the Huberman interview are proposals rather than measured evidence that a grant or journal program will solve the crisis.

What Changed

• Added biomedical and laboratory replication constraints beside social-science code and robustness auditing.
• Added career funding, searchable negative results, and cooperation incentives as proposed institutional repairs.
• Migrated the page to synthesis-v1 without dropping its original evidence source.

Related Concepts

• Research Integrity Incentives - career and inspection incentives that shape reproducible behavior.
• Negative Results As Scientific Data - failed or null findings needed for a less selective record.
• Publication Bias - selection mechanism that hides null or failed results.
• Scientific Self-Correction - broader correction process that replication operationalizes.
• Replication Package - data-and-code bundle supporting computational reproduction.
• Crowdsourced Academic Auditing - distributed checking model used by the Replication Games.
• Medical Dogma Trust Repair - public-trust consequence when evidence claims exceed their reliability.