Publication Bias

Publication bias is the selection problem in Don’t hate the replicator, hate the game where journals and researchers are more likely to publish novel, statistically significant findings than null, ambiguous, or failed results. In the episode, Abel Brodeur sees this directly when his smoking-ban paper becomes harder to place because the straightforward result is no effect.

The concept matters because it can distort the visible literature even when individual researchers are not fabricating data. If near-miss or null studies remain unpublished while significant-looking findings are polished and submitted, readers see a cleaner and stronger evidence base than the underlying research process produced.

Key Claims

• Publication bias can make the published record overstate effect sizes or certainty.
• Null results are evidence, but the source says they often have weaker publication incentives.
• Bias at the submission and acceptance stage can interact with P-Hacking because researchers learn what kinds of results are rewarded.
• A visible bunching of results near a Statistical Significance Threshold can be a warning sign that the literature has been filtered or tuned.
• Replication Games address publication bias indirectly by increasing the chance that fragile published results will be checked.

Connections

• Abel Brodeur and Replication Crisis - source researcher and broader credibility problem.
• P-Hacking and Statistical Significance Threshold - mechanisms that interact with publication selection.
• Replication Package, Preregistration, and Robustness Checks - safeguards against a polished but selective record.
• Economist Trust Crisis and Expert Trust Repair - credibility consequences.