We try to fix election polling (update)

Source note Episode guide 2026-10-21 Original audio Topics: Politics

Summary

This Planet Money update uses historical polling failures and a 2022 field experiment with the Marist Poll to explain why sample composition matters more than sample size and why falling response rates threaten representative election surveys. It compares random telephone sampling, text outreach, demographic weighting, and crowd-wisdom questions, finding that Marist’s conventional 2022 national House estimate closely matched the reported result while the experimental questions substantially overstated the Republican lead.

Key Claims

Election polling depends on representative selection, but unfamiliar-call avoidance, unequal response rates, and manual cell-phone dialing make random telephone samples increasingly costly and incomplete.
A large biased sample can perform worse than a smaller representative sample; interviewer-filled demographic quotas can also reintroduce selection bias.
Statistical weighting can correct known demographic imbalances only when pollsters identify characteristics that distinguish respondents from nonrespondents.
Asking respondents who will win or how their social circles will vote did not improve Marist’s 2022 national House estimate, although the questions may still illuminate expectations, turnout, or bandwagon and underdog effects.
Public polling supplies information beyond campaign-controlled claims and therefore matters as civic measurement, not only as election forecasting.

Key Quotes

“The composition of a sample matters more than its sheer size.” — episode summary of the 1936 polling failure

“Let’s try a few and see how it goes.” — interviewer training language intended to reduce survey refusal

Connections

Marist Poll — polling organization whose conventional and experimental methods are compared.
Barbara Carvalho — Marist Poll director explaining methodology, weighting, and the value of failed experiments.
Election Polling Representativeness — central problem of reaching and adjusting a sample that reflects the electorate.
Crowd-Wisdom Polling — experimental method tested against the 2022 result and retained for further research.
Survey Mode Effects — adjacent concern because telephone and text contact modes can alter participation and measurement.
Scientific Sampling Discipline — broader principle that evidence composition and provenance govern inference quality.
Federal Communications Commission — source-attributed regulatory context for manual cell-phone dialing and higher survey costs.
Hillary Clinton and Donald Trump — 2016 election context for the modern polling crisis described by the episode.

Contradictions

No settled contradiction with the existing wiki was found. The 2022 result directly qualifies the idea that crowd-wisdom questions necessarily improve election forecasts: in this test, they performed worse than Marist’s established method.