We try to fix election polling (update)

概览

Election polling depends on representative samples, but reaching a genuinely random cross-section of Americans has become increasingly difficult. Fewer people answer unfamiliar calls, some demographic groups respond at especially low rates, and restrictions on automated cell-phone dialing have sharply increased the cost of telephone surveys.

The episode traces three landmark polling failures—from the 1936 Literary Digest poll and the 1948 “Dewey Defeats Truman” error to the misses surrounding the 2016 election—to show why sample quality matters more than raw sample size and why human-built demographic quotas can introduce bias.

At Marist College, the hosts experience the modern response-rate crisis firsthand and examine three possible responses: moving surveys to text messages, statistically weighting underrepresented groups, and asking “wisdom of crowds” questions about who respondents expect to win or how their social circles will vote.

Marist’s conventional 2022 polling proved highly accurate, while the experimental crowd-wisdom questions pushed the estimate too far toward Republicans. The questions remain in use, however, as researchers explore whether they can reveal voter turnout, bandwagon effects, or underdog effects rather than simply predict the winner.

分段落总结

[00:30] Why Polling Has Become So Difficult

[事实] Planet Money revisits a 2022 experiment conducted with the Marist College Polling Center and adds an update on how its proposed polling method is still being used.

[事实] Pollsters struggle because many people either refuse to complete surveys or do not answer unfamiliar calls at all.

[事实] The episode argues that this problem extends beyond elections because measures such as unemployment and inflation also rely on survey research.

[推测] Declining participation threatens not only individual forecasts but public confidence in a much broader system of economic and social measurement.

[01:17] Training People to “Smile While You Dial”

[事实] Marist trainer Daniela Charter teaches interviewers phrases such as “Let’s try a few and see how it goes,” “Every opinion matters,” and “There are no right or wrong answers.”

[事实] Her most important advice is to smile while dialing so that the interviewer sounds upbeat and approachable.

[事实] These conversational techniques can reduce resistance, but they do not work every time.

[推测] The training illustrates how survey accuracy now depends partly on an interviewer’s ability to persuade people to participate, not just on statistical design.

[06:38] How Pollsters Work

[事实] Marist’s calling operation generally runs from around noon until midnight because respondents are more likely to be home in the evening.

[事实] Marist Poll director Barbara Carvalho oversees the questions, methodology, and statistical work behind its surveys.

[事实] Carvalho uses three famous polling failures to explain how modern polling methods developed.

[07:48] The 1936 Failure: A Huge but Biased Sample

[事实] The Literary Digest surveyed millions of subscribers, homeowners, and car owners and predicted that Alf Landon would defeat Franklin D. Roosevelt.

[事实] George Gallup surveyed far fewer people but predicted Roosevelt’s victory; Roosevelt ultimately won all but two states.

[事实] The magazine’s respondents were wealthier than average and did not represent the American electorate.

[事实] The central lesson was that the composition of a sample matters more than its sheer size.

[10:55] The 1948 Failure and the Rise of Random Sampling

[事实] Gallup and other pollsters incorrectly predicted that Thomas Dewey would defeat Harry Truman in 1948.

[事实] Interviewers were instructed to fill demographic quotas manually, but finding the prescribed mix of people was difficult and some interviewers cut corners.

[事实] Polling subsequently moved toward random selection, which reduces the bias introduced when interviewers personally choose respondents.

[事实] Even a relatively small random sample can produce useful generalizations about a larger population.

[13:26] The Telephone Polling Golden Age

[事实] By the 1970s and 1980s, nearly every American household had a telephone, allowing pollsters to select numbers randomly and approach a representative sample.

[事实] Computer-assisted dialing made telephone polling still more efficient despite later complications such as caller ID and the spread of cell phones.

[事实] Carvalho recalls that Marist’s 2014 polling matched the election results almost to the decimal point.

[推测] That success represented an unusually favorable alignment between technology, public willingness to answer calls, and statistical sampling methods.

[15:23] The Post-2016 Polling Crisis

[事实] Many polls and poll-based forecasts expected Hillary Clinton to win the 2016 presidential election, and further polling misses followed in 2018 and 2020.

[事实] Telephone-poll response rates fell below roughly one in twenty, making random representative samples far harder to obtain.

[事实] A 2015 FCC rule prevented pollsters from automatically dialing cell phones, forcing Marist to dial them manually and, according to Carvalho, tripling costs.

[事实] Rural and younger voters became especially difficult to reach; some analysts also believe Trump supporters are less likely to speak with pollsters.

[推测] If political preference is correlated with willingness to respond, increasing the number of calls alone cannot eliminate the resulting bias.

[18:40] Experiencing Nonresponse Firsthand

[事实] The hosts join Marist students in calling North Carolina voters during a competitive Senate race.

[事实] Jeff completes an interview relatively quickly, while Nick encounters hang-ups, refusals, voicemail systems, and fax machines.

[事实] Nick makes 236 calls before completing a single survey after three hours of dialing.

[事实] He attributes the successful call partly to already smiling when the respondent answered.

[推测] The experience vividly demonstrates the labor and expense hidden behind each completed telephone survey.

[22:09] Texting and Statistical Weighting

[事实] One proposed solution is to contact randomly selected respondents by text rather than requiring them to answer a phone call.

[事实] Another is weighting: pollsters give greater statistical influence to responses from groups that are underrepresented in the sample.

[事实] Pollsters commonly weight results by race and education; after 2016, Marist also paid closer attention to geography.

[事实] Marist has explored psychographic questions, including whether respondents have smoked at least 100 cigarettes, to capture differences that conventional demographics may miss.

[推测] Weighting can correct known imbalances, but its effectiveness depends on identifying the characteristics that actually distinguish respondents from nonrespondents.

[23:37] Testing the Wisdom of Crowds

[事实] Researchers proposed asking who respondents believe will win rather than only asking whom they personally support.

[事实] A second question asks how respondents think their friends, relatives, and coworkers will vote, thereby drawing information from their social networks.

[事实] Marist agreed to add both questions to a pre-midterm poll and analyze whether they could improve polling.

[推测] The approach attempts to make one respondent indirectly represent several observations, though social networks may themselves be politically unrepresentative.

[26:48] The 2022 Results

[事实] Marist estimated that Republican House candidates would lead Democrats nationally by three percentage points in 2022; the final margin was 2.8 points.

[事实] Adding the experimental wisdom-of-crowds questions would have shifted the estimate to approximately an eight-point Republican lead.

[事实] Carvalho characterized the disappointing experiment as part of the scientific process of testing ideas and learning from results.

[推测] In this election, Marist’s established methodology substantially outperformed the experimental questions as a direct forecasting tool.

[27:35] Why Marist Still Uses the Experimental Questions

[事实] Ahead of the 2026 midterms, Marist was still asking wisdom-of-crowds questions.

[事实] In some traditionally Republican states, direct polling showed a slight Democratic lean while respondents overwhelmingly expected Republicans to take office.

[事实] The episode says the election results would reveal whether Republicans were being underrepresented in those competitive-state polls.

[事实] Marist is also studying whether these questions can predict turnout, including possible bandwagon or underdog effects.

[推测] The questions may prove more valuable as measures of political expectations and motivation than as standalone predictions of election winners.

[29:23] Predicting Turnout and the Public Value of Polls

[事实] Factors that predict turnout change over time; in the current cycle, views of President Trump and the strength of party identification appeared especially informative.

[事实] Directly asking whether someone will vote can be unreliable because social pressure encourages respondents to say yes.

[事实] Pollsters therefore ask indirect questions, such as whether a person has a voting plan or how they feel about the election.

[事实] Carvalho says public polls reveal information campaigns may not release, counter political spin, influence policy, and show Americans what their fellow citizens think.

[推测] The episode ultimately presents polling as a public-information institution whose importance goes well beyond forecasting winners.

播客点评/总结

[推测] The episode’s strongest feature is its combination of historical explanation and firsthand reporting. The hosts’ calling-room experiment makes nonresponse bias tangible, while the earlier polling failures clarify why representativeness—not merely sample size—is the discipline’s central concern.

[推测] Its treatment of the experimental crowd-wisdom questions is notably candid: the method performed poorly in 2022, yet the program explains why an unsuccessful forecast can still generate useful research questions about turnout and voter psychology.

[推测] The discussion offers less detail about the mathematical assumptions and potential failure modes of weighting, text-based sampling, and psychographic adjustment. Listeners seeking a technical comparison of these methods would need additional material.

[推测] The episode is especially suitable for listeners interested in elections, statistics, survey methodology, or the economics of producing public information. Its broader value lies in showing that polls are uncertain instruments that must continually adapt rather than simple vote-counting machines.