Probabilistic Forecasting Can Mislead Voters About Certainty of Election Outcomes

Leading up to the 2016 U.S. presidential election, probabilistic forecasting — which presents polling data as the percentage likelihood that Candidate A will win over Candidate B — was mentioned an average of 16 times per day in cable news broadcasts. Instead of predicting Hillary Clinton would win 60% of the vote, the probabilistic forecasts said she had a 70-99% chance of winning the election.

Those forecasts led many people to think Clinton was guaranteed to win, which we now know was not the case. As much of the world was asking the question, “How could pollsters have been so wrong?” a group of political scientists asked a different crucial question: “How did these predictions of a landslide Clinton win impact the thinking — and perhaps the eventual choice — of voters?”

Entitled “Projecting Confidence: How the Probabilistic Horse Race Confuses and Demobilizes the Public,” the new study from Annenberg Assistant Professor Yphtach Lelkes and co-authors Sean Westwood (Dartmouth College) and Solomon Messing (Pew Research Center) found that probabilistic forecasts left voters thinking the election was far less competitive than it actually was.

The study is not a critique of probabilistic forecasting in and of itself, but rather the researchers aim to uncover how the public interprets these polls and how people behave based on their interpretations.

Probabilistic forecasting has risen in prominence since 2008, when a poll correctly predicted the winner of nearly every state’s Senate race that year. But Lelkes, Westwood, and Messing found that the numbers offered by probabilistic forecasting leave people with the impression that the election’s outcome is more certain than it really is, and thus the value of an individual’s vote in swaying that outcome is lessened.

While the authors aren’t able to make a definitive link to polling perception and the outcome of the 2016 election, they found that the impression left by probabilistic polling makes people less likely to actually vote when the time comes, which can then potentially affect the outcome of the election. In addition, the researchers suggest that the increased use of probabilistic forecasting in the media may have other impacts on things like campaign donations and mobilization of volunteers.

“The impression left by high-probability forecasts for a Clinton victory in 2016,” write the researchers, “which predicted the probability of a Clinton victory at between 70-99%, despite being based on electoral college vote share projections of 56-60% for Clinton, may not have effectively conveyed the closeness of the race in the electoral college to the public, potentially decreasing turnout.”

The study has been covered by New York Magazine, the Pew Research Center, and the Washington Post.