In the final days before the election, some major conservative pundits – such as Michael Barone in the Washington Examiner, Dick Morris for Fox News, and Steve Lombardo at the Huffington Post – predicted startling high electoral vote numbers for Romney, asserting that not only would he win, but win big.
These predictions contradicted the consensus of several websites, which average the results of many polls to make predictions, and which all indicated a strong likelihood that Obama would win a landslide number of electoral votes.
Ultimately, of course, the poll doubters were wrong, and the poll aggregators were right on the money.
Still, it’s worth looking at what the major objection to the polls were – namely, that their likely voter models systematically underestimated the size of demographic groups in the electorate that typically vote Republican.
Let’s turn the situation around and assume that the turnout model designed by, say, the Gallup Poll researchers had been the one that most or all pollsters used. Based on the Gallup results, Romney would be the winner. In fact, Gallup was the outlier among all the major polls, consistently showing stronger support for the GOP than what the other polls found.
What was there about the new likely voter model Gallup began using last month that caused it to skew toward Republicans?
In an insightful post, researchers at political science blog, The Monkey Cage, analyzed the composition of the electorate generated by Gallup’s likely voter model and compared it with the electorate revealed by the 2012 exit polls. They showed that Gallup underestimated Democrats by 3 percentage points, and over-estimated Republicans by 4 points – a net difference of 7 percentage points.
Looking specifically at the racial composition of the electorate, the Monkey Cage researchers found that Gallup’s likely voter model included 22% of non-whites, 6 points below the actual electorate. And the model suggested the electorate would include just 12% of young voters (18-29), also an underestimate of 6 percentage points. These are two groups that gave substantial majority support to Obama. Their significant under-representation in the likely voter model explains why Gallup skewed toward Romney.
This type of reasoning is exactly what the poll doubters were suggesting, but in the other direction.
So, it’s always a legitimate question to ask whether the likely voter models are really measuring what the electorate will look like on Election Day. This year, Gallup was wrong. And it’s been skewed before – in 2008, and 2010, for example. In 2008, however, it was skewed in favor of Obama; in 2010, its results skewed in the GOP direction.
How could observers know that this year, Gallup would be skewed in the GOP direction and all the other polls (on average) would be accurate? The opposite argument is the one that poll doubters were actually making – that Gallup was right, and (virtually) everyone else was wrong. Given Gallup’s historic reputation, that argument might not have seemed so outlandish.
But really! Let’s apply a little common sense.
It’s one thing if there were only a couple of polls, like the period of the 1960s and 1970s, when the two major pollsters were Gallup and Lou Harris. How could anyone know which of the two was correct when they disagreed?
But today, with all of the many other polls, is it really reasonable to assume that just one polling organization has a corner on accuracy, while all the others would be wrong? Moreover, as Nate Silver points out, Gallup’s likely voter model has been erratic going back to 1996.
Perhaps, it’s a Wisdom of the (Polling) Crowd phenomenon – while any single poll could be far off the mark, the average of them all gives us some greater confidence in their combined accuracy.
In my view, the poll doubters should have known better. Yes, they were asking the right questions about the legitimacy of the likely voters models. But their answers seemed informed more by their hopes than by empirical evidence. Steve Lombardo, who fearlessly predicted a Romney victory, signaled the problem when he wrote, “More than ever we are making this projection based on gut instincts wrapped around science.” It seems as though the gut instincts prevailed over the science.