Why do pollsters, like Andrew Kohut, do early polls even though they know the polls provide the public with useless information?
The Sunday before the election, the Director of Pew Research, Andrew Kohut, admitted on NPR’s “All Things Considered” what we pollsters have known all along, but have been able to keep out of the general public’s consciousness.
It was the polling equivalent of the little child saying out loud, The emperor has no clothes! Except that it was a pollster admitting this truth.
When NPR’s Andrea Seabrook asked Kohut why the polls had shown such divergent readings throughout the campaign season, Kohut admitted that pre-election polls simply couldn’t be trusted – at least not until just before people actually started to vote!
LISTEN HERE ON NPR’S “ALL THINGS CONSIDERED”
Andrea Seabrook: “Why are the polls – the polls seem to be converging across the polls towards the sort of five-point lead for Obama, when they had been pretty disparate – there had been a lot of difference between them?” Andrew Kohut: “The answer to that is that the closer we get to the election, the more crystallized public opinion is and therefore the more likely it is when we make a telephone call, we’re going to get the same answers as another polling organization will get. These voters have made up their minds and there’s not so many of ‘I’ll vote for this guy one day and if you call me tomorrow I’ll vote for someone else.’ So, it’s pretty typical that the polls – rigorous polls – all come together in the final weeks.” |
It was especially fortuitous for Pew that the polls all came together toward Election Day, for as Kohut acknowledged in the interview, only a few days earlier the Pew poll had shown Obama with a “boxcar kind of lead” – up by 15 points over McCain, a real outlier. That wouldn’t have looked good.
Wildly Disparate Polls Converge Just Before Election Day
Some of the other polls were showing a much closer race and had been for many days. But just in time for the election, the Pew poll suddenly bounced into line. Other polls that had been off the mark also mysteriously bounced in the same direction.
The veteran pollster, a former president of the Gallup Organization, where he first cut his teeth on the art of polling, was surprisingly candid about the limitations of polls during the election campaign.
Early on they measure voter responses that, he confessed to Ms. Seabrook, were not really “crystallized” views – instead, he implied, they were top-of-mind responses by many voters who hadn’t given a nanosecond’s worth of thought to the election, and had no idea who they might support (“I’ll vote for this guy one day and if you call me tomorrow I’ll vote for someone else”), but were treated by pollsters and the news media as the Holy Grail.
Early Polls Measure Babble
I realize this may come as a shock to most consumers, who obsess with poll results throughout the campaign, thinking that each new poll reveals a new twist or turn in the election campaign, rather than some new random movement caused by pollsters who measure “uncrystallized” opinions.
 Dan Balz |
What the consumers don’t realize is that even we pollsters are skeptical about those fluctuating (and often conflicting) numbers, because we know how flaky they are before Election Day.
It’s not as though pollsters have been completely secretive about this matter, though. Even before Kohut’s admission on NPR, some pollsters have at least alluded to the problem.
The Washington Post pollsters, Dan Balz and Jon Cohen, tried to warn us about the polls even before the primary election season, admitting that the “early poll results largely reflect name identification…[and] as a result, hypothetical matchups are often poor predictors of what will happen once the primary and caucus season arrives.”
 Kathy Frankovic |
Earlier, CBS’s polling director, Kathy Frankovic, also tried to warn consumers that the lesson she had learned from past experience was “not to put too much trust in national polls as predictors of primary outcomes.”
Having just warned consumers that the early primary polls are…well…close to worthless, the pollsters felt they had met the ethical standards of disclosure – like cigarette manufacturers who post a warning on the packs that you could die from smoking, and then tell you how cigarettes make your life so much happier.
In the ensuing months, all the major national media polls trumpeted their national primary results as though they had real meaning.
Going into the primary season, all the polls told us Rudy Giuliani was the Republican to beat for the presidential nomination, and Hillary Clinton had “solidified” her support among Democrats.
By the time of the first statewide contest in Iowa, the pundits and journalists were already over-analyzing what a Giuliani-Clinton race would entail, conveniently ignoring the warning that national primary polls can be hazardous to their health.
So, it was a shock when Hillary Clinton’s “solid” lead suddenly disappeared after she lost in Iowa, and ultimately Giuliani didn’t win even one delegate to the Republican National Convention. But it was exciting as well. How boring it would have been to know ahead of time that these candidates could lose.
Gallup’s Ghost
It would be too much to expect journalists to wait until just before election day, when – as Kohut warns us – we can finally trust the national poll results. After all, if polls are no good until then, why not wait 24 hours and get the actual election results?
Of course, if journalists wanted more accurate polling information during the campaign, they could demand that pollsters change their polling methods and distinguish between the crystallized and uncrystallized opinions. This would show early on in the campaign that while some people have made up their minds about whom to support, many others have not.
 George Gallup |
But apparently the ghost of George Gallup won’t let them do that.
Over seven decades ago, in 1935, Gallup first asked Americans who they would support for president if the election were held today, rather than who they would vote for in November of the following year.
This was a deliberate effort to measure voters’ uncrystallized opinions, because he knew that few people really knew then who they would support in next year’s election. Certainly the newspapers wouldn’t have wanted to headline a story, “Most Voters Don’t Know Who They Will Support in Election 18 Months From Now.”
The hypothetical question about who voters would support “today” worked well. It provided a kind of fake “news” that was a lot more interesting than the alternative, and newspapers duly reported the findings.
Pollsters have been following that example ever since.
I don’t mean to dismiss the importance of this type of fake news. It serves journalists and even us news junkies well, giving us daily doses of cud to chew over.
Can you imagine the alternative – early on in the campaign, being reminded every other day that voters aren’t paying attention to the election, and that only 3 percent have made up their minds, 1 percent each for McCain, Obama and Nader?
Instead, the polls give us much more entertaining (though admittedly, unreal) wildly fluctuating results. Often contradicting each other, the vacillating numbers give an illusion that something is real happening right in our midst.
Finding the Polls That Tell You What You Want to Hear
In March of this year, New York Times columnist Paul Krugman wrote what some people thought was a critical comment about the polls: “Polling numbers aren’t much help: for now, at least, you can find polls telling you anything you want to hear.” But I saw that more positively.
What, after all, is wrong with finding polls that tell us what we want to hear?
 Paul Krugman |
And it wasn’t just “for now,” as Krugman said in March. In mid-October, you could still find highly reputable polls telling you anything you might want to hear.
Did you think the presidential race was over? Then you could have looked at the NBC/Wall Street Journal poll, showing Obama with a double digit lead.
Or did you still hope that McCain could win? Then you could have found the Associated Press poll, conducted at the same time, showing a statistical dead heat.
Journalists were also delighted with the array of polls they could cite at the end of the campaign. The most tantalizing was the Pew poll, which heightened our excitement first by showing Obama surging to the largest lead reported by any poll during the campaign, followed by a dramatic “tightening” of the race just before election day.
But journalists could look to other polls if their stories wanted to emphasize how little had changed since the presidential last debate. In the last several weeks of the campaign, ABC/Washington Post and Gallup showed hardly a blip in Obama’s lead.
It’s true that virtually all the major media polls converged just before Election Day, but think of the variety of stories that journalists could write during the campaign with the many different poll results, often reported on the same day.
And it was important that the polls all agreed at the end, because it renewed our faith in the usefulness of polls to tell us what the electorate is thinking. That allows us in the future to search for our favorite poll results with a clear conscience.
Election 2012!
 Larry Hugick, Chairman of Princeton Survey Research Associates International, which does the polling for Newsweek |
If I was momentarily worried that Kohut’s revelation about the meaningless of early polls would hurt the polling industry, it’s because I hadn’t looked at the ThePollingReport, which showed us that even before this past Election Day, both the CNN and Newsweek polls had already measured public opinion about the 2012 election.
Let the entertainment begin – Anew!
Newsweek tells us that 91 percent of Republican voters have already decided whom to support for their party’s presidential nomination (Mitt Romney leads Mike Huckabee who leads Sarah Palin).
CNN tells us that 99 percent of American voters have already decided whether to support Sarah Palin for president (40 percent will, 59 percent will not).
Maybe that’s what you want to hear. If not, just wait – another poll will tell you something different.
Perhaps you believe that well over nine out of ten Americans have already made up their minds about the 2012 election? Or maybe you just like the results? Good luck either way.
For the unreasonable few who demand that polls tell the truth about the electorate before Election Day, I say – No Polls For You!
Polls are for the journalists who like to tell exciting stories. And the consumers who want to believe them.
But there’s a benefit to not following all those polls before the day before Election Day: the lowered tension could add several years to your life.
Think about that the next time you yearn for a poll that tells you what you want to hear.
David W. Moore is a senior fellow with the Carsey Institute at the University of New Hampshire. He is a former Vice President of the Gallup Organization and was a senior editor with the Gallup Poll for thirteen years. His new book is The Opinion Makers: An Insider Exposes the Truth Behind the Polls (Beacon, 2008). Publishers’ Weekly refers to it as a “succinct and damning critique…Keen and witty throughout.”
