Polls and commentary about the Tea Party movement and its supporters, their politics, their demographics and psychographics proliferate as I write.
In this column I start with a skeptical look at some of the recent estimates about the percentage of Tea party “supporters” in the USA. Well, like a lot of other topics in public opinion polling, much depends on how you ask the questions and on questions you don’t think to ask. Take the latest Gallup Poll on tea party supporters conducted just a couple weeks or so ago.
Asked point blank: “Do you consider yourself to be [a supporter of the Tea Party movement, an opponent of the Tea Party movement] or neither?” a sizable block (28%) of Americans said they were supporters; 26%, opponents; 38%, neither; and, not surprisingly given the forced-choice form used by Gallup, only 8% volunteered they had “no opinion.” Evidently, Gallup, like a lot of other pollsters, didn’t think it was important enough to find out what, if anything, respondents actually knew or had actually read about the Tea Party movement and its leadership. It’s so much easier, not to mention less expensive, to simply assume respondents know what you’re asking about (after all, it’s been in the news a lot) and force them to choose an answer. It’s certainly a tried and true way to generate one more illusion of an aware and opinionated public.
But Gallup’s not the only one who’s been, wittingly or unwittingly, inflating the figures on how many Tea Party supporters are out there. The pollsters at the Associated Press with GfK Roper Media told us just recently that it’s nearly a third (31%)—that’s right, nearly a third!— in a poll completed just a few days before the national rallies and protests on April 15th. How did they come up with such a stunning figure?
Well, first of all, unlike Gallup and to their credit, they did ask respondents: “How much do you know about the Tea Party movement?”
Only 16% claimed they knew a “great deal” or “a lot” (whatever that meant); about a third (32%) said “some”; and over half (52%) admitted “not too much” or “nothing at all”. But all this self-confessed public ignorance didn’t stop the AP/GfK pollsters from plowing ahead and asking respondents, just the same, whether they had “…a favorable, unfavorable, or neither favorable nor unfavorable opinion of the Tea Party movement?”
Forced to choose, only 8% of the sample volunteered a ‘don’t know” response; the rest picked one of the other news-making response categories. As if that were not enough to get respondents primed for answering the really big question about Tea Party support, AP/GfK interviewers then asked the presumptive, opinion-producing question of the day: “From what you know about the Tea party movement, would you say you generally agree, disagree or neither agree or disagree on the Tea Party movement’s positions on political issues [emphasis mine]?” Given this presumptive question form, just 5% of the respondents volunteered a “don’t know” response even though over half the sample (52%) had admitted just a few minutes earlier in the interview that they didn’t know jack about the Tea Party movement or its positions on issues, teapots or tea leaves. And, of course, none of this stopped AP/GfK’s pollsters from asking the big bottom line question next: “Do you consider yourself a supporter of the Tea Party movement, or are you not a supporter of the Tea Party movement?”
At this point, just 9% of the respondents could say “don’t know” but the rest had no problem coming up with a seemingly considered opinion of the movement, with roughly a third (31%) identifying themselves as “supporters” of the movement. And that’s how you lead the horse to drink the water.
Other pollsters have been more careful in asking their questions about support for the Tea Party movement, but even they leave a slippery slope for many respondents to slide down. To their great survey research credit as well, the much talked about, recent CBS/NYT poll on the movement’s supporters, “Poll Finds Tea Party Backers Wealthier and More Educated” took the trouble to ask respondents, first of all: “How much have you heard or read about the Tea Party movement—a lot, some, not much, or nothing?” Half the respondents (50%) readily acknowledged that they knew “nothing at all” (28%) or “not much” (22%) about the movement, results remarkably like those reported by the recent AP/GfK poll. But unlike their competitor, the CBS/NYT poll also gave respondents the chance to say they were either undecided or did not have an opinion of the movement when they asked: “Is your opinion of the Tea Party movement favorable, not favorable, undecided, or haven’t you heard enough about the Tea Party yet movement to say?”
Asked this way, 60% were either undecided (14%) or had not heard enough yet to say (46%), which is quite a contrast with the mere 8% who volunteered a “don’t know” response when asked a very similar question in the AP/GfK poll. So it should not be surprising that CBS/NYT found significantly fewer supporters (18%) of the movement when they asked their bottom line question of those who said they had heard a lot, some or not much about it: “Do you consider yourself to be a supporter of the Tea Party movement?” Had they left out those who said “not much” (22%), which they probably should have done, the percentage would have been even smaller, more like 12-13% or less, as found by other Tea polls, depending one more monotonous time, on how the question is asked. And if there is so much measurement noise with just this foundational question about how many Tea Party supporters are actually out there, how much stock can we put in the rest of the questions being asked and analyzed about the movement’s essential characteristics? It’s looking more and more like interpreting patterns in tea leaves. Maybe the pollsters should hire more tasseographers?
(Credit: Occultopedia.com)
George Bishop is Professor of Political Science and Director of the Graduate Certificate Program in Public Opinion & Survey Research at the University of Cincinnati. His most recent book, The Illusion of Public Opinion: Fact and Artifact in American Public Opinion Polls (Rowman & Littlefield, 2005) was included in Choice Magazine’s list of outstanding academic titles for 2005 (January 2006 issue).
David W. Moore is Senior Fellow with the Carsey Institute at the University of New Hampshire. He is a former Vice President of the Gallup Organization and was a senior editor with the Gallup Poll for thirteen years. He is author of The Opinion Makers: An Insider Exposes the Truth Behind the Polls (Beacon, 2008; trade paperback edition, 2009). Publishers’ Weekly refers to it as a “succinct and damning critique…Keen and witty throughout.”