Pollsters have been reporting that the public are most concerned about the economy. Are they asking the right questions? (Illustration Credit: London Shearer Allen)

Well, the economy must be the priority because that’s what all the pollsters are certainly telling us these days—repeated by countless pundits and politicians.

So said Jeffrey Jones in Gallup’s September 21 press release, “Economy, Jobs Easily Top Problems in Americans’ Minds.”

It’s not for me to deny that the recovery of the real-world economy in the USA continues to limp along, sluggishly at best, by most objective indicators.

Yet my beef is not with economists, market and policy analysts, but rather with the pollsters who ask respondents fuzzy questions about “the most important problem facing the country” and the like.

Take the latest CBS News/New York Times poll asking Americans their standard question about national priorities, which they typically pose, like Gallup, in a “free answer” open-ended form.  Not surprisingly, the results confirm, what seems to be, the absurdly obvious –the great majority (60%) saying it’s the economy or jobs, stupid!

How could it be otherwise, some respondents might wonder? “Why would anyone ask me such a dumb question?  Haven’t you pollster people been reading, watching, or listening to the news? Have you been living in a cave the last few years?”

The responses above from CBS News/New York Times poll asking the open-ended  question:  “What do you think is the most important problem facing this country today?” (Source: PollingReport)

But, when combined, the overall results look pretty much the same as those reported by CBS/ New York Times, with a huge majority (61%) saying it’s the economy or jobs?Gallup asked the exact same question as CBS/New York Times, at almost the exact same time (September 13-16). They categorized the answers respondents volunteered somewhat more finely, separating jobs and unemployment from the economy in general.

Or is it?

Are all these questions and numbers possibly telling us something else about how respondents try to answer media poll questions?

The responses above are from USA Today/Gallup poll asking “What do you think is the most important problem facing this country today?” (Source: PollingReport)

The CNN/Opinion Research poll poses their national priorities, or most important issue question somewhat differently, giving respondents a multiple-choice list of more or less plausible answers to the question: “Which of the following is the most important issue facing the country today?”Our old, broken Skeptic’s record (like a 78 rpm LP) keeps telling us: a lot depends on how you ask the question.

Once more, the “economy”—whatever that means—turned out to be the biggest issue, but noticeably less so when compared with the results of both the CBS/New York Times and Gallup polls (49% vs. 60% and 61%) all done roughly in the same period. Why?

Clearly because the question is framed and worded differently with respondents explicitly given other plausible answers to the question that draw them away from the cliché that it must be the economy.

After all, that’s what’s in the news all the time these days. Not only that, far fewer respondents in the CNN/ORC poll thought they were “unsure” about what was important or felt they had to volunteer “other” responses—again because they were explicitly offered a bunch of plausible alternatives that sound important or credible as “issues facing the country.”

The responses above from CNN/ORC poll asking “Which of the following is the most important issue facing the county today”

Or are they just regurgitating what strikes them, at the moment, as a plausible, correct answer to the pollster’s quiz question—plausible largely because of what the mass media are telling them are important matters they should be concerned about, regardless of what, if anything, it may have to do with their day-to-day lives?But are these “issues” or “problems” really all that important, personally, to most respondents?

The fancy theoretical label for all this in academia is “the agenda-setting function of the mass media.”

I prefer to call it “Figuring out How to Pass the Pollster Test.”

But the public’s mindset on this might be a whole lot different than the pollster’s – “Say…that’s one hell of a question you asked me. How am I, Joe or Jennifer Six-Pack or working stiff, mom, dad, supposed to know what’s the most important issue or problem facing the country? Who do you think I am: Barack Obama or the President of Harvard?”

In fact, at a bare minimum, media pollsters should be asking citizens much more relevant things such as “How important is this issue/problem [you mentioned] to you personally, or to your family: extremely important, somewhat important, of little importance, or not that important at all? And what exactly do you mean, when you say it’s ‘the economy” or whatever?”

That’s what pollsters, policymakers, and politicans need to know.

First things first:

Suggestions for new questions for media pollsters to ask citizens:  “Does this issue or problem matter all that much to you, personally? Or are you just making yourself look good by giving me what you think is the right answer to my important-sounding BIG question? And, by the way, what actually is the most important problem facing you and/or your family these days?”

The people in Washington D.C. might really like to know something the pollsters aren’t telling them.

George F. Bishop is Professor of Political Science and Director of the Graduate Certificate Program in Public Opinion & Survey Research at the University of Cincinnati. His most recent book, The Illusion of Public Opinion: Fact and Artifact in American Public Opinion Polls (Rowman & Littlefield, 2005) was included in Choice Magazine’s list of outstanding academic titles for 2005 (January 2006 issue).

David W. Moore is a Senior Fellow with the Carsey Institute at the University of New Hampshire. He is a former Vice President of the Gallup Organization and was a senior editor with the Gallup Poll for thirteen years. He is author of  The Opinion Makers: An Insider Exposes the Truth Behind the Polls (Beacon, 2008; trade paperback edition, 2009). Publishers’ Weekly refers to it as a “succinct and damning critique…Keen and witty throughout.” >