A new PollCheck by iMediaEthics finds more disengaged public on the government’s efforts to save the auto industry from what was reported by either Gallup or Pew in February.
While those polls suggested that about 95% of the public was sufficiently engaged in the issue to have a meaningful opinion, the iMediaEthics poll finds that about half or more of the public is disengaged, and that among those with an opinion, views are almost evenly divided.
The PollCheck also finds that question-wording differences – referring to the Federal Government’s efforts as a “bailout” versus giving “loans” to Chrysler and General Motors – are much less likely to affect the responses of people with strong opinions than people with weak opinions or people pressured to come up with opinions by the interviewers.
(The iMediaEthics PollCheck was sponsored by the Golkin Foundation and was fielded by SurveyUSA, March 16-19, 2012. It included a sample of 849 adults, which was designed to represent adults nationwide. See here for more details.)
Pew Poll versus iMediaEthics Poll
While both Pew and Gallup used “forced-choice” questions to measure opinion, iMediaEthics explicitly allowed respondents to indicate they had no opinion, and then followed up with a question to determine whether those expressing an opinion felt strongly about it. To differentiate the whim of the public from will of the public, iMediaEthics classifies people who do not feel strongly about their views as not having a “meaningful” opinion. Previous research has shown that people who don’t feel strongly about a proposal are also likely to say they wouldn’t care whether or not their views prevailed.
The forced-choice format used by Pew and Gallup, by contrast, pressures respondents to come up with opinions even if they have never thought much about an issue, thus giving the illusion that well over 90% of the people have meaningful opinions.
In the iMediaEthics poll, half the sample was asked the Gallup questions (referring to a government “bailout of the auto industry,” and the other half was asked the Pew question (referring to government “loans” to Chrysler and General Motors).
A comparison of the results between iMediaEthics and Pew is shown in the figure above. While both polls referred to government “loans” to Chrysler and General Motors, the iMediaEthics poll differentiated meaningful opinion from non-opinion, in contrast to the Pew poll which pressured respondents to express a view.
Note the iMediaEthics poll shows that about six in ten Americans (59%) do not have a meaningful opinion on whether the loans to Chrysler and GM were good or bad for the economy. By contrast, Pew shows only 6% without a meaningful opinion.
The comparison becomes even more significant in the distribution of opinion: Pew shows a substantial majority with a positive view of the loans (56% to 38%), while iMediaEthics shows the opinionated public evenly divided (20% to 21%).
Gallup Poll versus iMediaEthics Poll
The comparison between Gallup and iMediaEthics shows a somewhat different pattern (see Figure 2). For both questions, respondents were asked if they approved or disapproved of the government’s “financial bailout for U.S. automakers.” But, again, the iMediaEthics poll explicitly measured non-opinion and strength of opinion, while Gallup used the forced-choice format.
This comparison above shows that using “bailout” in the question elicits more negative than positive responses, whether among all respondents or among respondents with just a meaningful opinion – a 7-point margin in each case.
But the percentage of people without an opinion is much lower with Gallup’s forced-choice format (just 5%) than with iMediaEthic’s format (47%), which differentiates non-opinion from meaningful opinion.
Replicating the Gallup and Pew Polls
Given the large differences between the iMediaEthics results and the results from Gallup and Pew, one might argue that the reasons for these differences could be attributed to a different timing (iMediaEthics poll was in March, the other two in February), or to the different mode of interviewing (Gallup and Pew poll used live interviewers for all interviews, while SurveyUSA – which fielded the iMediaEthics poll – used live interviewers only for the cell phone interviews, and used automated voice recordings for the landline interviews).
To address these possible alternative explanations, the iMediaEthics poll included a follow-up question to those respondents who indicated they did not have an opinion – asking if they leaned to one option or another. This was an effort to pressure respondents into selecting an offered opinion, the same way that the original forced-choice questions by Pew and Gallup pressured their respondents.
The exact question wordings are shown in the table below:
The results suggest that the iMediaEthics poll was able to replicate the Gallup and Pew findings pretty closely. As shown in Figure 3, when the iMediaEthics poll pressured people without opinions to come up with one on the question of government “bailouts,” it found 48% who disapproved and 44% who approved, very close to the 51% to 44% that Gallup reported.
The iMediaEthics poll also replicated the pattern of Pew’s results, with both polls showing more people responding that the loans to Chrysler and GM were “mostly good” for the economy rather than “mostly bad.” Pew’s positive margin is somewhat larger than the one found by the iMediaEthics poll. One reason may be that Pew’s question on the loans followed several questions on other loans the government had made at the time, including one question that asked if the recipients had paid back any loans. That question may have triggered people’s memories of news stories that both GM and Chrysler had paid back all or most of the loans they received.
Despite these minor differences, when the iMediaEthics poll pressured respondents to come up with a position – as did Gallup and Pew – iMediaEthics produced the same conflicting pattern of results: More people positive than negative about the “loans,” but more people negative than positive about the “bailout.”
These results give us confidence that public opinion at the time of the Pew and Gallup polls was essentially the same as when iMediaEthics conducted its poll. Moreover, the mode of interviewing had little effect on the results.
What Is A Realistic Picture of Public Opinion?
As these results make clear, questions that “force” respondents to come up with an opinion will vastly overestimate what the public at large is thinking. Respondents in a sample are supposed to represent the general public, but the general public has not been forced to express an opinion – as were the respondents in the Pew and Gallup polls.
Thus, once respondents have been pressured to give an opinion, the Pew and Gallup samples of respondents no longer represent the larger population of Americans across the country.
The idea that many respondents might not have really formulated an opinion on the issue is illustrated by the findings of a question iMediaEthics asked about how closely respondents had been following the issue.
As shown in the table above, only about half the public (49%) indicated they had been following the issue extremely or very closely, which would suggest that a large number of people had probably not come to a judgment on the matter – rather than just the 5% or 6% as reported by Gallup and Pew respectively.
When we asked respondents for their view of the “bailout” or the “loans,” and gave them the opportunity to say they were “unsure,” 23% had no opinion on the bailout question, and 27% had no opinion on the loan question.
We then asked respondents who had expressed an opinion how strongly they felt about it. Experience has shown that many people without strong convictions will nevertheless express an opinion, even if given the opportunity to say they don’t have one. [1] In the bicycle lane poll in New York City, for example, iMediaEthics found that among people expressing an opinion, more than a third then immediately admitted that they would not be upset if the opposite happened to what their opinion was. Though they chose one of the two options offered in the poll, they said they wouldn’t mind if either option were followed.
In this poll, instead of asking people how upset they would be if the opposite happened, we asked how strongly people felt about their opinion. Previous research has shown that people who don’t feel very strongly about their position are also likely to say they wouldn’t be upset if their opinion did not prevail. [2]
Reality versus Myth
So, what is a realistic view of what the public is thinking about the Federal Government’s support for the auto industry? Do we really believe that about 95% of Americans have a meaningful opinion on the issue? And if so, how do we reconcile Gallup’s and Pew’s contradictory findings – Gallup showing a majority with a negative view or Pew showing an even larger majority with a positive view?
Using either the bailout question or the loan question, the iMediaEthics poll comes to similar conclusions about the state of public opinion. By taking into account non-opinion, iMediaEthics shows that anywhere from five to six Americans in ten are disengaged on the issue, while the opinionated public is about evenly divided, or slightly more negative than positive.
Either set of results is much more likely to reflect the actual state of public opinion than the manufactured results of Pew and Gallup – who continue to give us false reassurances with the myth of a highly engaged, fully informed, and opinionated public.
Read the methodology for iMediaEthics’ poll here.
This Poll is part of iMediaEthics’ PollCheck project to fact check media polls.
More iMediaEthics polls found here
FOOTNOTES
1. David Moore, The Opinion Makers
2. David W. Moore and Andrew E. Smith, ”Measuring Intensity of Opinion,” paper prepared for the annual meeting of the American Association for Public Opinion Research, Chicago, Illinois, May 13, 2010.