(Credit: AFP/YouTube, screenshot)
The issue of abortion has been around so long, you might think that people’s opinions are firmly set by now. But recent polls suggest that even on this issue, a slight change in question wording can produce a big change in what “public opinion” is measured.
The basic question being analyzed here is whether people think abortion should be severely restricted or generally available. Two recent polls come to quite different conclusions on this matter: CNN reports that 36% of Americans believe abortion should be legal in all or most circumstances, while 62% believe abortion should be legal only in a few circumstances or not at all.
By contrast, the Quinnipiac poll reports a clear majority (56%) saying abortion should be legal in most or all cases, while 39% think it should be illegal in most or all cases.
These poll differences represent a tsunami turn-around – from a 26-point margin against abortion being available in most or all cases, to an 18-point margin in favor: a 44-point swing caused by slightly different question wording.
Question Wording
CNN uses two questions to measure support for abortion:
CNN: “Do you think abortion should be legal under any circumstances, legal only under certain circumstances, or illegal in all circumstances?”
(Respondents who choose “legal only under certain circumstances” are then asked:) “Do you think abortion should be legal in most circumstances or only a few circumstances?”
These two questions produce the following results:
- Legal under any circumstances (25%)
- Legal under most circumstances (11%)
- Legal in only a few circumstances (42%)
- Illegal in all circumstances (20%)
- No opinion (1%)
By contrast, Quinnipiac asks just one question:
Quinnipiac: “Do you think abortion should be legal in all cases, legal in most cases, illegal in most cases or illegal in all cases?”
Quinnipiac’s final categories are somewhat different from CNN’s, though they mean the same thing:
Legal in all cases (23%)
Legal in most cases (33%)
Illegal in most cases (24%)
Illegal in all cases (14%)
No opinion (6%)
The graphed results make clear that the biggest distinction in the results comes in the middle two categories:
While the percentages saying “always legal” are quite close (CNN: 25%; Quinnipiac: 24%), and the percentages saying “always illegal” are fairly close (CNN: 20%; Quinnipiac: 14%), the percentages in the middle two categories show major differences.
Which is the better way to ask the question?
The Quinnipiac version seems to be the simpler version. It gives all response categories to the respondent in just the one question, and the positive and negative categories are symmetrical: legal in all or most cases, or illegal in all or most cases.
CNN’s version is not as straightforward, because the categories are not symmetrical, and they are not presented to the respondent at once, but serially. There are three answers that say abortion should be legal (always, mostly and rarely), and one that says illegal (always).
Still, the fundamental problem with both questions is that they are quite vague. What are “most” circumstances or cases? What are “only a few circumstances”? Since the question itself does not specify what is meant by those phrases, many respondents will take their cues from the way the question itself is phrased.
Objectively speaking, one might say that “legal in only a few circumstances” is practically the same sentiment is “illegal in most cases.” Both phrases imply relatively few abortions. But apparently many people are more willing to say abortion should be legal, even if rarely, than they are to say it should be illegal most of the time.
Whenever small changes in question wording produce large changes in poll results, it’s a sure bet that many people really don’t have strong opinions on the matter. Both pollsters indicate that very few people have no opinion: CNN, 1%; Quinnipiac, 6%. But these figures, of course, are completely unrealistic, obtained by pressuring those without an opinion to express one anyway.
The net result: Many people pressured to give an opinion were influenced by the structure of the questions they were asked. Different structures, different opinions. And, as it has happened many times before, the pollsters produced results that reaffirm how untrustworthy their polls often are.
