(Credit: Photo Illustration by London Shearer Allen. Window image by Malcolm Tredinnick via Flickr)
The Arizona Capitol Times reported yesterday that a statewide poll of Republican voters found a two-to-one margin in favor of Governor Jan Brewer vetoing SB1062.
However, a closer examination of both the poll, and the circumstances surrounding the poll, suggest the pollster manufactured a “public opinion” that would give the governor cover and allow her to veto the bill, in order to avoid severe financial repercussions and embarrassment for the state of Arizona. As described by CNN:
“In short, SB1062 would amend the existing Religious Freedom Restoration Act, allowing business owners to deny service to gay and lesbian customers so long as proprietors were acting solely on their religious beliefs.”
The bill, passed by a Republican-controlled House and Senate, has become highly controversial, causing three of its original GOP supporters in the state’s Senate to ask Brewer to veto it. Had they voted against it originally, the bill would have failed. Brewer has until Feb. 28 to veto it, or it becomes law.
Unbalanced and Biased Poll Question
The Arizona Capitol Times report of the poll is technically inaccurate, and the poll itself highly flawed. The reporter, Jim Small, writes:
“In the automated poll of 802 Republicans by Coleman Dahm, a Republican political consulting firm in Phoenix, 57.1 percent of respondents who were asked about the bill said they would like Brewer to veto it. Only 27.6 percent said they want her to sign SB1062. The remaining 15.3 percent had no opinion. The poll has a margin of error of +/- 4 percentage points.”
In fact, the poll question did not ask whether Brewer should veto the bill or sign it. The poll asked only if Brewer should veto it. Research has firmly established that unbalanced questions – questions that present only one side of an issue – are likely to produce inaccurate results.*
HuffPost Pollster reports that the wording of the question asked by the polling firm is as follows:
“Should Governor Brewer veto Senate Bill 1062, which redefines and expands the state’s definition of ‘exercise of religion’? This bill allows an individual, business, or corporation the right to refuse service to someone based on that individual, business, or corporation’s religious beliefs.”
The question is unbalanced because it asks only if Brewer should veto the bill, with no alternative given for the veto. The question should have been phrased along this line:
“What action do you think Governor Brewer should take on Senate Bill 1062 – should she sign it or veto it?”
For half the sample, the wording should have been reversed: “Should she veto it or sign it?”
The problem with asking such forced-choice questions (where no explicit “no opinion” option is provided) is that many people who have not given the issue much thought are highly influenced in their answer based on the way the question is phrased.
In this case, with no apparent preamble or question to find out how much people might know about the issue, a question is asked: Should the governor veto this bill? That is a leading question, suggesting it is what the interviewer is looking for. Why ask such a question if the answer isn’t “yes”? In fact, the pollster might have gotten the same pattern of results (but reversed) had the question asked: Should the governor sign the bill?
That’s why, in order to ask a “balanced” question, the wording should have included both options – to sign the bill or to veto the bill.
In addition, if the pollster wanted to be honest about the state of public opinion, the pollster should have added the tag line, “or don’t you have an opinion one way or the other?”
The report showed that 15% volunteered they had no opinion, a fairly high number of people in that category for a forced-choice question. Had the pollster added the suggested tag line, no doubt many more people would have admitted they didn’t have an opinion.
We can’t know how many of Arizona’s GOP voters would have chosen an option for Brewer to “sign” the bill, because that option was never offered. But we can be sure, based on past research, that the percentage would have been higher than the 28% reported by the pollster. This in turn suggests that the “veto” percentage would have been smaller.
What the final ratio of support to opposition might be if the question were worded fairly can only be determined by an objective poll, not one designed to achieve specific results.
As the Arizona Capital Times reported, Bert Coleman, a partner in the Republican firm of Coleman Dahm that conducted the poll, said
“Clearly, the electorate thinks that there are other priorities in Arizona. Anything that would hamper the economic recovery such as discriminatory extremist legislation will not be tolerated by the majority of Republican voters in the state.”
That would be welcomed news for Arizona if it were true. But the poll itself, flawed as it is, does not justify the assertion.
*For one of the earliest set of experiments, see Stanley L. Payne, The Art of Asking Questions (Princeton University Press, 1951), pp. 7-8, cited and discussed in David W. Moore, The Superpollsters: How They Measure and Manipulate Public Opinion in America (Basic Books, Paperback edition, 1995), pp. 327-328.
UPDATE: 8:49PM EST : The Governor of Arizona, Jan Brewer, Vetoes Anti-Gay Bill. Read the NBC News report
