Scientific polls can be valuable. They can give a great overview, track trends, gauge opinions and reveal national tenor. But what’s the use in an unscientific or even just a frivolous poll—not much.
That’s why it’s so strange that the New York Times published this September 27th article about the new CBS 60 Minutes/Vanity Fair poll without mentioning that the poll was admittedly unscientific and “should not be confused with the results of the scientific polls conducted by CBS News,” according to the fine print only revealed when you click on the “submit” button at 60 Minutes’ poll site.
In fact, it seems like the Times is doing exactly that—discussing this poll as if it were a part of CBS News’ larger, more reputable polling enterprise. “The initial survey, which was conducted by CBS News in August, also found that fully half the people would support a tax of 50 percent or higher on the country’s wealthiest millionaires,” Times reporter Brian Stelter writes.
Beyond millionaires, the poll also supposedly reveals vital information about our nation’s predilection for Walmart and Twitter, and women’s desire to switch lives with Michelle Obama.
“What is especially interesting is that there is no methodology statement – how or when it was conducted!” says David Moore, former Vice President of the Gallup Organization and a senior editor with the Gallup Poll for thirteen years, who iMediaEthics interviewed by email and phone.
Moore is correct. One has to leave the CBS site and the poll itself to find information about the methodology. Vanity Fair notes in fine print disclosure: “NOTE: This survey is based on a random sample of 1,097 respondents nationwide. It was conducted by telephone on August 27–31, 2009, by CBS News. That the poll used on a random sample of 1,097 respondents nationwide.” The minuscule font was found at the end of this discussion and under poll results. However, CBS News has no mention of the project on its main polling website.
According to Moore, the poll is supposed to be based on phone calls and yet absurdly also solicits votes from the readers through a web page “submit” button. Nowhere does it explain how these clicks will be incorporated. Vanity Fair even sends readers over to CBS to cast their “votes”, encouraging readers, “To answer the questions yourself, visit the 60 Minutes homepage at CBSNews.com.” Furthermore, when you land on The 60 Minutes/Vanity Fair Poll page, CBS excitedly instructs : “Check Out The Results And Weigh In!”
“The invitation to express one’s own opinion is, apparently, bogus,” says Moore. “Yes, you express it, but then what? Does anyone even count it? Just another sleazy gimmick…”
CBS may be right when saying that the poll is “topical, fun, amusing, illuminating and most importantly, interesting.” But those qualities don’t make up for the fact that the survey is not scientific, and therefore is infotainment and not actual national opinion.
The New York Times should have flagged the fact that CBS did not feature their admission that the poll is “not a scientific poll” right up front in large type in the introduction. One of the CBS commenters says it well, according to Moore, who wrote by email: “I concur with the comments below”:
JHenryBrown September 27, 2009 8:31 PM EDT:
“This is an amazingly unscientific and biased poll that gives choices apparently designed to either be cute or two-stroke advertisers of the publications. … I cannot believe that you would pass off this poll as something legitimate, and I was especially shocked that the New York Times already had an article about it, when it was just announced tonight on 60 Minutes. I cannot imagine the purpose of this poll, but it is definitely not to provide the public with any usable information.”
You may recall that pollsters Moore and George Bishop were co-authors of last year’s iMediaEthics’s “1st Annual: 2009 TOP TEN “DUBIOUS POLLING” AWARDS.” Word is that this CBS 60 Minutes/Vanity Fair Poll series is already a strong contender for 2010’s Top Ten list.