A recent “Dear Abby” column caused a minor stir among some members of AAPOR (American Association for Public Opinion Research), when it expressed little sympathy for a poll interviewer complaining about discourteous rejections. The writer said that when calling potential respondents, “I have been cursed at and called names you can’t print in your column…a little courtesy would go a long way.”
But Dear Abby wasn’t persuaded that the problem was with the impolite recipients of the calls. While she sympathized with the writer and didn’t condone “poor manners,” she pointed out that when companies make their “incessant” calls, “they are entering people’s homes without being invited,” which can make people angry, especially if interrupted during some important activity (like eating and sleeping).
Then Dear Abby really got pollsters’ dander up when she wrote that the people might be less hostile “if they hadn’t been called repeatedly…after they had refused four, five, or six times and had asked not to be called again. They might be more polite if they hadn’t registered on a ‘Do Not Call’ list.”
The response on AAPORNET (the listserv for AAPOR) was immediate, with one member calling on AAPOR to respond to Dear Abby’s comments which were “both inaccurate and…damaging to our profession.”
Others agreed that a response was called for to provide her “a little enlightenment,” especially about the Do Not Call list. (The law expressly exempts public opinion polls from the DNC list, which was designed mainly to limit telemarketing calls.)
Even others suggested that pollsters typically do not call multiple times, though – it turns out – there is a procedure for “converting refusals.” This means that people who initially refuse to participate are called back, usually just once, though in some cases multiple times – just as Dear Abby noted.
But another member pointed out that reminding the world that pollsters are not covered by the DNC list and that we can “invade your home whenever we want to” is not, perhaps, the ideal response to enhance the polling industry’s reputation.
Moreover, the persistent effort to get people to respond to a poll, even after they have said no on multiple occasions, is not something the polling industry wants to advertise.
Why Call Back People Who Have Refused?
In the 1970s and 1980s, when telephone polling first began to permeate American society (because of advances in the telephone industry), most people answered their phones and most people called by pollsters were willing to participate in the survey. The response rate was reportedly in the 80% to 90% range. This meant that only about 10% to 20% of called respondents would refuse to participate or could not be reached after multiple calls (usually because they were not at home).
Then came a geometric expansion of telephone polling, for research and for marketing, along with caller ID. The expansion of polling made the process more objectionable, while caller ID gave the potential respondent more control over which numbers to answer.
The net result is that these days, the response rate of good polling firms is in the 20% range, and for many media pollsters – who conduct three-day polls and thus have little time for call-backs to people who don’t answer the first time – the response rate is even lower.
This means that 80% to 90% of the people we call either refuse directly (after they answer their phones) or they refuse indirectly (by not picking up their phones in the first place). Of course, some of the ‘no answers’ are caused by people who are not at home, but the effect is the same: Pollsters are able to reach only a small minority of people whom they call.
The persistent question, then, is how representative of the larger population are the 10% to 20% of the population who are willing to be interviewed? With a 90% response rate, we’re not too concerned if the 10% we can’t interview are different from the 90%, because even if the 10% were included, they wouldn’t have much impact overall.
However, with a 10% response rate, we are very concerned that the 90% might have different views from the people we can interview, because the 90% would obviously have a big impact on our results overall if they were included.
Thus, the prevailing philosophy in the polling industry is that it’s important to obtain as high a response rate as possible, in order to minimize the likelihood of distorted polling results.
In fact, some government projects require the pollster to reach response rates above the 50% level. The only way that can happen, of course, is to call back the same numbers again and again and again…to the point where, understandably, the recipients of the calls get angry.
So intent on getting higher response rates are some pollsters that they will even call back people who have already refused, sometimes using incentives, other times just trying to use sweet talk (emphasizing how important the person’s opinion is, for example), to persuade the potential respondent to participate.
Which brings us back to Dear Abby’s charge against pollsters. Her main point is that many people are justifiably angry at the intrusion of telephone polling, a point the polling industry already recognizes and which causes it a great deal of concern. Polling results are increasingly based on a smaller and smaller percentage of people willing to answer questions. At some point, if not already, we may lose confidence in the validity of polls altogether.