(Credit: http://nyphotographic.com/)

Recently, a new Politico/Morning Consult poll announced, among other things, that just 7% of voters supported funding for a border wall if it was the only way to end the government shutdown, while 72% were opposed. (Those figures have since been corrected.)

It’s been more than a week since Morning Consult, the organization that actually conducted the poll, acknowledged to me in an email that the above conclusion was wrong and sent me an updated topline and set of crosstabs that made clear what the poll’s results actually were. Yet, Morning Consult refuses to clarify the confusion online.

The questions that cause confusion are POL17_1, POL17_2, POL18_1, and POL18_2. Anyone who looks at them on Morning Consult’s website might well believe that the reported results pertain to the whole American adult population.

But they don’t. The first two questions apply only to respondents who said they favored a border wall between the United States and Mexico, while the second two questions apply only to respondents who oppose such a border wall (based on POL16).

How could any reader know that the four questions are follow-ups to the border wall question? There is no written explanation that would alert the reader to that information.

In fact, Morning Consult’s client, Politico, misinterpreted those very numbers. Politico initially reported the results as though they pertained to the whole population.

It was POL18_1, for example, that showed just 7% of voters supported funding for a border wall if it was the only way to end the government shutdown, while 72% were opposed.

That figure was reported on Politico’s website as pertaining to all voters in the U.S. (it has since been corrected) and was cited by other sites (as of this writing, it is still uncorrected on at least one website).

This was just one finding from many questions asked in the poll, but the conclusion was stunning. By a margin of greater than 10 to 1, voters apparently opposed the president’s demand for a border wall in order to end the government shutdown.

Such an overwhelming consensus by the public would indicate that President Trump’s proposal was widely unacceptable even to most of his own base!

As it turns out, however, the 7% figure was woefully wrong. The poll really showed that about 46% — not just 7% — were willing to fund a border wall in order to end the government shutdown.

What happened?

In question POL16, the poll had asked respondents if they supported or opposed construction of a wall along the United States–Mexican border. Overall, 43% said yes; 49% no.

To gauge the intensity of feeling on the matter, only opponents of the wall were then asked if they would nevertheless support funding for a wall if that was the only way to end the shutdown (question POL18_1).

That is where the 7% figure applies – the percentage of wall opponents who were willing to compromise by funding the wall to end the shutdown (which consisted of 3% of the overall voter population).

The 43% who supported the wall in general were not asked that follow-up question, because presumably they would have all said yes. Added to the 3% just noted, that means that, according to the poll, probably about 46% of voters would support funding for the wall to end the shutdown – almost 7 times greater than the figure initially reported.

Confusion over the follow-up questions and Politico/Morning Consult’s responses

When I first read the results of the four follow-up questions, I did not recognize that they might have been asked of only supporters or opponents of the wall. Like the Politico reporter, I took them at face value – and was confused by their seemingly contradictory results.

I wrote to both Politico and Morning Consult about my concerns. I got an email response back from Morning Consult’s Director of Communications, T. Anthony Patterson, alerting me to the fact that the questions constituted a follow-up to the question about the wall. He included with his email two files that had corrected toplines – the corrections being an introductory phrase to each of the four follow-up questions, indicating that they had been asked only of supporters or opponents of the wall, as appropriate.

Politico did not respond to my query, but later in the day, I checked on Politico’s website with the poll story and noted that their story had been corrected, and that their topline and crosstabs online had been updated to indicate that the four questions were follow-ups to respondents’ views of the border wall.

Also, the revised Politico story included an explicit “Clarification” at the end, in which the author acknowledged the errors in interpretation.

CLARIFICATION: An earlier version of this story mischaracterized several poll responses.

The story originally reported that only 7 percent of voters said that they support dedicating funding to a border wall if it was the only way to end the government shutdown and 72 percent oppose. That question was not asked among all voters, but more narrowly among voters who said they oppose the construction of the border wall.

The 34 percent who said they approve of dedicating funds to border security, but not for a wall, if it was the only way to end the shutdown, were not drawn from the entire sample but rather from among those who support construction of a border wall. The same is true of the 51 percent who oppose dedicating funds.

I also checked in on Morning Consult’s website, but there was no acknowledgement of initial confusion, and more importantly there was no change in the topline/crosstabs posted on the pollster’s website.

I remembered that the initial story on Morning Consult had also made an error in interpretation, but Mr. Patterson assured me that was not the case. I accepted his assurance.

However, I asked him why Morning Consult had not updated its topline/crosstabs. I pointed out that his own email to me contained updated documents with the introductory phrases, but they were not posted online. Instead, the same results that caused Politico to make its misinterpretation, and that also caused me confusion, were still unchanged on the website. (As of this writing, they remain unchanged.)

His response was baffling:

“To confirm, all information/data within the article’s crosstabs you’re referring to are indeed CORRECT. As someone who worked for Pew, I’m sure you can tell that it was a filter question by looking at the N size. The updated crosstabs I shared with you made it clearer that it was a filter question by adding some language to the question text.” 

His argument is that because I “worked for Pew” (in fact, I was a senior editor of the Gallup Poll for 13 years), I should have looked at the reduced sample size and figured out that questions POL17_1 and POL17_2 were asked of supporters of the wall, and questions POL18_1 and POL18_2 were asked of opponents.

Remember, the crosstabs he’s referring to are for general consumption. Even Politico’s reporter was initially misled. So, telling me that as an experienced pollster, I should have been able to figure out that (based on the reduced sample sizes) the four questions were a follow-up to question POL16 sounds more like a childish taunt than a professional response to the issue: Why not clarify the crosstabs for the general reader?

Moreover, just looking at the sample sizes might not alert even the most attentive professional pollster to the fact that the four questions were a follow-up specifically to the question on the border wall. One reasonable take, for example, and consistent with reduced sample sizes, is that the questions were administered to split samples, but in fact pertained to the whole population.

In any case, Morning Consult’s refusal to clarify that POL17_1, POL17_2, POL18_1 and POL18_2 are follow-up questions to POL16 remains a mystery to me. Mr. Patterson sent me the corrected topline and crosstabs. Why not post them online? Politico did.

What is up with Morning Consult?