The Poynter Institute labeled 515 websites “unreliable,” essentially blacklisting the sites as untrustworthy.

Problem is, it appears Poynter didn’t contact the 515 outlets before publishing its list. Poynter also didn’t contact some of the creators of the lists it used to compile the names of the 515 outlets for permission or consultation. At least one of the lists was out-of-date or not maintained. These lapses are surprising given Poynter’s reputation and position as a vaunted journalism education organization.

Despite having unpublished the list and apologized for “weaknesses in the methodology,” the list is still available on the Internet Archive.

iMediaEthics reviewed the Poynter list, which was compiled from five other’s lists. We contacted those list creators and a sample of the outlets on the now-unpublished Poynter blacklist.

What we found: Most of the five list creators didn’t work with or hear from Poynter before Poynter used their lists (one creator did hear from the Columbia Journalism Review/Tow Center last year, but not Poynter; one creator received and answered questions related to the project).

iMediaEthics asked Poynter’s vice president Kelly McBride if Poynter contacted the 515 outlets before putting their names on the list, if and why Poynter didn’t contact all of the list creators beforehand, and if any outlet on the list threatened to sue. McBride hasn’t responded to these questions.

Were the 515 Sites Contacted?

iMediaEthics contacted ten of the 515 sites on the list: the Daily Caller, the Daily Signal, the Free Beacon, News Corp-owned the Australian Daily Telegraph, the India Times, Pakistan’s The Nation, Iran’s Press TV, the UK’s Private Eye magazine, Raw Story and the Washington Examiner. We have heard back from four, all of whom said they weren’t contacted prior to being put on the list.

The Daily Caller‘s editor-in-chief, Geoffrey Ingersoll, told iMediaEthics, “I went back and forth with Poynter over a handful of emails in an attempt to outline the lack of veracity and comprehension in their methodology.”

“Poynter did not contact the Beacon prior to publication,” the Washington Free Beacon‘s editor Matthew Continetti told iMediaEthics. “We were about to lodge a complaint when Poynter pulled the article.”

The Washington Examiner told iMediaEthics it also didn’t hear from Poynter before publication. The Washington Examiner complained and Poynter ended up admitting the site shouldn’t have been included, the Examiner itself reported.

The Daily Signal’s director and editor-in-chief Katrina Trinko told iMediaEthics by e-mail “Poynter as far as we know did not reach out to us prior to the list being published, and in fact, when I gave Poynter the chance to confirm or deny that they had reached out to us before I published an op-ed about the matter, Poynter made zero assertion that they had tried to reach us prior to the list being made public—or for that matter, until we started reaching out to them! “

Poynter’s List based off 5 Lists

Poynter’s report says the list was derived from databases found on Factcheck.org’s Misinformation Directory; Fake News Codex; Open Sources, a project from a class at Merrimack University; PolitiFact’s Fake News Almanac; and Snopes’ Field Guide to Fake News Sites and Hoax Purveyors.

An archive of the now-unpublished list shows that Poynter stated: “To create the index, we combined five major lists (see below), then eliminated the sites that were no longer active. We only used lists that were public and curated by established journalists or academics, contained original data (rather than information from other lists), stated their criteria for inclusion and defined how they graded different sites (see our methodology for more).”

iMediaEthics contacted all five to ask if they were reviewing their methodology after the article was unpublished and if Poynter contacted them before aggregating their lists.

PolitiFact, Factcheck.org and Snopes told iMediaEthics they didn’t work with Poynter on the list project. Factcheck.org and Snopes specifically said they never heard from Poynter beforehand. The Fake News Codex’s creator Christopher Herbert told iMediaEthics he was contacted “late last year, when it was a project associated with the Columbia Journalism Review/Tow Center.” Dr. Melissa Zimdars who set up OpenSources but didn’t maintain it anymore told iMediaEthics, “The journalist who compiled the resource asked me a lot of questions beforehand. I said I was no longer updating the resource and he said that was part of what he was going to do as part of this project.”

David Mikkelson, the publisher of Snopes Media Group, told iMediaEthics :

“Our ‘Field Guide’ is simply a collection of sites (many of them now defunct) that have published nothing other than hoaxes, fake news, and satire/humor, and have made no pretense of being legitimate news outlets.  Unlike the Poynter index, our guide did not presume to rate the ‘reliability’ (or lack thereof) of sites that identify as real news outlets, so no adjustment to our methodology is called for.

“Poynter did not consult with us about the compilation of their index, nor did they notify us that they intended to make use of our field guide.”

Dr. Melissa Zimdars of Merrimack University told iMediaEthics she hasn’t updated the OpenSources project since “summer/fall of 2017 because I found the whole thing to be a fruitless endeavor and it’s not my primary area of research interest.”

She said she wasn’t involved in the Poynter project but did answer questions regarding the list. “The freelancer who used my list and others for the Unreliable news list did clean up my outdated data, as far as I know, but I can’t speak to his methodology or this project more specifically as I was not involved in it,” she said.

OpenSources was a 2016 in-class project she then turned into “more of a public resource,” according to Zimdars.

“Some app developers used my work with my permission/assistance and called it OpenSources,” she e-mailed iMediaEthics. “I helped build/maintain OpenSources for several months with the help of some volunteer librarians. I/We analyzed websites as most fact checking organizations do (and my methodological training is in textual and discourse analysis). Of course, all analysis is subject to interpretation and I viewed the resource as dynamic and open to criticism and change.” (More on the background in this Washington Post article by Zimdars.)

Concerning political or ideological sites, Zimdars explained the list received more requests to review “right/right-leaning news websites,” and left-leaning sites on the list would often request removal.

“I noticed a trend in left leaning sources asking for removal while right leaning sources would instead publish smear campaigns about me and encourage their readers to harass me. They basically use the same outrage/victim/ontological security strategy that is so profitable for Fox News. Regardless, this meant that more political websites on the left were removed while leaving more political websites on the right to remain in the resource (and, again, this was before I stopped updating it entirely), which created a further political imbalance.”

Zimdars also pointed to her 2016 article about the list, which noted that initial coverage of OpenSources from 2016 didn’t contact her before publication and called her list a “‘blacklist,’ which is definitely not what I created.”

Angie Holan, the editor of PolitiFact, which is now owned by Poynter, told iMediaEthics by e-mail,

“PolitiFact did not work with the person who compiled it. I saw it only after it was published.  PolitiFact’s fake news list was accurate at the time we published it, but we had not reviewed it in some time. We had not intended for anyone to think it was being actively updated.”

The Fake News Codex creator, Chris Herbert, told iMediaEthics by e-mail he hasn’t reviewed his methodology or list after Poynter’s unpublishing. However, he noted that his list doesn’t score against news outlets for political or ideological bias. Herbert said he was contacted about the list “late last year, when it was a project associated with the Columbia Journalism Review/Tow Center.”

“I didn’t realize they unpublished the entire list.  The issue that critics raised — listing sites with a right slant but not those with a left slant — was one that I explicitly kept in mind when creating the Fake News Codex.  In case you haven’t seen it, here’s an excerpt from the “About” page:

“The Fake News Codex does not include sites that merely have a clear political or ideological bias – Daily Kos and Breitbart will not appear here.”

Eugene Kiely, the director of FactCheck.org, told iMediaEthics the FactCheck.org list “has evolved over time, but we haven’t made any changes to it or to our methodology” after Poynter’s unpublishing. Kiely explained that the list was created as a reference after working with Facebook to flag “fake news” on the site. FactCheck.org “periodically” updated the file, and last fall “gave the list a major facelift” to organize and update the list.

Kiely added, “I was’t aware of Poynter’s list until after it was taken down,” but when he reviewed the reports and archive of the list, “I saw that it included some sites for having a ‘bias.’ That’s not a factor in compiling our list.”

In addition, FactCheck.org’s list provided details about the sites on the list. Kiely detailed:

“At the time, there were some websites that would mimic real news sites, so we thought it would be helpful to provide readers with some information about them. For example, this was one of our entries when we first published the list:

“The Examiner  Theexaminer.site does not have a disclaimer. This should not be confused with the Washington Examiner, a legitimate news site that can be found at washingtonexaminer.com.

“Theexaminer.site – which we wrote about in June 2017 — doesn’t exist anymore. “

What Poynter Says

In a May 2 letter, Poynter managing editor Barbara Allen wrote that after complaints over the April 30 list, Poynter ran an “audit to test the accuracy and veracity of the list.” Allen said there were methodology issues.

iMediaEthics wrote to Allen to ask for more information. In response, we heard from Poynter vice president Kelly McBride. McBride said the Washington Examiner‘s complaint “was the first example that caused us to question the methodology,”

What are those methodology errors? ” Several, but the primary one was the inclusion of sites for the designation of bias,” McBride wrote to iMediaEthics. “Bias is impossible to fairly designate, because of, well, bias. Also, bias doesn’t equate with unreliable. 

We also asked if Poynter will republish the list after its audit is complete, to which McBride says, “that depends on what we find in the audit.”

“Finally, let me just say that one of the hallmarks of a responsible news organization is that they own their mistakes. We’re doing that. We messed up and so we took it down,” she added.

UPDATED: 5/16/2019 2:46 PM With response from Daily Signal.