Cappella and Colleagues Rate Accuracy in Genetic Cancer Research News Stories

Research Area: 

News stories about cancer research in the mass media often do not fairly represent the original science. The primary source of this problem seems to occur in the transmission of information from press release to final news story, although the process of turning highly technical scientific research into a press release is itself fraught with errors.

That was the finding of an analysis by researchers at the University of Pennsylvania’s Annenberg School for Communication, who looked at news reports of genetic cancer research that appeared in the press between 2004 and 2007. Researchers Jean M. Brechman, Ph.D., Chul-joo Lee, Ph.D., and Joseph Cappella, Ph.D., used a panel of genetic science experts to rate the accuracy of news stories about genetic cancer research. Their work, “Distorting Genetic Research about Cancer: From Bench Science to Press Release to Published News” is in press with the Journal of Communication. Among their findings:

  • Claims about genetic science in a press release are likely to be more accurate than claims made in a news story.
  • News articles that involve an interview with a third party, such as an expert on the subject who was not involved with the original science article, were considered to be more accurate than news stories without such “neutral” sources.
  • A press release or news story containing a direct quote from a science expert was rated more accurate than those that did not have a quote.
  • A prestigious news organization’s coverage of science news was not considered more accurate than, say, a wire service story on the same subject.

“There are thousands of news stories about genetics and health, and people make important personal decisions about their health based largely on such stories,” said Dr. Brechman. “One can feel overly fatalistic, or overly competent; or news coverage can make it more difficult for medical professionals to educate patients about health risks.”

Noting there are “documented errors in science journalism, including the omission of critical information and context, misquoting, and simplification/sensationalization of headlines,” the researchers analyzed the classic information transmission mode for science information. That is, science experts provide highly specialized information to public information officers, who then produce press releases in an effort to facilitate the transfer of this information to journalists who then popularize it via news stories.

The researchers used online news databases (LexisNexis, etc.) to identify nearly 6,000 news stories about cancer and genetic cancer research from 2004 to 2007. This enormous pool of sources was pared down to just over 100 stories by eliminating obits, community calendar mentions of the disease, using only stories that that received press coverage from more than one news source, and which would be traced to a press release and published scientific article.

A panel of 40 expert raters – graduate students studying genetic science – were asked to read the original science journal story. They were then asked to rate the accuracy of a series of statements about the original story. These statements came from a press release, a news story, or a false claim inserted in the story as a control device. The panel of raters did not know the source of each statement.

On a scale of one to seven, with “one” being least accurate and “seven” being completely accurate, the panel rated statements coming from press releases higher than those in news stories (an average score of 4.78 versus 4.0). They also rated stories containing information from a non-related expert (i.e. – a third party expert source) higher on the accuracy scale (4.2 versus 3.96 for stories with no additional information). That is, when the claim rated by the genetics doctoral students came from an article that contained information from a third party expert, the claim was judged as more accurate even though the doctoral students rating the claim had no idea that the article contained such additional information. They never read the press release or news article, only isolated statements the researchers extracted from the articles. When the journalists doing the writing took the extra step of getting additional perspective on the new scientific findings, the accuracy of the material being communicated went up.

“Our findings suggest that as scientific knowledge is filtered and translated for mass consumption, there are slippages and inconsistencies that result in coverage that does not fairly represent the original science,” Dr. Brechman said.  While the primary source of this distortion “occurs between the press release and news article,” the study also noted that “our findings do not dismiss the role of public information officers.” The study noted that nearly 16 percent of the press release claims were rated below 4, the accuracy scale midpoint.

“Perceived inaccuracies in news coverage may be the result of restrictions imposed on journalists,” the study noted. “Charged with communicating highly complex information to lay audiences, journalists must simplify scientific information and package it in a way that will appeal to readers and be understood.

The researchers noted limitations of their study, including the fact that many scientific claims could be taken out of context and the overall limited size of the pool of news stories available for the study.