Wall Street Journal Mistakes in Scientific Studies Surge


Because of the new business model of the WSJ Online, only subscribers can view the whole article. However, there is a somewhat sensationalist summary in MinnPost.com, an outlet unknown to me. Surge in scientific retractions raises troubling questions

Wow, Tollie, that's kind of frightening. I know there is enormous pressure to "publish", but publishing fraudulent stuff that may or does result in actual damage to patients, for example, is horrifying. OTOH, do you think this story is part of the movement to discredit science and bolster the science deniers in "controversies" like climate change?

Maybe this report will be retracted?

But seriously, thanks for the interesting find.

How many actual articles appear in 11,000+ journals annually? So ~400 retractions is still a very small percentage.

And does the increase really indicate a rise in either chicanery or sloppiness, as must be inferred?

The minnpost article acknowledges:

Another factor mentioned is the domino effect, itself also not indicative of a rise in sloppiness per se. I also wonder if there are other factors, not mentioned, perhaps related to the internet, that would indicate a rise in detection and reporting, more than a rise in occurrence. (A common problem when studying crime reports, especially of typically underreported crimes like child molestation and sexual assault.)

But I also wonder what it may say about what it means to "peer review"? Are so many articles in such arcane areas of specialty, that there are few "peers" truly qualified to review?

One of the things I like about our family doc (besides the fact that he always asks how scrabble is going) is that he stays very updated on new and improved therapies, tests, etc. Which I suppose can make him -- and us -- susceptible to being the early adopters that are potentially harmed by inadequately reviewed studies.

Yet may of these are self discovered and corrections to papers. Fraud such as the
infamous vaccine panic set off by a charlatan in the UK and picked up by the Antivaxers..

There may be an element of derogation of science and scientists in the report referenced by the Wall Street Journal. However, this report of rising retractions is consistent with what the scientific integrity office of the National Institutes of Health has been saying for years. It has also been the experience on too many university campuses. We had it ourselves, leading to the abolition of an entire research institute.

Cheating is found among those who are no longer under intense pressure to publish, i.e., the tenured faculty. At least in the biological sciences, there is another intense pressure: to make money off of a discovery that can be patented.

Part of the explanation may be that we are simply better at detecting errors and fraud. We may also be more alert to its likelihood. The sordid story of Andrew Wakefield and the MMR vaccine is very much on scientists' minds.

There is a serious problem: there is little or no reward for uncovering an error or fraud. In fact, whistleblowers have been shunned by their colleagues, particularly when their discovery destroyed a career. Very early in my career, I published a letter to the editor of a scientific journal in which I pointed out that the results reported in an article by a leading researcher were mathematically impossible. (Hektor will understand that if A is correlated with B, and B is correlated with C, then the correlation between A and C is bounded, the more so the stronger the correlations.) For some inexplicable reason, I ended up being hired in the department where that researcher still taught. That may have been evidence of honesty and honor by the researcher.

A classic recent case was that of Marc Hauser, whose work I had followed with great interest. He is/was an evolutionary biologist who taught in the Psychology Department at FAS, Harvard. His work was primarily with primates and involved the idea that a primitive moral code is embedded in our genes. It is basically a code of fair play, a primitive version of the Golden Rule that is universal to the world's religions.

Hauser had everything: tenure at Harvard, his own laboratory, a continuous flow of grants, the very rare and prestigious National Science Foundation Young Investigator Award, a science medal from the Collège de France, and a Guggenheim Fellowship. He was very widely published. Moral Minds: How Nature Designed Our Universal Sense of Right and Wrong was, for science, a bestseller.

Ironically and tragically, he was accused by his own graduate students of breaches of scientific integrity. Harvard investigated and found him guilty of eight counts of misconduct. His last day at Harvard was August 2, 2011. He had broken the very moral code that he was studying: fair play is fundamental to science.

I believe that Hauser is so convinced of the validity of his findings -- and of their practical importance -- that he tweaked the results to support his theories when necessary. However, I have not and will not see Harvard's internal report of what actually happened. I think Hauser is the first of the 21st century's Sir Cyril Burt's. I also suspect Hauser's findings are true, as Burt's findings are widely believed to be true. You can see evidence of a moral code in your own domestic companions. (This may not always apply to human companions. Some orangutans break the moral code, too.)

Pharmalot has more analysis. The Journal of Medical Ethics had a study of 742 papers withdrawn between 2000 and 2010 and found that 76% of them were withdrawn for mistakes and 24% were withdrawn for fraud. They have a great graph of retractions per paper published. The MinnPost article pointed out that much of the time a prominent paper that has to be withdrawn will trigger a cascade of papers to be withdrawn. I imagine that the increasing number of journals means that there are more follow-on papers now than in the past.

I also think there is something to the criticism of the Lancet, in particular, in taking on riskier papers and then taking years to withdraw them. Wakefield's study of autism was discredited, comprehensively, 6 years after it was published and 6 years before it was withdrawn from the Lancet! (Depending on when the Journal of Medical Ethics ended their review it might have been included in their study.)

May I opine that IMHO the lack of money has led to folks not doing their own basic research, but relying on others. If you don't have to repeat part of your experiment, but can accept a peer reviewed result of...

hazards of no cash....
YMMV

Was there any correlation between the withdrawn studies and corporate funding?

And government funded research.

There is a bill under consideration in Congress that could reduce the number of retractions of scientific articles. Under rules that have been in place since 2009, the National Institutes of Health have required that articles resulting from research that it funds be made publicly and freely available on the National Library of Medicine’s Web site. This availability might be a factor in the increase in the number of retractions of scientific articles because all sorts of people, including highly skeptical scientists, physicians, teachers, and students could read and evaluate the evidence presented. This is what has been meant by the truth that science must be conducted in the sunlight.

That rule is now under attack from the publishing industry, with the support of Representatives Carolyn B. Maloney, Democrat of New York, and Darrell Issa, Republican of California. Publishing houses are doing everything possible to ensure their existence even when the public interest dictates that those articles be made publicly available. You and I paid for the research, and in most cases, authors have paid a fee to a journal to cover the costs of publication. Those fees are built into budgets for NIH-funded research and have probably amounted to billions of dollars since 2009.

I oppose, on principle, anything that Issa supports, but why is Maloney a co-sponsor? To protect the publishing houses based in NYC? Why else would this "fiery liberal" support such a bill?

New York Times Jan. 10, 2011 "Research Bought, Then Paid For" by Michael B. Eisen, an associate professor of molecular and cell biology at the University of California, Berkeley.

It is, of course, true that even journals that are ostensibly peer-reviewed will sometimes publish trash. The recent case of the resignation of the Editor of the journal Remote Sensing, an "open-access" peer-reviewed journal, is a case in point. Open-access, peer-reviewed journals are not the perfect answer. Peer review must be complemented by strictly-enforced standards of scientific integrity in the universities and in corporations. In the notorious case of Marc Hauser, Harvard has shown how this can be done.

In 1998 Dr. Andrew Wakefield published a notorious paper in which he claimed to show that there is a link between the onset of autism and/or bowel disease and the administration of the mumps, measles, and rubella vaccine (MMR). Other investigators attempted to replicate his results but failed. A journalist, Brian Deer, began an expose of Wakefield in the Sunday Times, starting with an undisclosed financial conflict that Wakefield had. In January 2010 the British General Medical Council found that the bulk of the claims against Wakefield had been proved. Lancet formally retracted the 1998 paper, and an editorial in BMJ (formerly British Journal of Medicine) termed Wakefield's paper an "elaborate fraud". In the meantime, vaccination rates dropped in the U.K., the U.S., and other countries. For several years parents had every reason to believe that Wakefield's findings were valid, and they feared the MMR vaccine.

January 5 2012 The Guardian Andrew Wakefield sues BMJ for claiming MMR study was fraudulent: BMJ and investigative journalist stand by allegations in articles about research linking MMR vaccine to autism

Wakefield now lives in Austin, TX. I am not sure how that confers upon a Texas district court the authority to hear a suit against a British journal, a British newspaper, and a British journalist. I admit to confusion as to why only BMJ (not also Lancet) is being sued.

The self-policing of science is possible only if other scientists can write articles pointing out issues with earlier articles, including the issue that the original study's results have yet to be replicated.

Suing the journal in Texas appears to be for pure harassment purposes. Ordinarily, I would chalk something like this up to the fact that he lives in Texas and prefers a forum more convenient to him. However, this is a defamation case. American defamation law is very plaintiff-unfriendly, and Wakefield is certainly at least a limited public figure concerning his fraudulent paper, and therefore, would be held to the nearly impossible to prove "actual malice" standard, that is, he would have to prove that the defendants either knew what they were saying was false, or acted in reckless disregard as to its truth or falsity.

Frankly, given the undisputed facts of the case, I seriously doubt this charlatan and quack Wakefield could even plead such a case, much less get it past summary judgment, much less prove it. However, it will be costly and inconvenient for the British journal and other defendants to answer these claims. If he gets it past a motion to dismiss, he can commence, again, costly and inconvenient discovery. Despite that, the suit is ultimately doomed.

However, compare defamation law in England. It is notoriously plaintiff-friendly, perhaps the most defamation plaintiff-friendly forum in the world, to the point notorious criminals and terrorists come to England to sue for being described as what they are, and with the aid and assistance of bottom-feeding legal trolls like the Carter-Ruck firm, sometimes even win, but more often simply spend their victims into the ground. You see, it is up to the libel defendant in English defamation law to prove the truth of the allegations, or sufficiently plead an affirmative defense like fair comment. Why would a plaintiff with a valid libel suit proceed in Texas instead of England?

I suspect he wants to take advantage of the "American rule," that is, that American tort defendants generally must pay their own legal fees. Under the "English rule," the losing plaintiff would have to pay the defendant.

Frankly, the piece of shit Wakefield would lose under either standard. This way, though, he gets to drag the defendants through court in Texas, even if all they have to do is successfully contest jurisdiction or, failing that, file a motion to dismiss. Doing it the way a legitimate libel plaintiff with a real case would do it, though, he would file suit in England.

He doesn't have such a case. He's a rightfully discredited kook who deserved to have his career terminated with prejudice. Therefore, he files a harassing kook suit in Texas, where he can harass the defendants and probably get away with it scot-free.

Would it be possible and helpful for U.S. and U.K. scientific and medical societies to express themselves on this suit, perhaps filing amicus briefs? It is to the interest of the health of residents of the U.S. and the U.K. that Wakefield's very damaging article be even more fully exposed as fraud. Some articles about the Wakefield affair have it that vaccination levels fell to dangerously low levels, at least locally and maybe regionally. In other words, could Wakefield's filing of this suit be turned to his disadvantage?

I condemn Wakefield not only for reducing the level of herd immunity but also for placing parents of autistic children in the impossible emotional situation of feeling that something that they thought they were doing for the good of their child actually did harm. Nobody should have to bear that guilt.

Reuters March 28, 2012 "In cancer science, many 'discoveries' don't hold up"

Begley's study may be flawed. Without going inside the Nature paywall, it is not clear to me how Begley was able to attempt to replicate 53 studies. Replication can be difficult, time-consuming, and expensive. The original articles may not have disclosed all that would be required for a full replication. None of Begley's findings may lead to retractions of articles in scientific journals, much less to withdrawals from the market of any cancer drugs that were based upon these studies.

However, the fact cited near the end of the article is certainly true: academic advancement is based upon a rewards system in which saying that a study cannot be replicated gains few points and may even cost points. "Failures to replicate" are usually relegated to the Commentary or Letters section of a journal and do not count for as much as regular articles. Citation counts may be low for mere letters or commentaries. This is not restricted to the biological sciences but may be exacerbated there by the immense volume of grants and contracts available to researchers.

Scientific research is supposed to be self-policing through attempts to replicate findings. The system now inhibits and perhaps discourages self-policing.

That's a set of perverse incentives if I ever saw one. Science is advanced as much or more by failure as it is by success. Edison's invention of the light bulb would be one example, as he discovered innumerable substances that would not work for the incandescent filament before finally finding one that did.

A slightly different twist on this topic: Fake conferences.

Pretend publications and counterfeit conferences

And a related Blog Post referenced above:

A highly disputed 2008 paper somehow became a blockbuster proof in 2012. It may have begun with Vaccine Liberation Army's post of May 8, 2012.

The "vaccines cause autism" networks began lighting up with a claim that research at the University of Pittsburgh had shown "Cover Up Continues: Monkeys Injected With Vaccines Develop Autism". It showed up in Infowars and LewRockwell.com, as well as in Vactruth and Before It's News, to mention only a few. Before It's News called the paper a "Vaccine Bombshell."

"Influence of pediatric vaccines on amygdala growth and opioid ligand binding in rhesus macaque infants: A pilot study" by Dr. Laura C. Hewitson et al studied 11 macaque infant monkeys, two of which were in the control group that was not administered the standard vaccines of the 1990s. In both of the control group macaques, the amygdala region of the brain shrunk in volume. In the treatment group, the amygdala continued to grow. Although growth of the amygdala is normal for the macaque brain, this is taken as evidence that the vaccines caused the growth. This is then taken to be evidence that vaccines cause autism.

The article is dissected by Orac,

In his blog, Respectful Insolence, Orac published "Too much vaccine/autism monkey business for me to be involved in--but apparently not Laura Hewitson". I don't see that much remains of the article after Orac takes it apart. He also notes a line in the Acknowledgements

The paper is said to have been presented at the International Meeting for Autism Research in London in 2008. Like Sergeant Schultz, I know nothing about this "Meeting" or its sponsor(s).

The date of the meeting is important with regards to the affiliation with the University of Pittsburgh of the lead author. She was then an Adjunct Associate Professor. That affiliation lends credibility to the article, which was published in the Polish journal Acta Neurobiologiae Experimentalis in 2010.

Even in 2010, Dr. Hewitson also held an appointment with the Thoughtful House Center for Children in Austin. In 2004 Dr. Andrew Wakefield became the leader of Thoughtful House, although he was not permitted to practice medicine in Texas. He resigned in February, 2010. This was followed by Dr. Hewitson's resignation from her adjunct appointment at Pittsburgh in the middle of 2010. Thoughtful House has since been renamed the Johnson Center for Child Health and Development, to honor Betty Wold Johnson for her support.

Dr. Hewitson has a sad conflict of interest in the results of her research. She has a child with autism and has filed suit in the Vaccine Court. She has properly acknowledged this in some of her papers and presentations.

This saga illustrates a far-too-common problem in scientific publication: it is hard to dampen an article's impact once it has gone public. Wakefield's Lancet article is the prime example.

cha-ching!