Post-publication peer review:


Citation: Goodman S, Greenland S (2007) Why Most Published Research Findings Are False: Problems in the Analysis. PLoS Med 4(4): e168. doi:10.1371/journal.pmed.0040168


The article published in PLoS Medicine by Ioannidis [1<http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0040168#pmed-0040168-b001>] makes the dramatic claim in the title that "most published research claims are false," and has received extensive attention as a result. The article does provide a useful reminder that the probability of hypotheses depends on much more than just the p-value, a point that has been made in the medical literature for at least four decades, and in the statistical literature for decades previous. This topic has renewed importance with the advent of the massive multiple testing often seen in genomics studies.

Unfortunately, while we agree that there are more false claims than many would suspect-based both on poor study design, misinterpretation of p-values, and perhaps analytic manipulation-the mathematical argument in the PLoS Medicine paper underlying the "proof" of the title's claim has a degree of circularity. As we show in detail in a separately published paper [2<http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0040168#pmed-0040168-b002>], Dr. Ioannidis utilizes a mathematical model that severely diminishes the evidential value of studies-even meta-analyses-such that none can produce more than modest evidence against the null hypothesis, and most are far weaker. This is why, in the offered "proof," the only study types that achieve a posterior probability of 50% or more (large RCTs [randomized controlled trials] and meta-analysis of RCTs) are those to which a prior probability of 50% or more are assigned. So the model employed cannot be considered a proof that most published claims are untrue, but is rather a claim that no study or combination of studies can ever provide convincing evidence."


Ioannidis "proved" what we knew already, as summarized in the last sentence.

Christina


             ***********************************************
The BEE-L mailing list is powered by L-Soft's renowned
LISTSERV(R) list management software.  For more information, go to:
http://www.lsoft.com/LISTSERV-powered.html