All Hail AllTrials


Ben Goldacre (from his blog) is a medical doctor, academic, campaigner, and writer whose work focuses on uses and misuses of science and statistics by journalists, politicians, drug companies, and quacks. Among his more high profile targets have been British TV nutritionist Gillian McKeith; anti-immunization ghoul Andrew Wakefield; Brain Gym; and researcher/supplement entrepreneur/conspiracy theorist Matthias Rath.

Perhaps his biggest crusade is directed against what he calls “publication bias,” although he seems to be most concerned with what is more properly termed “positive-results bias,” a particular category of publication bias. As the name indicates, this phenomenon describes the tendency that researchers are more likely to submit—or journal editors accept—positive results, as opposed to those which are negative or inconclusive.

“Publication bias” is a bit more generic, occurring when the publication of research results depends not just on the quality of the research but on the hypothesis tested, and the significance and direction of effects detected. The term is said to have been originated in 1959 by statistician Theodore Sterling, from an article in Journal of the American Statistical Association, entitled “Publication decisions and their possible effects on inferences drawn from tests of significance—or vice versa.”

Here is the pull quote from that old article…

There is some evidence that in fields where statistical tests of significance are commonly used, research which yields non-significant results is not published. Such research being unknown to other investigators may be repeated independently until eventually by chance a significant result occurs—an ‘error of the first kind’—and is published. Significant results published in these fields are seldom verified by independent replication. The possibility thus arises that the literature of such a field consists in substantial part of false conclusions resulting from errors of the first kind in statistical tests of significance.

As to Goldacre, he raised a few disturbing —> Read More