How reliable is science? Perhaps not very (The limitations of science)

by David Turell , Wednesday, August 19, 2015, 04:56 (3383 days ago) @ David Turell

When studies are carefully monitored positive results tend to vanish:-http://www.nature.com/news/registered-clinical-trials-make-positive-findings-vanish-1.18181-"The launch of the clinicaltrials.gov registry in 2000 seems to have had a striking impact on reported trial results, according to a PLoS ONE study1 that many researchers have been talking about online in the past week.-"A 1997 US law mandated the registry's creation, requiring researchers from 2000 to record their trial methods and outcome measures before collecting data. The study found that in a sample of 55 large trials testing heart-disease treatments, 57% of those published before 2000 reported positive effects from the treatments. But that figure plunged to just 8% in studies that were conducted after 2000. Study author Veronica Irvin, a health scientist at Oregon State University in Corvallis, says this suggests that registering clinical studies is leading to more rigorous research. Writing on his NeuroLogica Blog, neurologist Steven Novella of Yale University in New Haven, Connecticut, called the study “encouraging” but also “a bit frightening” because it casts doubt on previous positive results.-***-"Many online observers applauded the evident power of registration and transparency, including Novella, who wrote on his blog that all research involving humans should be registered before any data are collected. However, he says, this means that at least half of older, published clinical trials could be false positives. “Loose scientific methods are leading to a massive false positive bias in the literature,” he writes.-***-"Still, of all the factors they studied, Irvin and Kaplan say that registration had strongest effect, even though it cannot erase all bias — even registered clinical studies showing positive results should be viewed with “healthy scepticism”, Irvin says. “Too often, the audience only reads the headline and the abstract.” It is only when you take a close look at the study details — such as effect sizes and response rates — that you can judge whether a result is likely to be clinically meaningful, she says."