fMRI: a very critical review, again (Introduction)

by David Turell , Monday, July 18, 2016, 15:11 (3052 days ago) @ David Turell

Another version of the discovery that some fMRI is not that statistically reliable:-https://www.newscientist.com/article/2097734-thousands-of-fmri-brain-studies-in-doubt-due-to-software-flaws/-"It's another blow for neuroscience. The discovery of major software flaws could render thousands of fMRI brain studies inaccurate.-***-"Anders Eklund and Hans Knutsson at Linköping University in Sweden and Thomas Nichols at the University of Warwick, UK, calculated this by analysing brain data from a collaborative open fMRI project called 1000 Functional Connectomes. Most fMRI statistical methods have been developed using simulated data, but in this case the team was able to use real brain information to validate the techniques for identifying significant patterns of activity in fMRI scans.-"This work enabled the researchers to confirm that a statistical software flaw they first identified in 2012 truly does produce false positives at an alarmingly high rate. Four years ago, they were not taken very seriously because their work at that time was based on data from a single person.
 
"Now we know that this is a very real problem indeed. Although the software error has now been corrected, potentially thousands of fMRI studies are in doubt.-***-"According to Ekland, it is difficult to determine which studies have been affected by the spatial autocorrelation flaw because raw data from past studies is rarely available.-"Nichols estimates that around one in 10 fMRI studies may be affected. This is somewhat lower than the full “15 years of brain research” that some publications have suggested are now called into question.-"However, when you take into account that, on top of the software flaw, a further 40 per cent of this type of study may be compromised by researchers failing to apply the right corrections in the first place, this hints that many studies indeed may have reported false positive results.-"The types of study most likely to be affected are those that are usually reported with headlines along the lines of “X causes part Y of your brain to light up”, or “This is your brain on drug Z”. Those that show relatively weak statistical associations are the likeliest to be inaccurate - whereas studies reporting strongly significant findings may ultimately still be sound.-***-"But not all fMRI researchers are worried. Jens Foell at Florida State University in Tallahassee likens fMRI to using a blurry magnifying glass to look at the most complex object in the universe. As a consequence, he says, we should always be sceptical of fMRI studies with results that have only borderline statistical significance.-"In light of this growing evidence of the unreliability of some fMRI research, it is more important than ever to treat findings with caution. There is a large body of psychological evidence that shows people are particularly easily swayed by neuroscience. But although neuroscience is an important field, it is a young one that relies on inherently fuzzy data. Psychological studies that measure behaviour and actions directly may often be more useful, relevant and reliable.-“'What I expect the fMRI research community to do now is neither to worry nor to be surprised, but to engage in rigorous discussion,” says Foell."-Comment: Caution, do not over-interpret.