Magazine article The Times Higher Education Supplement : THE

Voodoo Economics: Most Papers in the Field Are 'Not Reproducible'

Magazine article The Times Higher Education Supplement : THE

Voodoo Economics: Most Papers in the Field Are 'Not Reproducible'

Article excerpt

Questions about 'hacking' of data raised after discipline's rigour is tested. David Matthews writes

At least half of papers in economics are not reproducible, a new analysis has found, suggesting that the "reproducibility crisis" in academia is not confined to lab sciences.

Researchers from the United States Federal Reserve and the Department of the Treasury tried to replicate the results from 67 papers across 13 prestigious journals, but even after contacting authors when necessary, they were successful in only 49 per cent of cases where the data were not confidential and the researchers had the right software to analyse it.

"We assert that economics research is usually not replicable," the paper concludes.

The findings feed into broader concerns that academics are engaging in statistical sleights of hand, not being open with data and failing to control their own biases in order to get career-boosting positive results in top journals.

Last week, two studies reported that scientists doing experiments with animals were often failing to use well-known techniques - for example blinding themselves to which animals were receiving what drug - to mitigate their biases, and therefore potentially exaggerating the impact of new treatments.

But this new study highlights that the problem of reproducibility might not be confined to the lab.

According to the paper, the main reason for being unable to replicate findings was an inability to find the right data or the computer code that produced the original results, even after contacting the authors. Code was missing crucial functions, or certain variables were absent from the data, the paper says.

However, in nine cases for which the authors of the paper had the right dataset and code, they nonetheless got a different result or the code failed to finish executing, according to "Is Economics Research Replicable? …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.