TY - JOUR
T1 - The extent and consequences of p-hacking in science
AU - Head, Megan L.
AU - Holman, Luke
AU - Lanfear, Rob
AU - Kahn, Andrew T.
AU - Jennions, Michael D.
N1 - Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.
PY - 2015/3/13
Y1 - 2015/3/13
N2 - A focus on novel, confirmatory, and statistically significant results leads to substantial bias in the scientific literature. One type of bias, known as “p-hacking,” occurs when researchers collect or select data or statistical analyses until nonsignificant results become significant. Here, we use text-mining to demonstrate that p-hacking is widespread throughout science. We then illustrate how one can test for p-hacking when performing a meta-analysis and show that, while p-hacking is probably common, its effect seems to be weak relative to the real effect sizes being measured. This result suggests that p-hacking probably does not drastically alter scientific consensuses drawn from meta-analyses.
AB - A focus on novel, confirmatory, and statistically significant results leads to substantial bias in the scientific literature. One type of bias, known as “p-hacking,” occurs when researchers collect or select data or statistical analyses until nonsignificant results become significant. Here, we use text-mining to demonstrate that p-hacking is widespread throughout science. We then illustrate how one can test for p-hacking when performing a meta-analysis and show that, while p-hacking is probably common, its effect seems to be weak relative to the real effect sizes being measured. This result suggests that p-hacking probably does not drastically alter scientific consensuses drawn from meta-analyses.
UR - http://purl.org/au-research/grants/arc/FT140100843
UR - http://www.scopus.com/inward/record.url?scp=84926391432&partnerID=8YFLogxK
U2 - 10.1371/journal.pbio.1002106
DO - 10.1371/journal.pbio.1002106
M3 - Article
C2 - 25768323
SN - 1544-9173
VL - 13
SP - e1002106-1-e1002106-15
JO - PLoS Biology
JF - PLoS Biology
IS - 3
M1 - e1002106
ER -