Does health informatics have a replication crisis?

Research output: Contribution to journalArticlepeer-review

25 Citations (Scopus)
11 Downloads (Pure)


Objective: Many research fields, including psychology and basic medical sciences, struggle with poor reproducibility of reported studies. Biomedical and health informatics is unlikely to be immune to these challenges. This paper explores replication in informatics and the unique challenges the discipline faces. Methods: Narrative review of recent literature on research replication challenges. Results: While there is growing interest in re-analysis of existing data, experimental replication studies appear uncommon in informatics. Context effects are a particular challenge as they make ensuring replication fidelity difficult, and the same intervention will never quite reproduce the same result in different settings. Replication studies take many forms, trading-off testing validity of past findings against testing generalizability. Exact and partial replication designs emphasize testing validity while quasi and conceptual studies test generalizability of an underlying model or hypothesis with different methods or in a different setting. Conclusions: The cost of poor replication is a weakening in the quality of published research and the evidence-based foundation of health informatics. The benefits of replication include increased rigor in research, and the development of evaluation methods that distinguish the impact of context and the nonreproducibility of research. Taking replication seriously is essential if biomedical and health informatics is to be an evidence-based discipline.
Original languageEnglish
Pages (from-to)963-968
Number of pages6
JournalJournal of the American Medical Informatics Association : JAMIA
Issue number8
Publication statusPublished - Aug 2018

Bibliographical note

Copyright the Author(s) 2018. Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.


  • evaluation
  • replication
  • evidence-based informatics
  • research bias

Fingerprint Dive into the research topics of 'Does health informatics have a replication crisis?'. Together they form a unique fingerprint.

Cite this