TY - JOUR
T1 - Evaluating the efficiency and safety of speech recognition within a commercial electronic health record system
T2 - a replication study
AU - Hodgson, Tobias
AU - Magrabi, Farah
AU - Coiera, Enrico
PY - 2018/4/1
Y1 - 2018/4/1
N2 - Objective: To conduct a replication study to validate previously identified significant risks and inefficiencies associated with the use of speech recognition (SR) for documentation within an electronic health record (EHR) system. Methods: Thirty-five emergency department clinicians undertook randomly allocated clinical documentation tasks using keyboard and mouse (KBM) or SR using a commercial EHR system. The experiment design, setting, and tasks (E2) replicated an earlier study (E1), while technical integration issues that may have led to poorer SR performance were addressed. Results: Complex tasks were significantly slower to complete using SR (16.94%) than KBM (KBM: 191.9 s, SR: 224.4 s; p = 0.009; CI, 11.9-48.3), replicating task completion times observed in the earlier experiment. Errors (non-typographical) were significantly higher with SR compared with KBM for both simple (KBM: 3, SR: 84; p < 0.001; CI, 1.5- 2.5) and complex tasks (KBM: 23, SR: 53; p = 0.001; CI, 0.5-1.0), again replicating earlier results (E1: 170, E2: 163; p = 0.660; CI, 0.0-0.0). Typographical errors were reduced significantly in the new study (E1: 465, E2: 150; p < 0.001; CI, 2.0-3.0). Discussion: The results of this study replicate those reported earlier. The use of SR for clinical documentation within an EHR system appears to be consistently associated with decreased time efficiencies and increased errors. Modifications implemented to optimize SR integration in the EHR seem to have resulted in minor improvements that did not fundamentally change overall results. Conclusion: This replication study adds further evidence for the poor performance of SR-assisted clinical documentation within an EHR. Replication studies remain rare in informatics literature, especially where study results are unexpected or have significant implication; such studies are clearly needed to avoid overdependence on the results of a single study.
AB - Objective: To conduct a replication study to validate previously identified significant risks and inefficiencies associated with the use of speech recognition (SR) for documentation within an electronic health record (EHR) system. Methods: Thirty-five emergency department clinicians undertook randomly allocated clinical documentation tasks using keyboard and mouse (KBM) or SR using a commercial EHR system. The experiment design, setting, and tasks (E2) replicated an earlier study (E1), while technical integration issues that may have led to poorer SR performance were addressed. Results: Complex tasks were significantly slower to complete using SR (16.94%) than KBM (KBM: 191.9 s, SR: 224.4 s; p = 0.009; CI, 11.9-48.3), replicating task completion times observed in the earlier experiment. Errors (non-typographical) were significantly higher with SR compared with KBM for both simple (KBM: 3, SR: 84; p < 0.001; CI, 1.5- 2.5) and complex tasks (KBM: 23, SR: 53; p = 0.001; CI, 0.5-1.0), again replicating earlier results (E1: 170, E2: 163; p = 0.660; CI, 0.0-0.0). Typographical errors were reduced significantly in the new study (E1: 465, E2: 150; p < 0.001; CI, 2.0-3.0). Discussion: The results of this study replicate those reported earlier. The use of SR for clinical documentation within an EHR system appears to be consistently associated with decreased time efficiencies and increased errors. Modifications implemented to optimize SR integration in the EHR seem to have resulted in minor improvements that did not fundamentally change overall results. Conclusion: This replication study adds further evidence for the poor performance of SR-assisted clinical documentation within an EHR. Replication studies remain rare in informatics literature, especially where study results are unexpected or have significant implication; such studies are clearly needed to avoid overdependence on the results of a single study.
KW - Electronic health record
KW - Integration
KW - Medical errors
KW - Patient safety
KW - Speech recognition
UR - http://www.scopus.com/inward/record.url?scp=85054421414&partnerID=8YFLogxK
UR - http://purl.org/au-research/grants/nhmrc/1032664
U2 - 10.1055/s-0038-1649509
DO - 10.1055/s-0038-1649509
M3 - Article
C2 - 29768633
AN - SCOPUS:85054421414
VL - 9
SP - 326
EP - 335
JO - Applied Clinical Informatics
JF - Applied Clinical Informatics
SN - 1869-0327
IS - 2
ER -