Evaluating the usability of speech recognition to create clinical documentation using a commercial electronic health record

Tobias Hodgson, Farah Magrabi, Enrico Coiera

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Objective: To conduct a usability study exploring the value of using speech recognition (SR) for clinical documentation tasks within an electronic health record (EHR) system. Methods: Thirty-five emergency department clinicians completed a system usability scale (SUS) questionnaire. The study was undertaken after participants undertook randomly allocated clinical documentation tasks using keyboard and mouse (KBM) or SR. SUS scores were analyzed and the results with KBM were compared to SR results. Results: Significant difference in SUS scores between EHR system use with and without SR were observed (KBM 67, SR 61; P = 0.045; CI, 0.1 to 12.0). Nineteen of 35 participants scored higher for EHR with KBM, 11 higher for EHR with SR and 5 gave the same score for both. Factor analysis showed no significant difference in scores for the sub-element of usability (EHR with KBM 65, EHR with SR 62; P = 0.255; CI, −2.6 to 9.5). Scores for the sub-element of learnability were significantly different (KBM 72, SR 55; P < 0.001; CI, 9.8 to 23.5). A significant correlation was found between the perceived usability of the two system configurations (EHR with KBM or SR) and the efficiency of documentation (time to document) (P = 0.002; CI, 10.5 to −0.1) but not with safety (number of errors) (P = 0.90; CI, −2.3 to 2.6). Discussion: SR was associated with significantly reduced overall usability scores, even though it is often positioned as ease of use technology. SR was perceived to impose larger costs in terms of learnability via training and support requirements for EHR based documentation when compared to using KBM. Lower usability scores were significantly associated with longer documentation times. Conclusion: The usability of EHR systems with any input modality is an area that requires continued development. The addition of an SR component to an EHR system may cause a significant reduction in terms of perceived usability by clinicians.

LanguageEnglish
Pages38-42
Number of pages5
JournalInternational Journal of Medical Informatics
Volume113
DOIs
Publication statusPublished - 1 May 2018

Fingerprint

Electronic Health Records
Documentation
Training Support
Statistical Factor Analysis
Hospital Emergency Service

Keywords

  • Electronic health records
  • Integration
  • Learnability
  • Speech recognition
  • System usability

Cite this

@article{360f5228bac74b4cb861cd1a34394d4b,
title = "Evaluating the usability of speech recognition to create clinical documentation using a commercial electronic health record",
abstract = "Objective: To conduct a usability study exploring the value of using speech recognition (SR) for clinical documentation tasks within an electronic health record (EHR) system. Methods: Thirty-five emergency department clinicians completed a system usability scale (SUS) questionnaire. The study was undertaken after participants undertook randomly allocated clinical documentation tasks using keyboard and mouse (KBM) or SR. SUS scores were analyzed and the results with KBM were compared to SR results. Results: Significant difference in SUS scores between EHR system use with and without SR were observed (KBM 67, SR 61; P = 0.045; CI, 0.1 to 12.0). Nineteen of 35 participants scored higher for EHR with KBM, 11 higher for EHR with SR and 5 gave the same score for both. Factor analysis showed no significant difference in scores for the sub-element of usability (EHR with KBM 65, EHR with SR 62; P = 0.255; CI, −2.6 to 9.5). Scores for the sub-element of learnability were significantly different (KBM 72, SR 55; P < 0.001; CI, 9.8 to 23.5). A significant correlation was found between the perceived usability of the two system configurations (EHR with KBM or SR) and the efficiency of documentation (time to document) (P = 0.002; CI, 10.5 to −0.1) but not with safety (number of errors) (P = 0.90; CI, −2.3 to 2.6). Discussion: SR was associated with significantly reduced overall usability scores, even though it is often positioned as ease of use technology. SR was perceived to impose larger costs in terms of learnability via training and support requirements for EHR based documentation when compared to using KBM. Lower usability scores were significantly associated with longer documentation times. Conclusion: The usability of EHR systems with any input modality is an area that requires continued development. The addition of an SR component to an EHR system may cause a significant reduction in terms of perceived usability by clinicians.",
keywords = "Electronic health records, Integration, Learnability, Speech recognition, System usability",
author = "Tobias Hodgson and Farah Magrabi and Enrico Coiera",
year = "2018",
month = "5",
day = "1",
doi = "10.1016/j.ijmedinf.2018.02.011",
language = "English",
volume = "113",
pages = "38--42",
journal = "International Journal of Medical Informatics",
issn = "1386-5056",
publisher = "Elsevier",

}

Evaluating the usability of speech recognition to create clinical documentation using a commercial electronic health record. / Hodgson, Tobias; Magrabi, Farah; Coiera, Enrico.

In: International Journal of Medical Informatics, Vol. 113, 01.05.2018, p. 38-42.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Evaluating the usability of speech recognition to create clinical documentation using a commercial electronic health record

AU - Hodgson, Tobias

AU - Magrabi, Farah

AU - Coiera, Enrico

PY - 2018/5/1

Y1 - 2018/5/1

N2 - Objective: To conduct a usability study exploring the value of using speech recognition (SR) for clinical documentation tasks within an electronic health record (EHR) system. Methods: Thirty-five emergency department clinicians completed a system usability scale (SUS) questionnaire. The study was undertaken after participants undertook randomly allocated clinical documentation tasks using keyboard and mouse (KBM) or SR. SUS scores were analyzed and the results with KBM were compared to SR results. Results: Significant difference in SUS scores between EHR system use with and without SR were observed (KBM 67, SR 61; P = 0.045; CI, 0.1 to 12.0). Nineteen of 35 participants scored higher for EHR with KBM, 11 higher for EHR with SR and 5 gave the same score for both. Factor analysis showed no significant difference in scores for the sub-element of usability (EHR with KBM 65, EHR with SR 62; P = 0.255; CI, −2.6 to 9.5). Scores for the sub-element of learnability were significantly different (KBM 72, SR 55; P < 0.001; CI, 9.8 to 23.5). A significant correlation was found between the perceived usability of the two system configurations (EHR with KBM or SR) and the efficiency of documentation (time to document) (P = 0.002; CI, 10.5 to −0.1) but not with safety (number of errors) (P = 0.90; CI, −2.3 to 2.6). Discussion: SR was associated with significantly reduced overall usability scores, even though it is often positioned as ease of use technology. SR was perceived to impose larger costs in terms of learnability via training and support requirements for EHR based documentation when compared to using KBM. Lower usability scores were significantly associated with longer documentation times. Conclusion: The usability of EHR systems with any input modality is an area that requires continued development. The addition of an SR component to an EHR system may cause a significant reduction in terms of perceived usability by clinicians.

AB - Objective: To conduct a usability study exploring the value of using speech recognition (SR) for clinical documentation tasks within an electronic health record (EHR) system. Methods: Thirty-five emergency department clinicians completed a system usability scale (SUS) questionnaire. The study was undertaken after participants undertook randomly allocated clinical documentation tasks using keyboard and mouse (KBM) or SR. SUS scores were analyzed and the results with KBM were compared to SR results. Results: Significant difference in SUS scores between EHR system use with and without SR were observed (KBM 67, SR 61; P = 0.045; CI, 0.1 to 12.0). Nineteen of 35 participants scored higher for EHR with KBM, 11 higher for EHR with SR and 5 gave the same score for both. Factor analysis showed no significant difference in scores for the sub-element of usability (EHR with KBM 65, EHR with SR 62; P = 0.255; CI, −2.6 to 9.5). Scores for the sub-element of learnability were significantly different (KBM 72, SR 55; P < 0.001; CI, 9.8 to 23.5). A significant correlation was found between the perceived usability of the two system configurations (EHR with KBM or SR) and the efficiency of documentation (time to document) (P = 0.002; CI, 10.5 to −0.1) but not with safety (number of errors) (P = 0.90; CI, −2.3 to 2.6). Discussion: SR was associated with significantly reduced overall usability scores, even though it is often positioned as ease of use technology. SR was perceived to impose larger costs in terms of learnability via training and support requirements for EHR based documentation when compared to using KBM. Lower usability scores were significantly associated with longer documentation times. Conclusion: The usability of EHR systems with any input modality is an area that requires continued development. The addition of an SR component to an EHR system may cause a significant reduction in terms of perceived usability by clinicians.

KW - Electronic health records

KW - Integration

KW - Learnability

KW - Speech recognition

KW - System usability

UR - http://www.scopus.com/inward/record.url?scp=85042508381&partnerID=8YFLogxK

UR - http://purl.org/au-research/grants/nhmrc/1032664

U2 - 10.1016/j.ijmedinf.2018.02.011

DO - 10.1016/j.ijmedinf.2018.02.011

M3 - Article

VL - 113

SP - 38

EP - 42

JO - International Journal of Medical Informatics

T2 - International Journal of Medical Informatics

JF - International Journal of Medical Informatics

SN - 1386-5056

ER -