Do online information retrieval systems help experienced clinicians answer clinical questions?

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Objective: To assess the impact of clinicians' use of an online information retrieval system on their performance in answering clinical questions. Design: Pre-/post-intervention experimental design. Measurements: In a computer laboratory, 75 clinicians (26 hospital-based doctors, 18 family practitioners, and 31 clinical nurse consultants) provided 600 answers to eight clinical scenarios before and after the use of an online information retrieval system. We examined the proportion of correct answers pre- and post-intervention, direction of change in answers, and differences between professional groups. Results: System use resulted in a 21% improvement in clinicians' answers, from 29% (95% confidence interval [CI] 25.4-32.6) correct pre- to 50% (95% CI 46.0-54.0) post-system use. In 33% (95% CI 29.1-36.9) answers were changed from incorrect to correct. In 21% (95% CI 17.1-23.9) correct pre-test answers were supported by evidence found using the system, and in 7% (95% CI 4.9-9.1) correct pre-test answers were changed incorrectly. For 40% (35.4-43.6) of scenarios, incorrect pre-test answers were not rectified following system use. Despite significant differences in professional groups' pre-test scores [family practitioners: 41% (95% CI 33.0-49.0), hospital doctors: 35% (95% CI 28.5-41.2), and clinical nurse consultants: 17% (95% CI 12.3-21.7; χ2 = 29.0, df = 2, p < 0.01)], there was no difference in post-test scores. (χ2 = 2.6, df = 2, p = 0.73). Conclusions: The use of an online information retrieval system was associated with a significant improvement in the quality of answers provided by clinicians to typical clinical problems. In a small proportion of cases, use of the system produced errors. While there was variation in the performance of clinical groups when answering questions unaided, performance did not differ significantly following system use. Online information retrieval systems can be an effective tool in improving the accuracy of clinicians' answers to clinical questions.

LanguageEnglish
Pages315-321
Number of pages7
JournalJournal of the American Medical Informatics Association
Volume12
Issue number3
DOIs
Publication statusPublished - May 2005
Externally publishedYes

Fingerprint

Information Systems
Confidence Intervals
Consultants
Nurses
Quality Improvement
Research Design

Cite this

@article{6e31c3eca53f43ed9cf9550a743be066,
title = "Do online information retrieval systems help experienced clinicians answer clinical questions?",
abstract = "Objective: To assess the impact of clinicians' use of an online information retrieval system on their performance in answering clinical questions. Design: Pre-/post-intervention experimental design. Measurements: In a computer laboratory, 75 clinicians (26 hospital-based doctors, 18 family practitioners, and 31 clinical nurse consultants) provided 600 answers to eight clinical scenarios before and after the use of an online information retrieval system. We examined the proportion of correct answers pre- and post-intervention, direction of change in answers, and differences between professional groups. Results: System use resulted in a 21{\%} improvement in clinicians' answers, from 29{\%} (95{\%} confidence interval [CI] 25.4-32.6) correct pre- to 50{\%} (95{\%} CI 46.0-54.0) post-system use. In 33{\%} (95{\%} CI 29.1-36.9) answers were changed from incorrect to correct. In 21{\%} (95{\%} CI 17.1-23.9) correct pre-test answers were supported by evidence found using the system, and in 7{\%} (95{\%} CI 4.9-9.1) correct pre-test answers were changed incorrectly. For 40{\%} (35.4-43.6) of scenarios, incorrect pre-test answers were not rectified following system use. Despite significant differences in professional groups' pre-test scores [family practitioners: 41{\%} (95{\%} CI 33.0-49.0), hospital doctors: 35{\%} (95{\%} CI 28.5-41.2), and clinical nurse consultants: 17{\%} (95{\%} CI 12.3-21.7; χ2 = 29.0, df = 2, p < 0.01)], there was no difference in post-test scores. (χ2 = 2.6, df = 2, p = 0.73). Conclusions: The use of an online information retrieval system was associated with a significant improvement in the quality of answers provided by clinicians to typical clinical problems. In a small proportion of cases, use of the system produced errors. While there was variation in the performance of clinical groups when answering questions unaided, performance did not differ significantly following system use. Online information retrieval systems can be an effective tool in improving the accuracy of clinicians' answers to clinical questions.",
author = "Westbrook, {Johanna I.} and Coiera, {Enrico W.} and Gosling, {A. Sophie}",
year = "2005",
month = "5",
doi = "10.1197/jamia.M1717",
language = "English",
volume = "12",
pages = "315--321",
journal = "Journal of the American Medical Informatics Association : JAMIA",
issn = "1067-5027",
publisher = "Oxford University Press",
number = "3",

}

Do online information retrieval systems help experienced clinicians answer clinical questions? / Westbrook, Johanna I.; Coiera, Enrico W.; Gosling, A. Sophie.

In: Journal of the American Medical Informatics Association, Vol. 12, No. 3, 05.2005, p. 315-321.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Do online information retrieval systems help experienced clinicians answer clinical questions?

AU - Westbrook, Johanna I.

AU - Coiera, Enrico W.

AU - Gosling, A. Sophie

PY - 2005/5

Y1 - 2005/5

N2 - Objective: To assess the impact of clinicians' use of an online information retrieval system on their performance in answering clinical questions. Design: Pre-/post-intervention experimental design. Measurements: In a computer laboratory, 75 clinicians (26 hospital-based doctors, 18 family practitioners, and 31 clinical nurse consultants) provided 600 answers to eight clinical scenarios before and after the use of an online information retrieval system. We examined the proportion of correct answers pre- and post-intervention, direction of change in answers, and differences between professional groups. Results: System use resulted in a 21% improvement in clinicians' answers, from 29% (95% confidence interval [CI] 25.4-32.6) correct pre- to 50% (95% CI 46.0-54.0) post-system use. In 33% (95% CI 29.1-36.9) answers were changed from incorrect to correct. In 21% (95% CI 17.1-23.9) correct pre-test answers were supported by evidence found using the system, and in 7% (95% CI 4.9-9.1) correct pre-test answers were changed incorrectly. For 40% (35.4-43.6) of scenarios, incorrect pre-test answers were not rectified following system use. Despite significant differences in professional groups' pre-test scores [family practitioners: 41% (95% CI 33.0-49.0), hospital doctors: 35% (95% CI 28.5-41.2), and clinical nurse consultants: 17% (95% CI 12.3-21.7; χ2 = 29.0, df = 2, p < 0.01)], there was no difference in post-test scores. (χ2 = 2.6, df = 2, p = 0.73). Conclusions: The use of an online information retrieval system was associated with a significant improvement in the quality of answers provided by clinicians to typical clinical problems. In a small proportion of cases, use of the system produced errors. While there was variation in the performance of clinical groups when answering questions unaided, performance did not differ significantly following system use. Online information retrieval systems can be an effective tool in improving the accuracy of clinicians' answers to clinical questions.

AB - Objective: To assess the impact of clinicians' use of an online information retrieval system on their performance in answering clinical questions. Design: Pre-/post-intervention experimental design. Measurements: In a computer laboratory, 75 clinicians (26 hospital-based doctors, 18 family practitioners, and 31 clinical nurse consultants) provided 600 answers to eight clinical scenarios before and after the use of an online information retrieval system. We examined the proportion of correct answers pre- and post-intervention, direction of change in answers, and differences between professional groups. Results: System use resulted in a 21% improvement in clinicians' answers, from 29% (95% confidence interval [CI] 25.4-32.6) correct pre- to 50% (95% CI 46.0-54.0) post-system use. In 33% (95% CI 29.1-36.9) answers were changed from incorrect to correct. In 21% (95% CI 17.1-23.9) correct pre-test answers were supported by evidence found using the system, and in 7% (95% CI 4.9-9.1) correct pre-test answers were changed incorrectly. For 40% (35.4-43.6) of scenarios, incorrect pre-test answers were not rectified following system use. Despite significant differences in professional groups' pre-test scores [family practitioners: 41% (95% CI 33.0-49.0), hospital doctors: 35% (95% CI 28.5-41.2), and clinical nurse consultants: 17% (95% CI 12.3-21.7; χ2 = 29.0, df = 2, p < 0.01)], there was no difference in post-test scores. (χ2 = 2.6, df = 2, p = 0.73). Conclusions: The use of an online information retrieval system was associated with a significant improvement in the quality of answers provided by clinicians to typical clinical problems. In a small proportion of cases, use of the system produced errors. While there was variation in the performance of clinical groups when answering questions unaided, performance did not differ significantly following system use. Online information retrieval systems can be an effective tool in improving the accuracy of clinicians' answers to clinical questions.

UR - http://www.scopus.com/inward/record.url?scp=18044392051&partnerID=8YFLogxK

U2 - 10.1197/jamia.M1717

DO - 10.1197/jamia.M1717

M3 - Article

VL - 12

SP - 315

EP - 321

JO - Journal of the American Medical Informatics Association : JAMIA

T2 - Journal of the American Medical Informatics Association : JAMIA

JF - Journal of the American Medical Informatics Association : JAMIA

SN - 1067-5027

IS - 3

ER -