TY - GEN
T1 - Measuring the impact of online evidence retrieval systems using critical incidents & journey mapping
AU - Westbrook, Johanna I.
AU - Coiera, Enrico W.
AU - Braithwaite, Jeffrey
PY - 2005
Y1 - 2005
N2 - Online evidence retrieval systems are one potential tool in supporting evidencebased practice. We have undertaken a program of research to investigate how hospital-based clinicians (doctors, nurses and allied health professionals) use these systems, factors influencing use and their impact on decision-making and health care delivery. A central component of this work has been the development and testing of a broad range of evaluation techniques. This paper provides an overview of the results obtained from three stages of this evaluation and details the results derived from the final stage which sought to test two methods for assessing the integration of an online evidence system and its impact on decision making and patient care. The critical incident and journey mapping techniques were applied. Semi-structured interviews were conducted with 29 clinicians who were experienced users of the online evidence system. Clinicians were asked to described recent instances in which the information obtained using the online evidence system was especially helpful with their work. A grounded approach to data analysis was taken producing three categories of impact. The journey mapping technique was adapted as a method to describe and quantify clinicians' integration of CIAP into their practice and the impact of this on patient care. The analogy of a journey is used to capture the many stages in this integration process, from introduction to the system to full integration into everyday clinical practice with measurable outcomes. Transcribed interview accounts of system use were mapped against the journey stages and scored. Clinicians generated 85 critical incidents and one quarter of these provided specific examples of system use leading to improvements in patient care. The journey mapping technique proved to be a useful method for providing a quantification of the ways and extent to which clincians had integrated system use into practice, and insights into how information systems can influence organisational culture. Further work is required on this technique to assess its value as an evaluation method. The study demonstrates the strength of a triangulated evidence approach to assessing the use and impact of online clinical evidence systems.
AB - Online evidence retrieval systems are one potential tool in supporting evidencebased practice. We have undertaken a program of research to investigate how hospital-based clinicians (doctors, nurses and allied health professionals) use these systems, factors influencing use and their impact on decision-making and health care delivery. A central component of this work has been the development and testing of a broad range of evaluation techniques. This paper provides an overview of the results obtained from three stages of this evaluation and details the results derived from the final stage which sought to test two methods for assessing the integration of an online evidence system and its impact on decision making and patient care. The critical incident and journey mapping techniques were applied. Semi-structured interviews were conducted with 29 clinicians who were experienced users of the online evidence system. Clinicians were asked to described recent instances in which the information obtained using the online evidence system was especially helpful with their work. A grounded approach to data analysis was taken producing three categories of impact. The journey mapping technique was adapted as a method to describe and quantify clinicians' integration of CIAP into their practice and the impact of this on patient care. The analogy of a journey is used to capture the many stages in this integration process, from introduction to the system to full integration into everyday clinical practice with measurable outcomes. Transcribed interview accounts of system use were mapped against the journey stages and scored. Clinicians generated 85 critical incidents and one quarter of these provided specific examples of system use leading to improvements in patient care. The journey mapping technique proved to be a useful method for providing a quantification of the ways and extent to which clincians had integrated system use into practice, and insights into how information systems can influence organisational culture. Further work is required on this technique to assess its value as an evaluation method. The study demonstrates the strength of a triangulated evidence approach to assessing the use and impact of online clinical evidence systems.
UR - http://www.scopus.com/inward/record.url?scp=69849109890&partnerID=8YFLogxK
M3 - Conference proceeding contribution
C2 - 16160312
AN - SCOPUS:69849109890
SN - 1586035495
SN - 9781586035495
VL - 116
T3 - Studies in Health Technology and Informatics
SP - 533
EP - 538
BT - Connecting Medical Informatics and Bio-Informatics - Proceedings of MIE 2005
A2 - Engelbrecht, Rolf
A2 - Geissbuhler, Antoine
A2 - Lovis, Christian
A2 - Gmihalas, Mihalas
PB - IOS Press
CY - Geneva, Switzerland
T2 - 19th International Congress of the European Federation for Medical Informatics, MIE 2005
Y2 - 28 August 2005 through 1 September 2005
ER -