Measuring the impact of online evidence retrieval systems using critical incidents & journey mapping

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

1 Citation (Scopus)


Online evidence retrieval systems are one potential tool in supporting evidencebased practice. We have undertaken a program of research to investigate how hospital-based clinicians (doctors, nurses and allied health professionals) use these systems, factors influencing use and their impact on decision-making and health care delivery. A central component of this work has been the development and testing of a broad range of evaluation techniques. This paper provides an overview of the results obtained from three stages of this evaluation and details the results derived from the final stage which sought to test two methods for assessing the integration of an online evidence system and its impact on decision making and patient care. The critical incident and journey mapping techniques were applied. Semi-structured interviews were conducted with 29 clinicians who were experienced users of the online evidence system. Clinicians were asked to described recent instances in which the information obtained using the online evidence system was especially helpful with their work. A grounded approach to data analysis was taken producing three categories of impact. The journey mapping technique was adapted as a method to describe and quantify clinicians' integration of CIAP into their practice and the impact of this on patient care. The analogy of a journey is used to capture the many stages in this integration process, from introduction to the system to full integration into everyday clinical practice with measurable outcomes. Transcribed interview accounts of system use were mapped against the journey stages and scored. Clinicians generated 85 critical incidents and one quarter of these provided specific examples of system use leading to improvements in patient care. The journey mapping technique proved to be a useful method for providing a quantification of the ways and extent to which clincians had integrated system use into practice, and insights into how information systems can influence organisational culture. Further work is required on this technique to assess its value as an evaluation method. The study demonstrates the strength of a triangulated evidence approach to assessing the use and impact of online clinical evidence systems.

Original languageEnglish
Title of host publicationConnecting Medical Informatics and Bio-Informatics - Proceedings of MIE 2005
Subtitle of host publicationThe 19th International Congress of the European Federation for Medical Informatics
EditorsRolf Engelbrecht, Antoine Geissbuhler, Christian Lovis, Mihalas Gmihalas
Place of PublicationGeneva, Switzerland
PublisherIOS Press
Number of pages6
ISBN (Print)1586035495, 9781586035495
Publication statusPublished - 2005
Externally publishedYes
Event19th International Congress of the European Federation for Medical Informatics, MIE 2005 - Geneva, Switzerland
Duration: 28 Aug 20051 Sep 2005

Publication series

Name Studies in Health Technology and Informatics
PublisherIOS Press
ISSN (Print)0926-9630
ISSN (Electronic)1879-8365


Other19th International Congress of the European Federation for Medical Informatics, MIE 2005


Dive into the research topics of 'Measuring the impact of online evidence retrieval systems using critical incidents & journey mapping'. Together they form a unique fingerprint.

Cite this