An experimental investigation of the impact of alert frequency and relevance on alert dwell time

Melissa T. Baysari, Wu Yi Zheng, Amina Tariq, Maureen Heywood, Grace Scott, Ling Li, Bethany A. Van Dort, Kasun Rathnayake, Richard Day, Johanna I. Westbrook

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Aim: This study aimed to determine the impact of alert frequency and relevance on alert dwell time. Method: A 2 × 3 design was used where 127 university students completed 60 prescribing tasks and were presented with a variable frequency of computerized alerts (low, medium and high) with variable relevance (low and high). Participants were instructed to override an alert if it was not relevant to their prescription, and to cancel the order if the alert signalled an error in their order. Results: Participants presented with a small number of alerts spent more time attending to alert content than participants presented with a medium or high number of alerts (respectively median 15.6 s vs 10.8 vs 10.2 s). Alert relevance had no impact on alert dwell time. Alerts requiring an override response were 4.5 times more likely to be correctly actioned than alerts requiring the order to be cancelled. Discussion: Dwell time was influenced by alert frequency, with greater exposure to alerts associated with shorter dwell times. We hypothesize that this was because participants came to learn that spending time on alert information was unnecessary. We propose that when users experience no consequences or feedback from overriding alerts they quickly learn that this action is more efficient and so more rewarding than taking any other action.

LanguageEnglish
Article number104027
Pages1-4
Number of pages4
JournalInternational Journal of Medical Informatics
Volume133
DOIs
Publication statusPublished - 1 Jan 2020

Fingerprint

varespladib methyl
Prescriptions
Students

Keywords

  • Alert
  • Alert dwell time
  • CPOE

Cite this

@article{6f72ebe2d9524cfd951b0a59788f560e,
title = "An experimental investigation of the impact of alert frequency and relevance on alert dwell time",
abstract = "Aim: This study aimed to determine the impact of alert frequency and relevance on alert dwell time. Method: A 2 × 3 design was used where 127 university students completed 60 prescribing tasks and were presented with a variable frequency of computerized alerts (low, medium and high) with variable relevance (low and high). Participants were instructed to override an alert if it was not relevant to their prescription, and to cancel the order if the alert signalled an error in their order. Results: Participants presented with a small number of alerts spent more time attending to alert content than participants presented with a medium or high number of alerts (respectively median 15.6 s vs 10.8 vs 10.2 s). Alert relevance had no impact on alert dwell time. Alerts requiring an override response were 4.5 times more likely to be correctly actioned than alerts requiring the order to be cancelled. Discussion: Dwell time was influenced by alert frequency, with greater exposure to alerts associated with shorter dwell times. We hypothesize that this was because participants came to learn that spending time on alert information was unnecessary. We propose that when users experience no consequences or feedback from overriding alerts they quickly learn that this action is more efficient and so more rewarding than taking any other action.",
keywords = "Alert, Alert dwell time, CPOE",
author = "Baysari, {Melissa T.} and Zheng, {Wu Yi} and Amina Tariq and Maureen Heywood and Grace Scott and Ling Li and {Van Dort}, {Bethany A.} and Kasun Rathnayake and Richard Day and Westbrook, {Johanna I.}",
year = "2020",
month = "1",
day = "1",
doi = "10.1016/j.ijmedinf.2019.104027",
language = "English",
volume = "133",
pages = "1--4",
journal = "International Journal of Medical Informatics",
issn = "1386-5056",
publisher = "Elsevier",

}

An experimental investigation of the impact of alert frequency and relevance on alert dwell time. / Baysari, Melissa T.; Zheng, Wu Yi; Tariq, Amina; Heywood, Maureen; Scott, Grace; Li, Ling; Van Dort, Bethany A.; Rathnayake, Kasun; Day, Richard; Westbrook, Johanna I.

In: International Journal of Medical Informatics, Vol. 133, 104027, 01.01.2020, p. 1-4.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - An experimental investigation of the impact of alert frequency and relevance on alert dwell time

AU - Baysari, Melissa T.

AU - Zheng, Wu Yi

AU - Tariq, Amina

AU - Heywood, Maureen

AU - Scott, Grace

AU - Li, Ling

AU - Van Dort, Bethany A.

AU - Rathnayake, Kasun

AU - Day, Richard

AU - Westbrook, Johanna I.

PY - 2020/1/1

Y1 - 2020/1/1

N2 - Aim: This study aimed to determine the impact of alert frequency and relevance on alert dwell time. Method: A 2 × 3 design was used where 127 university students completed 60 prescribing tasks and were presented with a variable frequency of computerized alerts (low, medium and high) with variable relevance (low and high). Participants were instructed to override an alert if it was not relevant to their prescription, and to cancel the order if the alert signalled an error in their order. Results: Participants presented with a small number of alerts spent more time attending to alert content than participants presented with a medium or high number of alerts (respectively median 15.6 s vs 10.8 vs 10.2 s). Alert relevance had no impact on alert dwell time. Alerts requiring an override response were 4.5 times more likely to be correctly actioned than alerts requiring the order to be cancelled. Discussion: Dwell time was influenced by alert frequency, with greater exposure to alerts associated with shorter dwell times. We hypothesize that this was because participants came to learn that spending time on alert information was unnecessary. We propose that when users experience no consequences or feedback from overriding alerts they quickly learn that this action is more efficient and so more rewarding than taking any other action.

AB - Aim: This study aimed to determine the impact of alert frequency and relevance on alert dwell time. Method: A 2 × 3 design was used where 127 university students completed 60 prescribing tasks and were presented with a variable frequency of computerized alerts (low, medium and high) with variable relevance (low and high). Participants were instructed to override an alert if it was not relevant to their prescription, and to cancel the order if the alert signalled an error in their order. Results: Participants presented with a small number of alerts spent more time attending to alert content than participants presented with a medium or high number of alerts (respectively median 15.6 s vs 10.8 vs 10.2 s). Alert relevance had no impact on alert dwell time. Alerts requiring an override response were 4.5 times more likely to be correctly actioned than alerts requiring the order to be cancelled. Discussion: Dwell time was influenced by alert frequency, with greater exposure to alerts associated with shorter dwell times. We hypothesize that this was because participants came to learn that spending time on alert information was unnecessary. We propose that when users experience no consequences or feedback from overriding alerts they quickly learn that this action is more efficient and so more rewarding than taking any other action.

KW - Alert

KW - Alert dwell time

KW - CPOE

UR - http://www.scopus.com/inward/record.url?scp=85074409936&partnerID=8YFLogxK

UR - http://purl.org/au-research/grants/nhmrc/1054146

U2 - 10.1016/j.ijmedinf.2019.104027

DO - 10.1016/j.ijmedinf.2019.104027

M3 - Article

VL - 133

SP - 1

EP - 4

JO - International Journal of Medical Informatics

T2 - International Journal of Medical Informatics

JF - International Journal of Medical Informatics

SN - 1386-5056

M1 - 104027

ER -