Syndromic surveillance for health information system failures: A feasibility study

Mei Sing Ong*, Farah Magrabi, Enrico Coiera

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

12 Citations (Scopus)


Objective: To explore the applicability of a syndromic surveillance method to the early detection of health information technology (HIT) system failures. Methods: A syndromic surveillance system was developed to monitor a laboratory information system at a tertiary hospital. Four indices were monitored: (1) total laboratory records being created; (2) total records with missing results; (3) average serum potassium results; and (4) total duplicated tests on a patient. The goal was to detect HIT system failures causing: data loss at the record level; data loss at the field level; erroneous data; and unintended duplication of data. Time-series models of the indices were constructed, and statistical process control charts were used to detect unexpected behaviors. The ability of the models to detect HIT system failures was evaluated using simulated failures, each lasting for 24 h, with error rates ranging from 1% to 35%. Results: In detecting data loss at the record level, the model achieved a sensitivity of 0.26 when the simulated error rate was 1%, while maintaining a specificity of 0.98. Detection performance improved with increasing error rates, achieving a perfect sensitivity when the error rate was 35%. In the detection of missing results, erroneous serum potassium results and unintended repetition of tests, perfect sensitivity was attained when the error rate was as small as 5%. Decreasing the error rate to 1% resulted in a drop in sensitivity to 0.65-0.85. Conclusions: Syndromic surveillance methods can potentially be applied to monitor HIT systems, to facilitate the early detection of failures.

Original languageEnglish
Pages (from-to)506-512
Number of pages7
JournalJournal of the American Medical Informatics Association
Issue number3
Publication statusPublished - 2013
Externally publishedYes


Dive into the research topics of 'Syndromic surveillance for health information system failures: A feasibility study'. Together they form a unique fingerprint.

Cite this