Automated identification of extreme-risk events in clinical incident reports

Mei Sing Ong*, Farah Magrabi, Enrico Coiera

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

31 Citations (Scopus)


Objectives: To explore the feasibility of using statistical text classification to automatically detect extreme-risk events in clinical incident reports. Methods: Statistical text classifiers based on Naïve Bayes and Support Vector Machine (SVM) algorithms were trained and tested on clinical incident reports to automatically detect extreme-risk events, defined by incidents that satisfy the criteria of Severity Assessment Code (SAC) level 1. For this purpose, incident reports submitted to the Advanced Incident Management System by public hospitals from one Australian region were used. The classifiers were evaluated on two datasets: (1) a set of reports with diverse incident types (n1/4 120); (2) a set of reports associated with patient misidentification (n1/4 166). Results were assessed using accuracy, precision, recall, F-measure, and area under the curve (AUC) of receiver operating characteristic curves. Results: The classifiers performed well on both datasets. In the multi-type dataset, SVM with a linear kernel performed best, identifying 85.8% of SAC level 1 incidents (precision1/4 0.88, recall1/4 0.83, F-measure1/4 0.86, AUC=0.92). In the patient misidentification dataset, 96.4% of SAC level 1 incidents were detected when SVM with linear, polynomial or radial-basis function kernel was used (precision1/4 0.99, recall1/4 0.94, F-measure1/4 0.96, AUC1/4 0.98). Naïve Bayes showed reasonable performance, detecting 80.8% of SAC level 1 incidents in the multi-type dataset and 89.8% of SAC level 1 patient misidentification incidents. Overall, higher prediction accuracy was attained on the specialized dataset, compared with the multi-type dataset. Conclusion: Text classification techniques can be applied effectively to automate the detection of extremerisk events in clinical incident reports.

Original languageEnglish
Pages (from-to)e110-e118
Number of pages9
JournalJournal of the American Medical Informatics Association
Issue numberE1
Publication statusPublished - Jun 2012
Externally publishedYes


Dive into the research topics of 'Automated identification of extreme-risk events in clinical incident reports'. Together they form a unique fingerprint.

Cite this