Automated categorisation of clinical incident reports using statistical text classification

Mei Sing Ong*, Farah Magrabi, Enrico Coiera

*Corresponding author for this work

Research output: Contribution to journalArticle

31 Citations (Scopus)

Abstract

Objectives: To explore the feasibility of using statistical text classification techniques to automatically categorise clinical incident reports. Methods Statistical text classifiers based on Naïve Bayes and Support Vector Machine algorithms were trained and tested on incident reports submitted by public hospitals to identify two classes of clinical incidents: Inadequate clinical handover and incorrect patient identification. Each classifier was trained on 600 reports (300 positives, 300 negatives), and tested on 372 reports (248 positives, 124 negatives). The results were evaluated using standard measures of accuracy, precision, recall, F-measure and area under curve (AUC) of receiver operating characteristics (ROC). Classifier learning rates were also evaluated, using classifier accuracy against training set size. Results All classifiers performed well in categorising clinical handover and patient identification incidents. Naïve Bayes attained the best performance on handover incidents, correctly identifying 86.29% of reporterclassified incidents (precision=0.84, recall=0.90, F-measure=0.87, AUC=0.93) and 91.53% of expertclassified incidents (precision=0.87, recall=0.98, F-measure=0.92, AUC=0.97). For patient identification incidents, the best results were obtained when Support Vector Machine with radial-basis function kernel was used to classify reporter-classified reports (accuracy=97.98%, precision=0.98, recall=0.98, F-measure=0.98, AUC=1.00); and when Naïve Bayes was used on expert-classified reports (accuracy=95.97%, precision=0.95, recall=0.98, F-measure=0.96, AUC=0.99). A relatively small training set was found to be adequate, with most classifiers achieving an accuracy above 80% when the training set size was as small as 100 samples. Conclusions: This study demonstrates the feasibility of using text classification techniques to automatically categorise clinical incident reports.

Original languageEnglish
Article numbere55
Pages (from-to)1-7
Number of pages7
JournalQuality and Safety in Health Care
Volume19
Issue number6
DOIs
Publication statusPublished - Dec 2010
Externally publishedYes

Fingerprint Dive into the research topics of 'Automated categorisation of clinical incident reports using statistical text classification'. Together they form a unique fingerprint.

Cite this