Automated classification of primary care patient safety incident report content and severity using supervised machine learning (ML) approaches

Huw Prosser Evans, Athanasios Anastasiou, Adrian Edwards, Peter Hibbert, Meredith Makeham, Saturnino Luz, Aziz Sheikh, Liam Donaldson, Andrew Carson-Stevens

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Learning from patient safety incident reports is a vital part of improving healthcare. However, the volume of reports and their largely free-text nature poses a major analytic challenge. The objective of this study was to test the capability of autonomous classifying of free text within patient safety incident reports to determine incident type and the severity of harm outcome. Primary care patient safety incident reports (n=31333) previously expert-categorised by clinicians (training data) were processed using J48, SVM and Naïve Bayes. The SVM classifier was the highest scoring classifier for incident type (AUROC, 0.891) and severity of harm (AUROC, 0.708). Incident reports containing deaths were most easily classified, correctly identifying 72.82% of reports. In conclusion, supervised ML can be used to classify patient safety incident report categories. The severity classifier, whilst not accurate enough to replace manual processing, could provide a valuable screening tool for this critical aspect of patient safety.

LanguageEnglish
JournalHealth Informatics Journal
DOIs
Publication statusE-pub ahead of print - 7 Mar 2019

Fingerprint

Patient Safety
Primary Health Care
Supervised Machine Learning
Learning
Delivery of Health Care

Keywords

  • incident reporting
  • machine learning
  • natural language processing
  • patient safety
  • quality improvement

Cite this

Evans, Huw Prosser ; Anastasiou, Athanasios ; Edwards, Adrian ; Hibbert, Peter ; Makeham, Meredith ; Luz, Saturnino ; Sheikh, Aziz ; Donaldson, Liam ; Carson-Stevens, Andrew. / Automated classification of primary care patient safety incident report content and severity using supervised machine learning (ML) approaches. In: Health Informatics Journal. 2019.
@article{277fea26ecfb434bbd9603b2ed2c0ce2,
title = "Automated classification of primary care patient safety incident report content and severity using supervised machine learning (ML) approaches",
abstract = "Learning from patient safety incident reports is a vital part of improving healthcare. However, the volume of reports and their largely free-text nature poses a major analytic challenge. The objective of this study was to test the capability of autonomous classifying of free text within patient safety incident reports to determine incident type and the severity of harm outcome. Primary care patient safety incident reports (n=31333) previously expert-categorised by clinicians (training data) were processed using J48, SVM and Na{\"i}ve Bayes. The SVM classifier was the highest scoring classifier for incident type (AUROC, 0.891) and severity of harm (AUROC, 0.708). Incident reports containing deaths were most easily classified, correctly identifying 72.82{\%} of reports. In conclusion, supervised ML can be used to classify patient safety incident report categories. The severity classifier, whilst not accurate enough to replace manual processing, could provide a valuable screening tool for this critical aspect of patient safety.",
keywords = "incident reporting, machine learning, natural language processing, patient safety, quality improvement",
author = "Evans, {Huw Prosser} and Athanasios Anastasiou and Adrian Edwards and Peter Hibbert and Meredith Makeham and Saturnino Luz and Aziz Sheikh and Liam Donaldson and Andrew Carson-Stevens",
year = "2019",
month = "3",
day = "7",
doi = "10.1177/1460458219833102",
language = "English",
journal = "Health Informatics Journal",
issn = "1460-4582",
publisher = "SAGE Publications",

}

Automated classification of primary care patient safety incident report content and severity using supervised machine learning (ML) approaches. / Evans, Huw Prosser; Anastasiou, Athanasios; Edwards, Adrian; Hibbert, Peter; Makeham, Meredith; Luz, Saturnino; Sheikh, Aziz; Donaldson, Liam; Carson-Stevens, Andrew.

In: Health Informatics Journal, 07.03.2019.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Automated classification of primary care patient safety incident report content and severity using supervised machine learning (ML) approaches

AU - Evans, Huw Prosser

AU - Anastasiou, Athanasios

AU - Edwards, Adrian

AU - Hibbert, Peter

AU - Makeham, Meredith

AU - Luz, Saturnino

AU - Sheikh, Aziz

AU - Donaldson, Liam

AU - Carson-Stevens, Andrew

PY - 2019/3/7

Y1 - 2019/3/7

N2 - Learning from patient safety incident reports is a vital part of improving healthcare. However, the volume of reports and their largely free-text nature poses a major analytic challenge. The objective of this study was to test the capability of autonomous classifying of free text within patient safety incident reports to determine incident type and the severity of harm outcome. Primary care patient safety incident reports (n=31333) previously expert-categorised by clinicians (training data) were processed using J48, SVM and Naïve Bayes. The SVM classifier was the highest scoring classifier for incident type (AUROC, 0.891) and severity of harm (AUROC, 0.708). Incident reports containing deaths were most easily classified, correctly identifying 72.82% of reports. In conclusion, supervised ML can be used to classify patient safety incident report categories. The severity classifier, whilst not accurate enough to replace manual processing, could provide a valuable screening tool for this critical aspect of patient safety.

AB - Learning from patient safety incident reports is a vital part of improving healthcare. However, the volume of reports and their largely free-text nature poses a major analytic challenge. The objective of this study was to test the capability of autonomous classifying of free text within patient safety incident reports to determine incident type and the severity of harm outcome. Primary care patient safety incident reports (n=31333) previously expert-categorised by clinicians (training data) were processed using J48, SVM and Naïve Bayes. The SVM classifier was the highest scoring classifier for incident type (AUROC, 0.891) and severity of harm (AUROC, 0.708). Incident reports containing deaths were most easily classified, correctly identifying 72.82% of reports. In conclusion, supervised ML can be used to classify patient safety incident report categories. The severity classifier, whilst not accurate enough to replace manual processing, could provide a valuable screening tool for this critical aspect of patient safety.

KW - incident reporting

KW - machine learning

KW - natural language processing

KW - patient safety

KW - quality improvement

UR - http://www.scopus.com/inward/record.url?scp=85062702543&partnerID=8YFLogxK

U2 - 10.1177/1460458219833102

DO - 10.1177/1460458219833102

M3 - Article

JO - Health Informatics Journal

T2 - Health Informatics Journal

JF - Health Informatics Journal

SN - 1460-4582

ER -