Detecting technical anomalies in high-frequency water-quality data using Artificial Neural Networks

Javier Rodriguez-Perez, Catherine Leigh, Benoit Liquet, Claire Kermorvant, Erin Peterson, Damien Sous, Kerrie Mengersen*

*Corresponding author for this work

    Research output: Contribution to journalArticlepeer-review

    8 Citations (Scopus)

    Abstract

    Anomaly detection (AD) in high-volume environmental data requires one to tackle a series of challenges associated with the typical low frequency of anomalous events, the broad-range of possible anomaly types and local non-stationary environmental conditions, suggesting the need for flexible statistical methods that are able to cope with unbalanced high-volume data problems. Here, we aimed to detect anomalies caused by technical errors in water-quality (turbidity and conductivity) data collected by automated in-situ sensors deployed in contrasting riverine and estuarine environments. We first applied a range of Artificial Neural Networks (ANN) that differed in both learning method and hyper-parameter values, then calibrated models using a Bayesian multi-objective optimisation procedure, and selected and evaluated the "best" model for each water-quality variable, environment and anomaly type. We found that semi-supervised classification was better able to detect sudden spikes, sudden shifts and small sudden spikes whereas supervised classification had higher accuracy for predicting long-term anomalies associated with drifts and periods of otherwise unexplained high-variability.

    Original languageEnglish
    Pages (from-to)13719-13730
    Number of pages12
    JournalEnvironmental Science and Technology
    Volume54
    Issue number21
    Early online date28 Aug 2020
    DOIs
    Publication statusPublished - 3 Nov 2020

    Fingerprint

    Dive into the research topics of 'Detecting technical anomalies in high-frequency water-quality data using Artificial Neural Networks'. Together they form a unique fingerprint.

    Cite this