Reduce or remove: individual sensor reliability profiling and data cleaning

Yihong Zhang*, Claudia Szabo, Quan Z. Sheng

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

3 Citations (Scopus)


Environmental sensing using multitudes of wirelessly connected sensors is becoming critical for resolving environmental problems, given recent technology advances in the Internet of Things (IoT). Current environmental sensing projects typically deploy commodity sensors, which are known to be unreliable and prone to produce noisy and erroneous data. Moreover, the majority of current sensor data cleaning techniques have not moved beyond using the mean or the median of spatially correlated readings, thus providing unsatisfying accuracies. In this paper, we propose a sensor reliability-based cleaning method, called Influence Mean (IM), which uses weighted aggregation based on individual sensor reliabilities. We investigate whether reducing or removing unreliable sensors can be more effective to provide accurate cleaning results, by designing and testing respective algorithms on synthetic and real datasets. The experimental results show that our method generally improves the data cleaning accuracy, particularly when the behaviors of unreliable sensors vary drastically from reliable sensors.

Original languageEnglish
Pages (from-to)979-995
Number of pages17
JournalIntelligent Data Analysis
Issue number5
Publication statusPublished - 2016
Externally publishedYes


  • Data cleaning
  • environmental sensing
  • internet of things


Dive into the research topics of 'Reduce or remove: individual sensor reliability profiling and data cleaning'. Together they form a unique fingerprint.

Cite this