A time efficient approach for detecting errors in big sensor data on cloud

Chi Yang*, Chang Liu, Xuyun Zhang, Surya Nepal, Jinjun Chen

*Corresponding author for this work

Research output: Contribution to journalArticle

63 Citations (Scopus)

Abstract

Big sensor data is prevalent in both industry and scientific research applications where the data is generated with high volume and velocity it is difficult to process using on-hand database management tools or traditional data processing applications. Cloud computing provides a promising platform to support the addressing of this challenge as it provides a flexible stack of massive computing, storage, and software services in a scalable manner at low cost. Some techniques have been developed in recent years for processing sensor data on cloud, such as sensor-cloud. However, these techniques do not provide efficient support on fast detection and locating of errors in big sensor data sets. For fast data error detection in big sensor data sets, in this paper, we develop a novel data error detection approach which exploits the full computation potential of cloud platform and the network feature of WSN. Firstly, a set of sensor data error types are classified and defined. Based on that classification, the network feature of a clustered WSN is introduced and analyzed to support fast error detection and location. Specifically, in our proposed approach, the error detection is based on the scale-free network topology and most of detection operations can be conducted in limited temporal or spatial data blocks instead of a whole big data set. Hence the detection and location process can be dramatically accelerated. Furthermore, the detection and location tasks can be distributed to cloud platform to fully exploit the computation power and massive storage. Through the experiment on our cloud computing platform of U-Cloud, it is demonstrated that our proposed approach can significantly reduce the time for error detection and location in big data sets generated by large scale sensor network systems with acceptable error detecting accuracy.

Original languageEnglish
Pages (from-to)329-339
Number of pages11
JournalIEEE Transactions on Parallel and Distributed Systems
Volume26
Issue number2
DOIs
Publication statusPublished - Feb 2015
Externally publishedYes

Keywords

  • Big data
  • cloud computing
  • data abnormality
  • error detection
  • time efficiency
  • sensor networks
  • complex network systems

Cite this