Significance-based failure and interference detection in data streams

Nickolas J G Falkner, Quan Z. Sheng

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review


Detecting the failure of a data stream is relatively easy when the stream is continually full of data. The transfer of large amounts of data allows for the simple detection of interference, whether accidental or malicious. However, during interference, data transmission can become irregular, rather than smooth. When the traffic is intermittent, it is harder to detect when failure has occurred and may lead to an application at the receiving end requesting retransmission or disconnecting. Request retransmission places additional load on a system and disconnection can lead to unnecessary reversion to a checkpointed database, before reconnecting and reissuing the same request or response. In this paper, we model the traffic in data streams as a set of significant events, with an arrival rate distributed with a Poisson distribution. Once an arrival rate has been determined, over-time, or lost, events can be determined with a greater chance of reliability. This model also allows for the alteration of the rate parameter to reflect changes in the system and provides support for multiple levels of data aggregation. One significant benefit of the Poisson-based model is that transmission events can be deliberately manipulated in time to provide a steganographic channel that confirms sender/receiver identity.

Original languageEnglish
Title of host publicationDatabase and Expert Systems Applications - 20th International Conference, DEXA 2009, Proceedings
Number of pages15
Volume5690 LNCS
Publication statusPublished - 2009
Externally publishedYes
Event20th International Conference on Database and Expert Systems Applications, DEXA 2009 - Linz, Austria
Duration: 31 Aug 20094 Sept 2009

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume5690 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349


Other20th International Conference on Database and Expert Systems Applications, DEXA 2009


Dive into the research topics of 'Significance-based failure and interference detection in data streams'. Together they form a unique fingerprint.

Cite this