Real-time event detection from the Twitter data stream using the TwitterNews+ framework

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Detecting events in real-time from the Twitter data stream has gained substantial attention in recent years from researchers around the world. Different event detection approaches have been proposed as a result of these research efforts. One of the major challenges faced in this context is the high computational cost associated with event detection in real-time. We propose, TwitterNews+, an event detection system that incorporates specialized inverted indices and an incremental clustering approach to provide a low computational cost solution to detect both major and minor newsworthy events in real-time from the Twitter data stream. In addition, we conduct an extensive parameter sensitivity analysis to fine-tune the parameters used in TwitterNews+ to achieve the best performance. Finally, we evaluate the effectiveness of our system using a publicly available corpus as a benchmark dataset. The results of the evaluation show a significant improvement in terms of recall and precision over five state-of-the-art baselines we have used.

LanguageEnglish
Pages1146-1165
Number of pages20
JournalInformation Processing and Management
Volume56
Issue number3
Early online date13 Mar 2018
DOIs
Publication statusPublished - May 2019

Fingerprint

twitter
event
Sensitivity analysis
Costs
costs
time
Twitter
Event detection
Data streams
evaluation
performance

Keywords

  • Event detection
  • Incremental clustering
  • Microblog
  • Parameter sensitivity analysis
  • Social media
  • Twitter

Cite this

@article{b302310ccf5f4ec4a936b179d04a82f3,
title = "Real-time event detection from the Twitter data stream using the TwitterNews+ framework",
abstract = "Detecting events in real-time from the Twitter data stream has gained substantial attention in recent years from researchers around the world. Different event detection approaches have been proposed as a result of these research efforts. One of the major challenges faced in this context is the high computational cost associated with event detection in real-time. We propose, TwitterNews+, an event detection system that incorporates specialized inverted indices and an incremental clustering approach to provide a low computational cost solution to detect both major and minor newsworthy events in real-time from the Twitter data stream. In addition, we conduct an extensive parameter sensitivity analysis to fine-tune the parameters used in TwitterNews+ to achieve the best performance. Finally, we evaluate the effectiveness of our system using a publicly available corpus as a benchmark dataset. The results of the evaluation show a significant improvement in terms of recall and precision over five state-of-the-art baselines we have used.",
keywords = "Event detection, Incremental clustering, Microblog, Parameter sensitivity analysis, Social media, Twitter",
author = "Mahmud Hasan and Orgun, {Mehmet A.} and Rolf Schwitter",
year = "2019",
month = "5",
doi = "10.1016/j.ipm.2018.03.001",
language = "English",
volume = "56",
pages = "1146--1165",
journal = "Information Processing and Management",
issn = "0306-4573",
publisher = "Elsevier",
number = "3",

}

Real-time event detection from the Twitter data stream using the TwitterNews+ framework. / Hasan, Mahmud; Orgun, Mehmet A.; Schwitter, Rolf.

In: Information Processing and Management, Vol. 56, No. 3, 05.2019, p. 1146-1165.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Real-time event detection from the Twitter data stream using the TwitterNews+ framework

AU - Hasan, Mahmud

AU - Orgun, Mehmet A.

AU - Schwitter, Rolf

PY - 2019/5

Y1 - 2019/5

N2 - Detecting events in real-time from the Twitter data stream has gained substantial attention in recent years from researchers around the world. Different event detection approaches have been proposed as a result of these research efforts. One of the major challenges faced in this context is the high computational cost associated with event detection in real-time. We propose, TwitterNews+, an event detection system that incorporates specialized inverted indices and an incremental clustering approach to provide a low computational cost solution to detect both major and minor newsworthy events in real-time from the Twitter data stream. In addition, we conduct an extensive parameter sensitivity analysis to fine-tune the parameters used in TwitterNews+ to achieve the best performance. Finally, we evaluate the effectiveness of our system using a publicly available corpus as a benchmark dataset. The results of the evaluation show a significant improvement in terms of recall and precision over five state-of-the-art baselines we have used.

AB - Detecting events in real-time from the Twitter data stream has gained substantial attention in recent years from researchers around the world. Different event detection approaches have been proposed as a result of these research efforts. One of the major challenges faced in this context is the high computational cost associated with event detection in real-time. We propose, TwitterNews+, an event detection system that incorporates specialized inverted indices and an incremental clustering approach to provide a low computational cost solution to detect both major and minor newsworthy events in real-time from the Twitter data stream. In addition, we conduct an extensive parameter sensitivity analysis to fine-tune the parameters used in TwitterNews+ to achieve the best performance. Finally, we evaluate the effectiveness of our system using a publicly available corpus as a benchmark dataset. The results of the evaluation show a significant improvement in terms of recall and precision over five state-of-the-art baselines we have used.

KW - Event detection

KW - Incremental clustering

KW - Microblog

KW - Parameter sensitivity analysis

KW - Social media

KW - Twitter

UR - http://www.scopus.com/inward/record.url?scp=85043452764&partnerID=8YFLogxK

U2 - 10.1016/j.ipm.2018.03.001

DO - 10.1016/j.ipm.2018.03.001

M3 - Article

VL - 56

SP - 1146

EP - 1165

JO - Information Processing and Management

T2 - Information Processing and Management

JF - Information Processing and Management

SN - 0306-4573

IS - 3

ER -