Joint operator scaling and placement for Distributed Stream Processing applications in edge computing

Qinglan Peng, Yunni Xia, Yan Wang, Chunrong Wu, Xin Luo, Jia Lee

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

3 Citations (Scopus)

Abstract

Distributed Stream Processing (DSP) systems are well acknowledged to be potent in processing huge volume of real-time stream data with low latency and high throughput. Recently, the edge computing paradigm shows great potentials in supporting and boosting the DSP applications, especially the time-critical and latency-sensitive ones, over the Internet of Things (IoT) or mobile devices by means of offloading the computation from remote cloud to edge servers for further reduced communication latencies. Nevertheless, various challenges, especially the joint operator scaling and placement, are yet to be properly explored and addressed. Traditional efforts in this direction usually assume that the data-flow graph of a DSP application is pre-given and static. The resulting models and methods can thus be ineffective and show bad user-perceived quality-of-service (QoS) when dealing with real-world scenarios with reconfigurable data-flow graphs and scalable operator placement. In contrast, in this paper, we consider that the data-flow graphs are configurable and hence propose the joint operator scaling and placement problem. To address this problem, we first build a queuing-network-based QoS estimation model, then formulate the problem into an integer-programming one, and finally propose a two-stage approach for finding the near-optimal solution. Experiments based on real-world DSP test cases show that our method achieves higher cost effectiveness than traditional ones while meeting the user-defined QoS constraints.
Original languageEnglish
Title of host publicationService-Oriented Computing
Subtitle of host publication17th International Conference, ICSOC 2019, Proceedings
EditorsSami Yangui, Ismael Bouassida Rodriguez, Khalil Drira, Zahir Tari
Place of PublicationCham
PublisherSpringer, Springer Nature
Pages461-476
Number of pages16
ISBN (Electronic)9783030337025
ISBN (Print)9783030337018
DOIs
Publication statusPublished - 2019
Event17th International Conference on Service-Oriented Computing, ICSOC 2019 - Toulouse, France
Duration: 28 Oct 201931 Oct 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11895 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference17th International Conference on Service-Oriented Computing, ICSOC 2019
CountryFrance
CityToulouse
Period28/10/1931/10/19

Keywords

  • edge computing
  • distributed stream processing
  • operator placement
  • operator replication
  • Operator placement
  • Operator replication
  • Distributed stream processing
  • Edge computing

Fingerprint Dive into the research topics of 'Joint operator scaling and placement for Distributed Stream Processing applications in edge computing'. Together they form a unique fingerprint.

Cite this