Robust scheduling for large-scale distributed systems

Young Choon Lee, Jayden King, Young Ki Kim, Seok-Hee Hong

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

Abstract

In large-scale distributed systems, such as clouds, failures are rather the norm than the exception. These failures include job failures, server failures, network outage and power failure. Among them, server failures are most common. With the wide adoption of cloud computing, the impact of server failures in clouds is far greater than that in traditional computer clusters as jobs of different tenants are often co-located (multi-tenancy). In this paper, we address the problem of robust scheduling, with realistic failure modeling, to minimize such impact on the execution of (co-located) jobs. To this end, we develop four online failure-aware (FA) scheduling algorithms, FAFF-WJ, FAFF-FC, FABF-WJ and FABF-FC, considering the availability and reliability of servers. In particular, FF (First-Fit) and BF (Best-Fit) indicate how the availability of servers is checked while WJ (Waiting Job) and FC (Failure Count) differ primarily in whether the reliability is measured from job's perspective or server's perspective. All four algorithms are designed essentially by combining these availability and reliability check methods. We evaluate our scheduling algorithms with failures generated based on our failure modeling of six real-world server failure traces. Our evaluation results show the effectiveness of our scheduling algorithms in robust job execution, with respect to both performance and cost.

Original languageEnglish
Title of host publicationProceedings - 2020 IEEE 19th International Conference on Trust, Security and Privacy in Computing and Communications, TrustCom 2020
EditorsGuojun Wang, Ryan Ko, Md Zakirul Alam Bhuiyan, Yi Pan
Place of PublicationPiscataway, NJ
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages38-45
Number of pages8
ISBN (Electronic)9781665403924
ISBN (Print)9780738143804
DOIs
Publication statusPublished - 2020
Event19th IEEE International Conference on Trust, Security and Privacy in Computing and Communications, TrustCom 2020 - Guangzhou, China
Duration: 29 Dec 20201 Jan 2021

Publication series

NameIEEE International Conference on Trust Security and Privacy in Computing and Communications
PublisherIEEE COMPUTER SOC
ISSN (Print)2324-898X

Conference

Conference19th IEEE International Conference on Trust, Security and Privacy in Computing and Communications, TrustCom 2020
CountryChina
CityGuangzhou
Period29/12/201/01/21

Keywords

  • Clouds
  • Reliability
  • Robust scheduling
  • Server failures

Fingerprint

Dive into the research topics of 'Robust scheduling for large-scale distributed systems'. Together they form a unique fingerprint.

Cite this