Holistic approach for studying resource failures at scale

Young Choon Lee; Jayden King; Seok-Hee Hong

doi:10.1109/NCA.2019.8935032

Holistic approach for studying resource failures at scale

Young Choon Lee, Jayden King, Seok-Hee Hong

School of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

2 Citations (Scopus)

Abstract

In large-scale distributed systems, such as data centers resource failures are the norm rather than an exception. In this paper, we propose a holistic approach to study resource failures from resource failure modelling to distributed system simulation to failure-aware scheduling algorithm design. In particular, we present (1) a simple and yet practical way to model resource failures using real-world failure traces, (2) a new distributed systems simulator and (3) two failure-aware scheduling algorithms. These scheduling algorithms are designed primarily to validate (1) and (2). Our evaluation results demonstrate the feasibility and effectiveness of our holistic approach.

Original language	English
Title of host publication	2019 IEEE 18th International Symposium on Network Computing and Applications, NCA 2019
Editors	Aris Gkoulalas-Divanis, Mirco Marchetti, Dimiter R. Avresky
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	1-4
Number of pages	4
ISBN (Electronic)	9781728125213
ISBN (Print)	9781728125220
DOIs	https://doi.org/10.1109/NCA.2019.8935032
Publication status	Published - 2019
Event	18th IEEE International Symposium on Network Computing and Applications, NCA 2019 - Cambridge, United States Duration: 26 Sept 2019 → 28 Sept 2019

Publication series

Name	2019 IEEE 18th International Symposium on Network Computing and Applications, NCA 2019

Conference

Conference	18th IEEE International Symposium on Network Computing and Applications, NCA 2019
Country/Territory	United States
City	Cambridge
Period	26/09/19 → 28/09/19

Access to Document

10.1109/NCA.2019.8935032

Cite this

Lee, Y. C., King, J., & Hong, S-H. (2019). Holistic approach for studying resource failures at scale. In A. Gkoulalas-Divanis, M. Marchetti, & D. R. Avresky (Eds.), 2019 IEEE 18th International Symposium on Network Computing and Applications, NCA 2019 (pp. 1-4). [8935032] (2019 IEEE 18th International Symposium on Network Computing and Applications, NCA 2019). Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/NCA.2019.8935032

Lee, Young Choon ; King, Jayden ; Hong, Seok-Hee. / Holistic approach for studying resource failures at scale. 2019 IEEE 18th International Symposium on Network Computing and Applications, NCA 2019. editor / Aris Gkoulalas-Divanis ; Mirco Marchetti ; Dimiter R. Avresky. Institute of Electrical and Electronics Engineers (IEEE), 2019. pp. 1-4 (2019 IEEE 18th International Symposium on Network Computing and Applications, NCA 2019).

@inproceedings{d57e463a8b954bdd9894732cd899f9ab,

title = "Holistic approach for studying resource failures at scale",

abstract = "In large-scale distributed systems, such as data centers resource failures are the norm rather than an exception. In this paper, we propose a holistic approach to study resource failures from resource failure modelling to distributed system simulation to failure-aware scheduling algorithm design. In particular, we present (1) a simple and yet practical way to model resource failures using real-world failure traces, (2) a new distributed systems simulator and (3) two failure-aware scheduling algorithms. These scheduling algorithms are designed primarily to validate (1) and (2). Our evaluation results demonstrate the feasibility and effectiveness of our holistic approach.",

author = "Lee, {Young Choon} and Jayden King and Seok-Hee Hong",

year = "2019",

doi = "10.1109/NCA.2019.8935032",

language = "English",

isbn = "9781728125220",

series = "2019 IEEE 18th International Symposium on Network Computing and Applications, NCA 2019",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

pages = "1--4",

editor = "Aris Gkoulalas-Divanis and Mirco Marchetti and Avresky, {Dimiter R.}",

booktitle = "2019 IEEE 18th International Symposium on Network Computing and Applications, NCA 2019",

address = "United States",

note = "18th IEEE International Symposium on Network Computing and Applications, NCA 2019 ; Conference date: 26-09-2019 Through 28-09-2019",

}

Lee, YC, King, J & Hong, S-H 2019, Holistic approach for studying resource failures at scale. in A Gkoulalas-Divanis, M Marchetti & DR Avresky (eds), 2019 IEEE 18th International Symposium on Network Computing and Applications, NCA 2019., 8935032, 2019 IEEE 18th International Symposium on Network Computing and Applications, NCA 2019, Institute of Electrical and Electronics Engineers (IEEE), pp. 1-4, 18th IEEE International Symposium on Network Computing and Applications, NCA 2019, Cambridge, United States, 26/09/19. https://doi.org/10.1109/NCA.2019.8935032

Holistic approach for studying resource failures at scale. / Lee, Young Choon; King, Jayden; Hong, Seok-Hee.
2019 IEEE 18th International Symposium on Network Computing and Applications, NCA 2019. ed. / Aris Gkoulalas-Divanis; Mirco Marchetti; Dimiter R. Avresky. Institute of Electrical and Electronics Engineers (IEEE), 2019. p. 1-4 8935032 (2019 IEEE 18th International Symposium on Network Computing and Applications, NCA 2019).

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

TY - GEN

T1 - Holistic approach for studying resource failures at scale

AU - Lee, Young Choon

AU - King, Jayden

AU - Hong, Seok-Hee

PY - 2019

Y1 - 2019

N2 - In large-scale distributed systems, such as data centers resource failures are the norm rather than an exception. In this paper, we propose a holistic approach to study resource failures from resource failure modelling to distributed system simulation to failure-aware scheduling algorithm design. In particular, we present (1) a simple and yet practical way to model resource failures using real-world failure traces, (2) a new distributed systems simulator and (3) two failure-aware scheduling algorithms. These scheduling algorithms are designed primarily to validate (1) and (2). Our evaluation results demonstrate the feasibility and effectiveness of our holistic approach.

AB - In large-scale distributed systems, such as data centers resource failures are the norm rather than an exception. In this paper, we propose a holistic approach to study resource failures from resource failure modelling to distributed system simulation to failure-aware scheduling algorithm design. In particular, we present (1) a simple and yet practical way to model resource failures using real-world failure traces, (2) a new distributed systems simulator and (3) two failure-aware scheduling algorithms. These scheduling algorithms are designed primarily to validate (1) and (2). Our evaluation results demonstrate the feasibility and effectiveness of our holistic approach.

UR - http://www.scopus.com/inward/record.url?scp=85077956235&partnerID=8YFLogxK

UR - http://purl.org/au-research/grants/arc/DP180102553

U2 - 10.1109/NCA.2019.8935032

DO - 10.1109/NCA.2019.8935032

M3 - Conference proceeding contribution

SN - 9781728125220

T3 - 2019 IEEE 18th International Symposium on Network Computing and Applications, NCA 2019

SP - 1

EP - 4

BT - 2019 IEEE 18th International Symposium on Network Computing and Applications, NCA 2019

A2 - Gkoulalas-Divanis, Aris

A2 - Marchetti, Mirco

A2 - Avresky, Dimiter R.

PB - Institute of Electrical and Electronics Engineers (IEEE)

T2 - 18th IEEE International Symposium on Network Computing and Applications, NCA 2019

Y2 - 26 September 2019 through 28 September 2019

ER -

Lee YC, King J, Hong S-H. Holistic approach for studying resource failures at scale. In Gkoulalas-Divanis A, Marchetti M, Avresky DR, editors, 2019 IEEE 18th International Symposium on Network Computing and Applications, NCA 2019. Institute of Electrical and Electronics Engineers (IEEE). 2019. p. 1-4. 8935032. (2019 IEEE 18th International Symposium on Network Computing and Applications, NCA 2019). doi: 10.1109/NCA.2019.8935032

Holistic approach for studying resource failures at scale

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this