adCFS: adaptive completely fair scheduling policy for containerised workflows systems

Eidah J. Alzahrani; Zahir Tari; Young Choon Lee; Deafallah Alsadie; Albert Y. Zomaya

doi:10.1109/NCA.2017.8171362

adCFS

adaptive completely fair scheduling policy for containerised workflows systems

Eidah J. Alzahrani, Zahir Tari, Young Choon Lee, Deafallah Alsadie, Albert Y. Zomaya

Department of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution

2 Citations (Scopus)

Abstract

Scientific workflows are increasingly containerised, which requires rethinking central processing unit (CPU) sharing policies to accommodate different workload types. However, container engines running scientific workflows struggle to share the CPU fairly, as workload characteristics are not taken into account. This paper proposes a sharing policy called the Adaptive Completely Fair Scheduling policy (adCFS), which considers the future state of CPU usage and proactively shares CPU cycles between various containers based on their corresponding workload metrics (e.g., CPU usage, task runtime, #tasks). adCFS estimates the weight of workload characteristics and redistributes the CPU based on the corresponding weights. The Markov chain model is used to predict CPU state use, and the adCFS policy is triggered to dynamically allocate containers to the proper CPU portions. Experimental results show enhanced container CPU response time for those containers that run heavy and large jobs: these display 12% faster response time compared with the default CFS (Completely Fair Scheduler). adCFS therefore enhances CFS by considering workload metrics, which leads to the CPU being shared fairly when it is fully used.

Original language	English
Title of host publication	2017 IEEE 16th International Symposium on Network Computing and Applications
Subtitle of host publication	NCA 2017
Place of Publication	Piscataway, NJ
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	1-8
Number of pages	8
Volume	2017
ISBN (Electronic)	9781538614655
ISBN (Print)	9781538614648
DOIs	https://doi.org/10.1109/NCA.2017.8171362
Publication status	Published - 8 Dec 2017
Event	16th IEEE International Symposium on Network Computing and Applications, NCA 2017 - Cambridge, United States Duration: 30 Oct 2017 → 1 Nov 2017

Conference

Conference	16th IEEE International Symposium on Network Computing and Applications, NCA 2017
Country	United States
City	Cambridge
Period	30/10/17 → 1/11/17

Keywords

Completely Fair Scheduler
Containerised Scientific Workflows
Containers
Docker Engine
Markov Chain

Access to Document

10.1109/NCA.2017.8171362

Link to publication in Scopus

Fingerprint Dive into the research topics of 'adCFS: adaptive completely fair scheduling policy for containerised workflows systems'. Together they form a unique fingerprint.

Cite this

Alzahrani, E. J., Tari, Z., Lee, Y. C., Alsadie, D., & Zomaya, A. Y. (2017). adCFS: adaptive completely fair scheduling policy for containerised workflows systems. In 2017 IEEE 16th International Symposium on Network Computing and Applications: NCA 2017 (Vol. 2017, pp. 1-8). Piscataway, NJ: Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/NCA.2017.8171362

Alzahrani, Eidah J. ; Tari, Zahir ; Lee, Young Choon ; Alsadie, Deafallah ; Zomaya, Albert Y. / adCFS : adaptive completely fair scheduling policy for containerised workflows systems. 2017 IEEE 16th International Symposium on Network Computing and Applications: NCA 2017. Vol. 2017 Piscataway, NJ : Institute of Electrical and Electronics Engineers (IEEE), 2017. pp. 1-8

@inproceedings{12ce34e88f964a1fbc385ed26a0e58d5,

title = "adCFS: adaptive completely fair scheduling policy for containerised workflows systems",

abstract = "Scientific workflows are increasingly containerised, which requires rethinking central processing unit (CPU) sharing policies to accommodate different workload types. However, container engines running scientific workflows struggle to share the CPU fairly, as workload characteristics are not taken into account. This paper proposes a sharing policy called the Adaptive Completely Fair Scheduling policy (adCFS), which considers the future state of CPU usage and proactively shares CPU cycles between various containers based on their corresponding workload metrics (e.g., CPU usage, task runtime, #tasks). adCFS estimates the weight of workload characteristics and redistributes the CPU based on the corresponding weights. The Markov chain model is used to predict CPU state use, and the adCFS policy is triggered to dynamically allocate containers to the proper CPU portions. Experimental results show enhanced container CPU response time for those containers that run heavy and large jobs: these display 12{\%} faster response time compared with the default CFS (Completely Fair Scheduler). adCFS therefore enhances CFS by considering workload metrics, which leads to the CPU being shared fairly when it is fully used.",

keywords = "Completely Fair Scheduler, Containerised Scientific Workflows, Containers, Docker Engine, Markov Chain",

author = "Alzahrani, {Eidah J.} and Zahir Tari and Lee, {Young Choon} and Deafallah Alsadie and Zomaya, {Albert Y.}",

year = "2017",

month = "12",

day = "8",

doi = "10.1109/NCA.2017.8171362",

language = "English",

isbn = "9781538614648",

volume = "2017",

pages = "1--8",

booktitle = "2017 IEEE 16th International Symposium on Network Computing and Applications",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

address = "United States",

}

Alzahrani, EJ, Tari, Z, Lee, YC, Alsadie, D & Zomaya, AY 2017, adCFS: adaptive completely fair scheduling policy for containerised workflows systems. in 2017 IEEE 16th International Symposium on Network Computing and Applications: NCA 2017. vol. 2017, Institute of Electrical and Electronics Engineers (IEEE), Piscataway, NJ, pp. 1-8, 16th IEEE International Symposium on Network Computing and Applications, NCA 2017, Cambridge, United States, 30/10/17. https://doi.org/10.1109/NCA.2017.8171362

adCFS : adaptive completely fair scheduling policy for containerised workflows systems. / Alzahrani, Eidah J.; Tari, Zahir; Lee, Young Choon; Alsadie, Deafallah; Zomaya, Albert Y.

2017 IEEE 16th International Symposium on Network Computing and Applications: NCA 2017. Vol. 2017 Piscataway, NJ : Institute of Electrical and Electronics Engineers (IEEE), 2017. p. 1-8.

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution

TY - GEN

T1 - adCFS

T2 - adaptive completely fair scheduling policy for containerised workflows systems

AU - Alzahrani, Eidah J.

AU - Tari, Zahir

AU - Lee, Young Choon

AU - Alsadie, Deafallah

AU - Zomaya, Albert Y.

PY - 2017/12/8

Y1 - 2017/12/8

N2 - Scientific workflows are increasingly containerised, which requires rethinking central processing unit (CPU) sharing policies to accommodate different workload types. However, container engines running scientific workflows struggle to share the CPU fairly, as workload characteristics are not taken into account. This paper proposes a sharing policy called the Adaptive Completely Fair Scheduling policy (adCFS), which considers the future state of CPU usage and proactively shares CPU cycles between various containers based on their corresponding workload metrics (e.g., CPU usage, task runtime, #tasks). adCFS estimates the weight of workload characteristics and redistributes the CPU based on the corresponding weights. The Markov chain model is used to predict CPU state use, and the adCFS policy is triggered to dynamically allocate containers to the proper CPU portions. Experimental results show enhanced container CPU response time for those containers that run heavy and large jobs: these display 12% faster response time compared with the default CFS (Completely Fair Scheduler). adCFS therefore enhances CFS by considering workload metrics, which leads to the CPU being shared fairly when it is fully used.

AB - Scientific workflows are increasingly containerised, which requires rethinking central processing unit (CPU) sharing policies to accommodate different workload types. However, container engines running scientific workflows struggle to share the CPU fairly, as workload characteristics are not taken into account. This paper proposes a sharing policy called the Adaptive Completely Fair Scheduling policy (adCFS), which considers the future state of CPU usage and proactively shares CPU cycles between various containers based on their corresponding workload metrics (e.g., CPU usage, task runtime, #tasks). adCFS estimates the weight of workload characteristics and redistributes the CPU based on the corresponding weights. The Markov chain model is used to predict CPU state use, and the adCFS policy is triggered to dynamically allocate containers to the proper CPU portions. Experimental results show enhanced container CPU response time for those containers that run heavy and large jobs: these display 12% faster response time compared with the default CFS (Completely Fair Scheduler). adCFS therefore enhances CFS by considering workload metrics, which leads to the CPU being shared fairly when it is fully used.

KW - Completely Fair Scheduler

KW - Containerised Scientific Workflows

KW - Containers

KW - Docker Engine

KW - Markov Chain

UR - http://www.scopus.com/inward/record.url?scp=85046480259&partnerID=8YFLogxK

U2 - 10.1109/NCA.2017.8171362

DO - 10.1109/NCA.2017.8171362

M3 - Conference proceeding contribution

SN - 9781538614648

VL - 2017

SP - 1

EP - 8

BT - 2017 IEEE 16th International Symposium on Network Computing and Applications

PB - Institute of Electrical and Electronics Engineers (IEEE)

CY - Piscataway, NJ

ER -