Adaptive multiple-workflow scheduling with task rearrangement

Wei Chen, Young Choon Lee*, Alan Fekete, Albert Y. Zomaya

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

30 Citations (Scopus)

Abstract

Large-scale distributed computing systems like grids and more recently clouds are a platform of choice for many resource-intensive applications. Workflow applications account for the majority of these applications, particularly in science and engineering. A workflow application consists of multiple precedence-constrained tasks with data dependencies. Since resources in those systems are shared by many users and applications deployed there are very diverse, scheduling is complicated. Often, the actual execution of applications differs from the original schedule following delays such as those caused by resource contention and other issues in performance prediction. These delays have further impact when running multiple workflow applications due to inter-task dependencies. In this paper, we investigate the problem of scheduling multiple workflow applications concurrently, explicitly taking into account scheduling robustness. We present a dynamic task rearrangement and rescheduling algorithm that exploits the scheduling flexibility from precedence constraints among tasks. The algorithm optimizes resource allocation among multiple workflows, and it often stops the influence of delayed execution passing to subsequent tasks. The experimental results demonstrate that our approach can significantly improve performance in multiple-workflow scheduling.

Original languageEnglish
Pages (from-to)1297-1317
Number of pages21
JournalJournal of Supercomputing
Volume71
Issue number4
DOIs
Publication statusPublished - 1 Apr 2015
Externally publishedYes

Fingerprint

Dive into the research topics of 'Adaptive multiple-workflow scheduling with task rearrangement'. Together they form a unique fingerprint.

Cite this