Resource demand aware scheduling for workflows in clouds

Khaled Almi'ani, Young Choon Lee, Bernard Mans

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contribution

6 Citations (Scopus)

Abstract

A major challenge of running applications in clouds is to determine the right number of resources (virtual machines or VMs) to rent in terms of both performance and cost. Such a challenge becomes greater if the application requires to run across multiple resources. In this paper, we address the problem of scheduling scientific workflow applications. The structure of workflows, dictated by precedence/data dependencies, and the diversity of resources in clouds both at large scale make the resource provisioning and task scheduling very complex. To this end, we design the Resource Demand Aware Scheduling (RDAS) algorithm that schedules workflows based on their resource demands and priorities considering workflow structure. RDAS partitions workflows and allocates resources of possibly different capacities/types to the partitions in a 'fair' manner such that their execution times do not vary significantly. RDAS turns resource and application heterogeneity (a major hindering factor in clouds) into an opportunity for optimizing resource provisioning for scientific workflows. Based on our experimental results, RDAS demonstrates its capacity of minimizing the overall workflow completion time (makespan) and in turn minimizing costs of the execution. In particular, RDAS outperforms three existing algorithms by 22%, 13% and 33%, on average, in terms of makespan, cost and the number of resources used, respectively.

Original languageEnglish
Title of host publication2017 IEEE 16th International Symposium on Network Computing and Applications, NCA 2017
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages1-5
Number of pages5
Volume2017-January
ISBN (Electronic)9781538614655
DOIs
Publication statusPublished - 8 Dec 2017
Event16th IEEE International Symposium on Network Computing and Applications, NCA 2017 - Cambridge, United States
Duration: 30 Oct 20171 Nov 2017

Conference

Conference16th IEEE International Symposium on Network Computing and Applications, NCA 2017
CountryUnited States
CityCambridge
Period30/10/171/11/17

Fingerprint Dive into the research topics of 'Resource demand aware scheduling for workflows in clouds'. Together they form a unique fingerprint.

  • Cite this

    Almi'ani, K., Lee, Y. C., & Mans, B. (2017). Resource demand aware scheduling for workflows in clouds. In 2017 IEEE 16th International Symposium on Network Computing and Applications, NCA 2017 (Vol. 2017-January, pp. 1-5). Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/NCA.2017.8171368