Handling uncertainty: Pareto-Efficient BoT scheduling on hybrid clouds

M. Reza Hoseiny Farahabady, Hamid R.D. Samani, Luke M. Leslie, Young Choon Lee, Albert Y. Zomaya

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

29 Citations (Scopus)


Coping with uncertainty is a challenging and complex problem particularly in hybrid cloud environments-private cloud plus public cloud. Conflicting goals of minimizing the cost and performance, unknown prior knowledge about task running times, and a lack of estimation tools are just a few of the challenges that resource management systems in those environments encounter. The aim in this paper is to find Pareto-optimal schedules for large-scale Bag-of-Tasks (BoT) applications that meet userdefined constraints, such as deadline or budget or some tradeoff between them. BoT applications are common in science and engineering and consist of many independent tasks. To achieve the user's chosen Pareto-optimal schedule, we develop a dynamic resource allocation process for hybrid clouds. We also present a hybrid approach to estimating task running times that incorporates several estimators with a feedback control system to cope with the inherent uncertainty in such estimation. Through extensive experiments on a testbed hybrid cloud, using Amazon EC2 as a public cloud, we show that the proposed approach can achieve nearoptimality with little overhead, and consistently achieves a solution within 2% of the user's chosen Pareto-optimal schedule. Further, we demonstrate that our approach performs better than an extended List scheduling approach by reducing both the total cost and time needed to run the application by almost 20% and 5% on average, respectively.

Original languageEnglish
Title of host publicationProceedings: International Conference on Parallel Processing - The 42nd Annual Conference, ICPP 2013
Place of PublicationPiscataway, NJ
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Number of pages10
ISBN (Print)9780769551173
Publication statusPublished - 2013
Externally publishedYes
Event42nd Annual International Conference on Parallel Processing, ICPP 2013 - Lyon, France
Duration: 1 Oct 20134 Oct 2013


Other42nd Annual International Conference on Parallel Processing, ICPP 2013


Dive into the research topics of 'Handling uncertainty: Pareto-Efficient BoT scheduling on hybrid clouds'. Together they form a unique fingerprint.

Cite this