Handling uncertainty

Pareto-Efficient BoT scheduling on hybrid clouds

M. Reza Hoseiny Farahabady, Hamid R.D. Samani, Luke M. Leslie, Young Choon Lee, Albert Y. Zomaya

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contribution

22 Citations (Scopus)

Abstract

Coping with uncertainty is a challenging and complex problem particularly in hybrid cloud environments-private cloud plus public cloud. Conflicting goals of minimizing the cost and performance, unknown prior knowledge about task running times, and a lack of estimation tools are just a few of the challenges that resource management systems in those environments encounter. The aim in this paper is to find Pareto-optimal schedules for large-scale Bag-of-Tasks (BoT) applications that meet userdefined constraints, such as deadline or budget or some tradeoff between them. BoT applications are common in science and engineering and consist of many independent tasks. To achieve the user's chosen Pareto-optimal schedule, we develop a dynamic resource allocation process for hybrid clouds. We also present a hybrid approach to estimating task running times that incorporates several estimators with a feedback control system to cope with the inherent uncertainty in such estimation. Through extensive experiments on a testbed hybrid cloud, using Amazon EC2 as a public cloud, we show that the proposed approach can achieve nearoptimality with little overhead, and consistently achieves a solution within 2% of the user's chosen Pareto-optimal schedule. Further, we demonstrate that our approach performs better than an extended List scheduling approach by reducing both the total cost and time needed to run the application by almost 20% and 5% on average, respectively.

Original languageEnglish
Title of host publicationProceedings: International Conference on Parallel Processing - The 42nd Annual Conference, ICPP 2013
Place of PublicationPiscataway, NJ
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages419-428
Number of pages10
ISBN (Print)9780769551173
DOIs
Publication statusPublished - 2013
Externally publishedYes
Event42nd Annual International Conference on Parallel Processing, ICPP 2013 - Lyon, France
Duration: 1 Oct 20134 Oct 2013

Other

Other42nd Annual International Conference on Parallel Processing, ICPP 2013
CountryFrance
CityLyon
Period1/10/134/10/13

Fingerprint Dive into the research topics of 'Handling uncertainty: Pareto-Efficient BoT scheduling on hybrid clouds'. Together they form a unique fingerprint.

Cite this