Optimizing scientific workflows in the cloud

a montage example

Qingye Jiang, Young Choon Lee, Manuel Arenaz, Luke M. Leslie, Albert Y. Zomaya

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contribution

9 Citations (Scopus)

Abstract

As scientific workflows are increasingly deployed in clouds, a myriad of studies have been conducted-including the development of workflow execution systems and scheduling/resource-management algorithms-for optimizing the execution of these workflows. However, the efficacy of most, if not all, of these previous works is limited by the original design and structure of workflow, i.e., Sequential code and few bottleneck tasks. In this paper, we address the optimization of scientific workflow execution in clouds by exploiting multi-core systems with the parallelization of bottleneck tasks. To this end, we develop a workflow visualization toolkit to synthetize resource consumption and data transfer patterns, as well as to identify the bottleneck, of the workflow being studied. Parallelization techniques are then applied to the module that is identified as the bottleneck in order to take full advantage of the underlying multicore computing environment. Testing results with a 6.0-degree Montage example on Amazon EC2 with various configurations show that our optimization of workflows (bottleneck tasks in particular) reduces completion time (or make span) by 21% to 43% depending on the instance type being used to run the workflow, without any impact on the cost.

Original languageEnglish
Title of host publicationProceedings - 2014 IEEE/ACM 7th International Conference on Utility and Cloud Computing, UCC 2014
Place of PublicationPiscataway, NJ
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages517-522
Number of pages6
ISBN (Electronic)9781479978816
DOIs
Publication statusPublished - 29 Jan 2014
Externally publishedYes
Event7th IEEE/ACM International Conference on Utility and Cloud Computing, UCC 2014 - London, United Kingdom
Duration: 8 Dec 201411 Dec 2014

Publication series

NameInternational Conference on Utility and Cloud Computing
PublisherIEEE
ISSN (Print)2373-6860

Other

Other7th IEEE/ACM International Conference on Utility and Cloud Computing, UCC 2014
CountryUnited Kingdom
CityLondon
Period8/12/1411/12/14

Fingerprint Dive into the research topics of 'Optimizing scientific workflows in the cloud: a montage example'. Together they form a unique fingerprint.

Cite this