DEWE: a framework for distributed elastic scientific workflow execution

As cloud computing is increasingly adopted, the open, on-demand nature of public clouds makes explicit consideration of the underlying environment highly advantageous in terms of decreasing cost and increasing elasticity. In this paper, we address the exploitation of such cloud capabilities and present DEWE, a framework for the distributed, elastic execution of scientific workflows. DEWE is designed to be easily extensible and customizable, and to provide a simple interface to automate deployment and elasticity in public clouds. Using Montage, an astronomical image mosaic engine, as a case study, and Amazon Web Services (AWS) as the cloud environment, we demonstrate the benefits DEWE can provide to scientists seeking to design job scheduling, data management, and resource allocation strategies with potentially unlimited on-demand resources at hand. Further, DEWE's visualization tool much leverages the analysis and evaluation of those strategies.