TY - JOUR
T1 - Constructing Performance-Predictable Clusters with Performance-Varying Resources of Clouds
AU - Adam, Omer Y.
AU - Lee, Young Choon
AU - Zomaya, Albert Y.
PY - 2016/9/1
Y1 - 2016/9/1
N2 - Although most current cloud providers, such as Amazon Web Services (AWS) and Microsoft Azure offer different types of computing instances with different capacities, cloud users tend to hire a cluster of instances of particular type to ensure performance predictability for their applications. Nowadays, many large-scale applications including big data analytics applications feature workload patterns that have heterogeneous resource demands, for which, accounting for heterogeneity of virtual cloud instances to allocate would be highly advantageous to the application performance. However, performance predictability has been always an issue in such clusters of heterogeneous resources. In particular, to precisely decide on what instances from which types to enclose in a cluster, such that the desired performance is attained, remains an open question. To this end, we devise a resource allocation mechanism by formulating it as a Mixed-Integer programming model representing an optimization problem. Our resource allocation mechanism incorporates predictable average performance as a unified performance metric, which concerns two key performance-related issues: (a) Performance variation within same-type instances, and (b) Correlations of performance variabilities across different types. Our experimental results demonstrate that target performance is predictable and attainable for clusters of heterogeneous resources. Our mechanism constructs clusters whose performance is within 95 percent of the performance of optimal ones, hence deadlines are always met. By reoptimisation, our mechanism can react to performance mispredictions and support autoscaling for varying workloads. We experimentally verify our findings using clusters on Amazon EC2 with MapReduce workloads, and on a private cloud as well. We conduct comparison experiments with an existing recent resource allocation approach in literature.
AB - Although most current cloud providers, such as Amazon Web Services (AWS) and Microsoft Azure offer different types of computing instances with different capacities, cloud users tend to hire a cluster of instances of particular type to ensure performance predictability for their applications. Nowadays, many large-scale applications including big data analytics applications feature workload patterns that have heterogeneous resource demands, for which, accounting for heterogeneity of virtual cloud instances to allocate would be highly advantageous to the application performance. However, performance predictability has been always an issue in such clusters of heterogeneous resources. In particular, to precisely decide on what instances from which types to enclose in a cluster, such that the desired performance is attained, remains an open question. To this end, we devise a resource allocation mechanism by formulating it as a Mixed-Integer programming model representing an optimization problem. Our resource allocation mechanism incorporates predictable average performance as a unified performance metric, which concerns two key performance-related issues: (a) Performance variation within same-type instances, and (b) Correlations of performance variabilities across different types. Our experimental results demonstrate that target performance is predictable and attainable for clusters of heterogeneous resources. Our mechanism constructs clusters whose performance is within 95 percent of the performance of optimal ones, hence deadlines are always met. By reoptimisation, our mechanism can react to performance mispredictions and support autoscaling for varying workloads. We experimentally verify our findings using clusters on Amazon EC2 with MapReduce workloads, and on a private cloud as well. We conduct comparison experiments with an existing recent resource allocation approach in literature.
UR - http://www.scopus.com/inward/record.url?scp=84982156291&partnerID=8YFLogxK
U2 - 10.1109/TC.2015.2510648
DO - 10.1109/TC.2015.2510648
M3 - Article
AN - SCOPUS:84982156291
VL - 65
SP - 2709
EP - 2724
JO - IEEE Transactions on Computers
JF - IEEE Transactions on Computers
SN - 0018-9340
IS - 9
M1 - 7362012
ER -