ADREA: a framework for adaptive resource allocation in distributed computing systems

Large-scale distributed computing systems (LDCSs) can be best characterized by their dynamic nature particularly in terms of availability and performance. Typically, these systems deal with various types of jobs in many aspects, such as resource requirements, quality of service (QoS) and other temporal constraints. These diverse characteristics in both resources and jobs impose a great burden on scheduling and resource allocation. That is, inefficient resource allocation brings about poor resource utilization issues and often unreliable job execution. We present the ADaptive-REliable Allocation (ADREA) scheme, which attempts to ensure reliable job execution effectively exploiting heterogeneity in both resources and jobs using a novel clustering technique and a dynamic job migration policy. Specifically, ADREA intends to pave the way in producing better performance (e.g., response time, resource utilization) with reliable computation. Extensive simulations with varying processing capacities and different job arrival rates have been carried out to evaluate our scheme. The results demonstrate that the proposed scheme provides better performance over other algorithms as it significantly improves both job completion time and resource utilization.