A resource allocation controller for key-value data stores

Recent distributed key-value data stores, such as Aerospike are getting the momentum with ever-increasing need for large-scale real-time data processing. While these data stores can provide significantly improved performance, they still struggle to meet Quality of Service (QoS) during workload surges. In this paper, we address the problem of QoS-aware resource allocation for burst workloads in key-value data stores. To this end, we design a resource allocation controller, which enables each application to independently regulate the releases of its requests taking into account QoS. In particular, the proposed controller monitors the actual performance metrics of the target system and dynamically releases requests from a buffer owned by each application accordingly. We have implemented the proposed controller in an Aerospike cluster for our performance evaluation. Experiments have been conducted with various workload intensities (with up to 36,180 write operations per second) in comparison with the default Aerospike policy. Experimental results confirm that the proposed controller decreases the overall average latency up to 41% on high-rate workload while maintaining the QoS of high priority applications.