Janus

The move from the traditional Software-as-a-Product (SaaP) model to the Software-as-a-Service (SaaS) model is apparent with the wide adoption of cloud computing. Unlike the SaaP model, the SaaS model delivers a diverse set of software features directly from public clouds to a large number of arbitrary users with varying quality of service (QoS) requirements. QoS is typically assured by admission control. However, there are two outstanding issues with traditional QoS systems: (1) they are usually designed and developed with a special purpose, making them difficult to be reused for other use cases; and (2) they have limited scalability (i.e., vertical scalability) due to the write-intensive nature of admission control workload. In this paper, we present Janus - a QoS framework that is generic and scalable for SaaS applications taking full advantage of cloud's inherent horizontal scalability (scaling-out). Janus uses a multi-layer architecture to eliminate the communication between nodes (being scaled out) in the same layer achieving horizontal scalability without sacrificing vertical scalability. Janus ensures accurate admission control (QoS decisions) using a distributed set of leaky buckets with a refill mechanism. Janus also adopts a key-value request-response mechanism for easy integration with the actual application. We extensively evaluate Janus on AWS cloud with both Apache HTTP server benchmarking tool and a photo sharing web application. Our experimental results demonstrate that (a) Janus achieves linear scalability both vertically and horizontally, and (b) Janus can be integrated with existing applications with a minimum amount of code change. In particular, Janus achieves more than 100,000 requests per second with only 10 nodes (4 vCPU cores on each node) in the QoS server layer and 90% of the admission control decisions were made in 3 milliseconds.