Glider: rethinking congestion control with deep reinforcement learning

Traditional congestion control protocols may fail to achieve consistently-high performance over a wide range of networking environments as their hardwired policies are optimized over specific network conditions. In this paper, we depart from conventional wisdom and propose Glider, a new congestion control protocol that uses deep reinforcement learning to be more versatile and adaptive to dynamic environments. In particular, Glider uses a framework based on Deep Q-Network, that a sender keeps adapting its congestion control strategies by continuously interacting with the network environment. In addition, the sender constantly sends data, making it challenging to apply reinforcement learning algorithms that require step-by-step state computation to congestion control. Therefore, we design a Dynamic Bisection Division Algorithm (DBDA) to discretize the packet transmission process into steps to ensure Glider’s feasibility on congestion control. We have used an extensive array of experiments on Pantheon to show that Glider can adapt well to varying buffer sizes and is resilient to random loss. Moreover, on wide-area inter-data center links, it can achieve 6.4× and 1.4× higher throughput than TCP CUBIC and BBR, respectively, and comparable performance as other learning-based congestion control protocols in the literature.