Optimized MARL for latency-sensitive collaborative service placement in edge computing

By distributing computing and storage resources to edge servers (ESs), edge computing enables service execution closer to requests, alleviating high latency caused by long-distance data transmission. The communication between ESs facilitates collaborative service placement (CSP) on multiple ESs, leading to latency reduction and thereby improving the quality of service. However, the discrepancies in service request distributions among ESs pose challenge to effective CSP, while limited prior knowledge further increases the difficulty in policy formation. Additionally, the delayed awareness of placement strategies among other ESs exacerbates the influence of non-Markovian nature. To overcome above challenges, a leader-follower based multi-agent reinforcement learning for CSP scheme, named LFMCSP, is proposed in this paper. Specifically, a CSP model is formulated, with ESs abstracted as intelligent agents within it. Then, the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm is utilized to explore potential patterns in complex requests by experience sharing. To achieve transparency of global states for all agents, the leader-follower architecture is integrated into MADDPG. Through rigorous experiments and comparisons with various baseline schemes, LFMCSP demonstrates effectiveness in reducing service execution delays.