Training deep reinforcement learning agents to coordinate with humans using perceptual-motor primitives of human behavior

Effective collaboration between humans and artificial agents (AAs) requires balancing task performance with human-compatible behavior. While deep reinforcement learning (DRL) enables high performance, it often produces behaviors misaligned with human expectations. This study investigates integrating dynamical perceptual-motor primitives (DPMPs), models of goal-directed human action, into AA architecture and training. We evaluated three AA types in human-participant interaction tasks: a heuristic DPMP-based agent, a DRL agent trained via self-play, and a DRL agent trained alongside a heuristic DPMP-based partner. Results revealed a significant performance-compatibility gap; although DRL agents excelled during training, the heuristic DPMP agent outperformed both DRL variants in live interaction regarding task completion time, behavioral predictability, and participant preference. While DRL agents trained with DPMP partners showed modest coordination gains, they still trailed the pure heuristic agent in efficiency and user preference. These findings highlight the complexities of designing AAs that balance task efficiency with human-compatible behaviors, emphasizing the need for training methods that enhance real-world human-AA coordination.