Reinforcement learning for real-world control applications

If reinforcement learning (RL) techniques are to be used for “real world” dynamic system control, the problems of noise and plant disturbance will have to be addressed, along with various issues resulting from learning in non-Markovian settings. We present experimental results from three domains: A simulated noisy pole-and-cart system, an artificial non-Markovian decision problem, and a real six-legged walking robot. The results from each of these domains suggest that that actual return (Monte Carlo) approaches to the credit-assignment problem may be more suited than temporal difference (TD) methods for many real- world control applications. A new algorithm we call C-Trace, a variant of the P-Trace RL algorithm is introduced, and some possible advantages of using algorithms of this type are discussed.