Numerical solutions of stochastic control problems: Markov chain approximation methods

Zhuo Jin, Ky Tran, George Yin*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingChapterpeer-review

2 Citations (Scopus)


This chapter is devoted to Markov chain approximation methods for solving stochastic control problems numerically. While the underlying problems are in continuous time, the numerical methods are based on the construction of discrete-time controlled Markov chains that are locally consistent so that the local mean and covariance coincide with that of the continuous-time controlled stochastic systems. One of the main advantages of the Markov chain approximation method is that not much of a priori knowledge of the regularity of the related partial differential equations is needed.

Original languageEnglish
Title of host publicationNumerical Control
Subtitle of host publicationPart A
EditorsEmmanuel Trélat, Enrique Zuazua
Place of PublicationAmsterdam
Number of pages32
ISBN (Print)9780323850599
Publication statusPublished - 2022

Publication series

NameHandbook of Numerical Analysis
ISSN (Print)1570-8659


  • Controlled switching diffusion
  • Markov chain approximation
  • Numerical method


Dive into the research topics of 'Numerical solutions of stochastic control problems: Markov chain approximation methods'. Together they form a unique fingerprint.

Cite this