Remote Access Transactions of the American Mathematical Society
Green Open Access

Transactions of the American Mathematical Society

ISSN 1088-6850(online) ISSN 0002-9947(print)



Randomized dynamic programming principle and Feynman-Kac representation for optimal control of McKean-Vlasov dynamics

Authors: Erhan Bayraktar, Andrea Cosso and Huyên Pham
Journal: Trans. Amer. Math. Soc. 370 (2018), 2115-2160
MSC (2010): Primary 49L20, 93E20, 60K35, 60H10, 60H30
Published electronically: November 15, 2017
Full-text PDF

Abstract | References | Similar Articles | Additional Information

Abstract: We analyze a stochastic optimal control problem, where the state process follows a McKean-Vlasov dynamics and the diffusion coefficient can be degenerate. We prove that its value function $ V$ admits a nonlinear Feynman-Kac representation in terms of a class of forward-backward stochastic differential equations, with an autonomous forward process. We exploit this probabilistic representation to rigorously prove the dynamic programming principle (DPP) for $ V$. The Feynman-Kac representation we obtain has an important role beyond its intermediary role in obtaining our main result: in fact it would be useful in developing probabilistic numerical schemes for $ V$. The DPP is important in obtaining a characterization of the value function as a solution of a nonlinear partial differential equation (the so-called Hamilton-Jacobi-Belman equation), in this case on the Wasserstein space of measures. We should note that the usual way of solving these equations is through the Pontryagin maximum principle, which requires some convexity assumptions. There were attempts in using the dynamic programming approach before, but these works assumed a priori that the controls were of Markovian feedback type, which helps write the problem only in terms of the distribution of the state process (and the control problem becomes a deterministic problem). In this paper, we will consider open-loop controls and derive the dynamic programming principle in this most general case. In order to obtain the Feynman-Kac representation and the randomized dynamic programming principle, we implement the so-called randomization method, which consists of formulating a new McKean-Vlasov control problem, expressed in weak form taking the supremum over a family of equivalent probability measures. One of the main results of the paper is the proof that this latter control problem has the same value function $ V$ of the original control problem.

References [Enhancements On Off] (What's this?)

Similar Articles

Retrieve articles in Transactions of the American Mathematical Society with MSC (2010): 49L20, 93E20, 60K35, 60H10, 60H30

Retrieve articles in all journals with MSC (2010): 49L20, 93E20, 60K35, 60H10, 60H30

Additional Information

Erhan Bayraktar
Affiliation: Department of Mathematics, University of Michigan, 530 Church Street, Ann Arbor, Michigan 48109

Andrea Cosso
Affiliation: Politecnico di Milano, Dipartimento di Matematica, via Bonardi 9, 20133 Milano, Italy
Address at time of publication: Dipartimento di Matematica, Universitá di Bologna, Piazza di Porta S. Donato, 5, 40126 Bologna, Italy

Huyên Pham
Affiliation: Laboratoire de Probabilités et Modèles Aléatoires, CNRS, UMR 7599, Université Paris Diderot, 75205 Paris Cedex 13, France–and-CREST-ENSAE

Keywords: Controlled McKean-Vlasov stochastic differential equations, dynamic programming principle, randomization method, forward-backward stochastic differential equations
Received by editor(s): June 26, 2016
Received by editor(s) in revised form: October 25, 2016
Published electronically: November 15, 2017
Additional Notes: The first author was supported in part by the National Science Foundation under grant DMS-1613170 and the Susan M. Smith Professorship.
The third author was supported in part by the ANR project CAESARS (ANR-15-CE05-0024)
Article copyright: © Copyright 2017 American Mathematical Society

American Mathematical Society