Publications Meetings The Profession Membership Programs Math Samplings Policy & Advocacy In the News About the AMS
|
   
Available in electronic format
Available in print format 		Memoirs of the American Mathematical Society
Memoirs of the American Mathematical Society
ISSN 1947-6221(e) ISSN 0065-9266(p)

     

Ergodicity, stabilization, and singular perturbations for Bellman-Isaacs equations

Author(s): Olivier Alvarez; Martino Bardi
Journal: Memoirs of the AMS 204 (2010), no. 960.
MSC (2000): Primary 35Bxx, 35Kxx, 93C70, 49N70; Secondary 49L25, 60J60, 91A23, 93E20
Posted: November 5, 2009
MathSciNet review: 2640736
Retrieve article in: PDF

Abstract | References | Similar articles | Additional information

Abstract: We study singular perturbations of optimal stochastic control problems and differential games arising in the dimension reduction of system with multiple time scales. We analyze the uniform convergence of the value functions via the associated Hamilton-Jacobi-Bellman-Isaacs equations, in the framework of viscosity solutions. The crucial properties of ergodicity and stabilization to a constant that the Hamiltonian must possess are formulated as differential games with ergodic cost criteria. They are studied under various different assumptions and with PDE as well as control-theoretic methods. We construct also an explicit example where the convergence is not uniform. Finally we give some applications to the periodic homogenization of Hamilton-Jacobi equations with non-coercive Hamiltonian and of some degenerate parabolic PDEs.

References:

[AA67]
V.I. Arnold and A. Avez, Problèmes ergodiques de la mécanique classique, Gauthier-Villars, Paris, 1967, english translation: W. A. Benjamin, New York, 1968. MR 0209436 (35:334)

[AB01]
O. Alvarez and M. Bardi, Viscosity solutions methods for singular perturbations in deterministic and stochastic control, SIAM J. Control Optim. 40 (2001), 1159-1188. MR 1882729 (2003a:49034)

[AB03]
-, Singular perturbations of degenerate parabolic PDEs: a general convergence result, Arch. Rational Mech. Anal. 170 (2003), 17-61. MR 2012646 (2004h:35012)

[AB07]
-, Ergodic problems in differential games, Advances in Dynamic Game Theory (S. Jorgensen, M. Quincampoix, and T.L. Vincent, eds.), Ann. Internat. Soc. Dynam. Games, vol. 9, Birkhäuser, 2007, pp. 131-152. MR 2341242 (2008k:91030)

[ABM07]
O. Alvarez, M. Bardi, and C. Marchi, Multiscale problems and homogenization for second-order Hamilton-Jacobi equations, J. Differential Equations 243 (2007), 349-387. MR 2371792 (2009a:35030)

[ABM08]
-, Multiscale singular perturbations and homogenization of optimal control problems, Geometric Control and Nonsmooth Analysis (F. Ancona, A. Bressan, P. Cannarsa, F.H. Clarke, and P.R. Wolenski, eds.), Ser. Adv. Math. Appl. Sci., 76, World Scientific Publishing, Hackensack, NJ, 2008, pp. 1-27. MR 2487745

[AG97]
Z. Artstein and V. Gaitsgory, Tracking fast trajectories along a slow dynamics: a singular perturbations approach, SIAM J. Control Optim. 35 (1997), 1487-1507. MR 1466912 (98h:49027)

[AG00]
-, The value function of singularly perturbed control systems, Appl. Math. Optim. 41 (2000), 425-445. MR 1739395 (2001b:49035)

[AK87]
L. Arnold and W. Kliemann, On unique ergodicity of degenerate diffusions, Stochastics 21 (1987), 41-61. MR 899954 (88g:60190)

[AL98]
M. Arisawa and P.-L. Lions, On ergodic stochastic control, Comm. Partial Differential Equations 23 (1998), 2187-2217. MR 1662180 (2000a:49052)

[All92]
G. Allaire, Homogenization and two-scale convergence, SIAM J. Math. Anal. 23 (1992), 1482-1518. MR 1185639 (93k:35022)

[Ari97]
M. Arisawa, Ergodic problem for the Hamilton-Jacobi-Bellman equation I, Ann. Inst. H. Poincaré Anal. Non Linéaire 14 (1997), 415-438. MR 1464529 (98k:49040)

[Ari98]
-, Ergodic problem for the Hamilton-Jacobi-Bellman equation II, Ann. Inst. H. Poincaré Anal. Non Linéaire 15 (1998), 1-24. MR 1614615 (99g:49029)

[Art99]
Z. Artstein, Invariant measures of differential inclusions applied to singular perturbations, J. Differential Equations 152 (1999), 289-307. MR 1674533 (99m:34023)

[Art04]
-, On the value function of singularly perturbed optimal control systems, Proceedings of the 43rd IEEE Conference on Decision and Control, Paradise Island, Bahamas, 2004, pp. 432-437.

[Bar]
M. Bardi, On differential games with long-time-average cost, Advances in dynamic games and their applications, Ann. Internat. Soc. Dynam. Games 10, Birkhäuser Boston, Inc., Boston, MA, 2009, pp. 3-18. MR 2521660

[Bar07]
G. Barles, Some homogenization results for non-coercive Hamilton-Jacobi equations, Calc. Var. Partial Differential Equations 30 (2007), 449-466. MR 2332422 (2008f:35011)

[BB98]
F. Bagagiolo and M. Bardi, Singular perturbation of a finite horizon problem with state-space constraints, SIAM J. Control Optim. 36 (1998), 2040-2060. MR 1638940 (99h:49040)

[BBG97]
G.K. Basak, V.S. Borkar, and M.K. Ghosh, Ergodic control of degenerate diffusions, Stochastic Anal. Appl. 15 (1997), 1-17. MR 1429854 (97j:93061)

[BCD97]
M. Bardi and I. Capuzzo-Dolcetta, Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations, Birkhäuser, Boston, 1997. MR 1484411 (99e:49001)

[Ben88]
A. Bensoussan, Perturbation methods in optimal control, Wiley/Gauthiers-Villars, Chichester, 1988. MR 949208 (89m:93002)

[Bet05]
P. Bettiol, On ergodic problem for Hamilton-Jacobi-Isaacs equations, ESAIM Control Optim. Calc. Var. 11 (2005), 522-541. MR 2167873 (2006k:35026)

[BG05]
V. Borkar and V. Gaitsgory, On existence of limit occupational measures set of a controlled stochastic differential equation, SIAM J. Control Optim. 44 (2005), 1436-1473. MR 2178039 (2006f:93103)

[BL99]
M. Bardi and F. Da Lio, On the strong maximum principle for fully nonlinear degenerate elliptic equations, Arch. Math (Basel) 73 (1999), 276-285. MR 1710100 (2001h:35024)

[BL01]
-, Propogation of maxima and strong maximum principle for viscosity solutions of degenerate elliptic equations. I: convex operators., Nonlinear Anal. Ser. A: Theory Methods 44 (2001), 991-1006. MR 1829815 (2002g:35088)

[BMT96]
M. Biroli, U. Mosco, and N. Tchou, Homogenization for degenerate operators with periodical coefficients with respect to the Heisenberg group, C. R. Acad. Sci. Paris Ser. I Math. 322 (1996), 439-444. MR 1381780 (97e:35015)

[BOFM92]
S. Brahim-Otsmane, G.A. Francfort, and F. Murat, Correctors for the homogenization of the wave and heat equations, J. Math. Pures Appl. 71 (1992), 197-231. MR 1172450 (93d:35012)

[Bon69]
J.M. Bony, Principe du maximum, inégalité de Harnack et unicité du problème de Cauchy pour les opérateurs elliptiques dégénérés, Ann. Inst. Fourier (Grenoble) 19 (1969), 277-304. MR 0262881 (41:7486)

[BS89]
T. Bielecki and Ł. Stettner, On ergodic control problems for singularly perturbed Markov processes, Appl. Math. Optim. 11 (1989), 131-161. MR 998401 (90c:93111)

[BW03]
I. Birindelli and J. Wigniolle, Homogenization of Hamilton-Jacobi equations in the Heisenberg group, Commun. Pure Appl. Anal. 2 (2003), 461-479. MR 2019062 (2004i:35022)

[CC95]
X. Cabré and L. Caffarelli, Fully nonlinear elliptic equations, Amer. Math. Soc., Providence, 1995. MR 1351007 (96h:35046)

[CDM88]
I. Capuzzo-Dolcetta and J.-L. Menaldi, Asymptotic behavior of the first order obstacle problem, J. Differential Equations 75 (1988), 303-328. MR 961158 (89j:49023)

[CFS82]
I.P. Cornfeld, S.V. Fomin, and Ya.G. Sinai, Ergodic theory, Springer-Verlag, Berlin, 1982. MR 832433 (87f:28019)

[CIL92]
M.G. Crandall, H. Ishii, and P.-L. Lions, User's guide to viscosity solutions of second order partial differential equations, Bull. Amer. Math. Soc. (N.S.) 27 (1992), 1-67. MR 1118699 (92j:35050)

[CK00]
F. Colonius and W. Kliemann, The dynamics of control, Birkhäuser, Boston, 2000. MR 1752730 (2001e:93001)

[CKS00]
M.G. Crandall, M. Kocan, and A. Świech, $ L^p$-theory for fully nonlinear uniformly parabolic equations, Comm. Partial Differential Equations 25 (2000), 1997-2053. MR 1789919 (2003b:35093)

[DaL04]
F. DaLio, Remarks on the strong maximum principle for degenerate parabolic equations, Comm. Pure Appl. Analysis 3 (2004), 395-415. MR 2098291 (2005i:35121)

[DZ93]
A.L. Dontchev and T. Zolezzi, Well-posed optimization problems, Lecture Notes in Mathematics, no. 1543, Springer-Verlag, Berlin, 1993. MR 1239439 (95a:49002)

[EI84]
L. Evans and H. Ishii, Differential games and nonlinear first order PDE on bounded domains, Manuscripta Math. 49 (1984), 109-139. MR 767202 (86f:35048)

[ES84]
L. Evans and P.E. Souganidis, Differential games and representation formulas for solutions of Hamilton-Jacobi-Isaacs equations, Indiana Univ. Math. J. 33 (1984), 773-797. MR 756158 (86d:90185)

[Eva89]
L. Evans, The perturbed test function method for viscosity solutions of nonlinear P.D.E., Proc. Roy. Soc. Edinburgh Sect. A 111 (1989), 359-375. MR 1007533 (91c:35017)

[Eva92]
-, Periodic homogenisation of certain fully nonlinear partial differential equations, Proc. Roy. Soc. Edinburgh Sect. A 120 (1992), 245-265. MR 1159184 (93a:35016)

[Eva04]
-, A survey of partial differential equations methods in weak KAM theory, Comm. Pure Appl. Math. 57 (2004), 445-480. MR 2026176 (2005b:49038)

[FM95]
W.H. Fleming and W.M. McEneaney, Risk-sensitive control on an infinite time horizon, SIAM J. Control Optim. 33 (1995), 1881-1915. MR 1358100 (96j:93070)

[FS89]
W.H. Fleming and P.E. Souganidis, On the existence of value functions of two-players, zero-sum stochastic differential games, Indiana Univ. Math. J. 38 (1989), 293-314. MR 997385 (90e:93089)

[FS06]
W.H. Fleming and H.M. Soner, Controlled Markov processes and viscosity solutions, 2nd edition, Springer-Verlag, Berlin, 2006. MR 2179357 (2006e:93002)

[FT02]
B. Franchi and M.C. Tesi, Two-scale homogenization in the Heisenberg group, J. Math. Pures Appl. 81 (2002), 495-532. MR 1912410 (2003d:35016)

[Gai92]
V. Gaitsgory, Suboptimization of singularly perturbed control systems, SIAM J. Control Optim. 30 (1992), 1228-1249. MR 1178660 (93h:49044)

[Gai96]
-, Limit Hamilton-Jacobi-Isaacs equations for singularly perturbed zero-sum differential games, J. Math. Anal. Appl. 202 (1996), 862-899. MR 1408359 (97i:90127)

[Gai04]
-, On a representation of the limit occupational measures set of a control system with applications to singularly perturbed control systems, SIAM J. Control Optim. 43 (2004), 325-340. MR 2082704 (2005g:49049)

[GL99]
V. Gaitsgory and A. Leizarowitz, Limit occupational measures set for a control system and averaging of singularly perturbed control systems, J. Math. Anal. Appl. 233 (1999), 461-475. MR 1689645 (2000c:93064)

[Gom07]
D.A. Gomes, Hamilton-Jacobi methods for vakonomic mechanics, NoDEA Nonlinear Differential Equations Appl. 14 (2007), 233-257. MR 2364892 (2009a:37121)

[Gru98]
L. Grüne, On the relation between discounted and average optimal value functions, J. Differential Equations 148 (1998), 65-99. MR 1637521 (99e:49004)

[GR05]
M.K. Ghosh and K.S.M. Rao, Differential games with ergodic payoff, SIAM J. Control Optim. 43 (2005), 2020-2035. MR 2179475 (2006g:91026)

[GT83]
D. Gilbarg and N.S. Trudinger, Elliptic partial differential equations of second order, 2nd ed., Springer-Verlag, New York, 1983. MR 737190 (86c:35035)

[Hor68]
L. Hörmander, Hypoelliptic second order differential equations, Acta Math. 119 (1968), 147-171. MR 0222474 (36:5526)

[Has80]
R.Z. Has'minskiĭ, Stochastic stability of differential equations, Sijthoff & Noordhoff, Alphen aan den Rijn, 1980. MR 600653 (82b:60064)

[IK74]
K. Ichihara and H. Kunita, A classification of the second order degenerate elliptic operators and its probabilistic characterization, Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 30 (1974), 235-254. MR 0381007 (52:1904)

[IK77]
-, Supplements and corrections to the paper: "A classification of the second order degenerate elliptic operators and its probabilistic characterization", Z. Wahrscheinlichkeitstheorie und Verw. Gebiete 39 (1977), 81-84. MR 0488328 (58:7877)

[IL90]
H. Ishii and P.-L. Lions, Viscosity solutions of fully nonlinear second-order elliptic partial differential equations, J. Differential Equations 83 (1990), 26-78. MR 1031377 (90m:35015)

[JKO94]
V.V. Jikov, S. M. Kozlov, and O.A. Oleinik, Homogenization of differential operators and integral functionals, Springer-Verlag, Berlin, 1994. MR 1329546 (96h:35003b)

[JL84]
R. Jensen and P.-L. Lions, Some asymptotic problems in fully nonlinear elliptic equations and stochastic control, Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4) 11 (1984), 129-176. MR 752583 (85m:49054)

[KKO86]
P.V. Kokotović, H.K. Khalil, and J. O'Reilly, Singular perturbation methods in control: analysis and design, Academic Press, London, 1986. MR 950486 (89e:93002)

[KP97]
Y. Kabanov and S. Pergamenshchikov, On convergence of attainability sets for controlled two-scale stochastic linear systems, SIAM J. Control Optim. 35 (1997), 134-159. MR 1430286 (97m:93069)

[KP03]
-, Two-scale stochastic systems. Asymptotic analysis and control, Springer-Verlag, Berlin, 2003. MR 1942740 (2004c:93003)

[KS98]
T.G. Kurtz and R.H. Stockbridge, Existence of Markov controls and characterization of optimal Markov controls, SIAM J. Control Optim. 36 (1998), 609-653. MR 1616514 (99b:93051)

[Kus90]
H.J. Kushner, Weak convergence methods and singularly perturbed stochastic control and filtering problems, Birkhäuser, Boston, 1990. MR 1102242 (92d:93003)

[Lei02]
A. Leizarowitz, Order reduction is invalid for singularly perturbed control problems with a vector fast variable, Math. Control Signals Systems 15 (2002), 101-119. MR 1910691 (2003c:49041)

[Lio82]
P.-L. Lions, Generalized solutions of Hamilton-Jacobi equations, Pitman, Boston, 1982.MR 0667669 (84a:49038)

[Lio83]
-, Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations. Part 1: The dynamic programming principle and applications, Part 2: Viscosity solutions and uniqueness, Comm. Partial Differential Equations 8 (1983), 1101-1174 and 1229-1276.MR 0709164 (85i:49043a); MR 0709162 (85i:49043b)

[Lio85]
-, Neumann type boundary conditions for Hamilton-Jacobi equations, Duke Math. J. 52 (1985), 793-820. MR 816386 (87h:35055)

[LPV86]
P.-L. Lions, G. Papanicolaou, and S.R.S. Varadhan, Homogeneization of Hamilton-Jacobi equations, Unpublished, 1986.

[LS05]
P.-L. Lions and P.E. Souganidis, Homogenization of degenerate second-order PDE in periodic and almost periodic environments and applications, Ann. Inst. H. Poincaré Anal. Non Linéaire 22 (2005), 667-677. MR 2171996 (2006d:35019)

[Nai02]
D.S. Naidu, Singular perturbations and time scales in control theory and applications: an overview, Dyn. Contin. Discrete Impuls. Syst. Ser. B Appl. Algorithms 9 (2002), 233-278. MR 1897791 (2003b:93060)

[O'M74]
R.E. O'Malley, Introduction to singular perturbations, Academic Press, New York-London, 1974. MR 0402217 (53:6038)

[QW03]
M. Quincampoix and F. Watbled, Averaging method for discontinuous Mayer's problem of singularly perturbed control systems, Nonlinear Anal. 54 (2003), 819-837. MR 1992506 (2004e:49040)

[Saf89]
M.V. Safonov, On the classical solution of nonlinear elliptic equations of second-order, Math. USSR-Izv. 33 (1989), 597-612, Engl. transl. of Izv. Akad. Nauk SSSR Ser. Mat., 52 (1988). MR 984219 (90d:35104)

[Sor93]
P. Soravia, Pursuit-evasion problems and viscosity solutions of Isaacs equations, SIAM J. Control Optim. 31 (1993), 604-623. MR 1214756 (94b:49056)

[Sub96]
N.N. Subbotina, Asymptotic properties of minimax solutions of Isaacs-Bellman equations in differential games with fast and slow motions, J. Appl. Math. Mech. 60 (1996), 883-890. MR 1627703 (99f:90209)

[Sub99]
-, Asymptotics of singularly perturbed Hamilton-Jacobi equations, J. Appl. Math. Mech. 63 (1999), 213-222. MR 1717733 (2000j:35035)

[Sub00]
-, Singular approximations of minimax and viscosity solutions to Hamilton-Jacobi equations, Proc. Steklov Inst. Math. Suppl. 1 (2000), S210-S227.MR 2066026 (2005e:35033)

[Sub01]
-, Asymptotics for singularly perturbed differential games, Game theory and applications, Vol. VII, Nova Sci. Publ., Huntington, NY, 2001, pp. 175-196. MR 1924394 (2003k:49078)

[Swi96]
A. Swiech, Another approach to the existence of value functions of stochastic differential games, J. Math. Anal. Appl. 204 (1996), 884-897. MR 1422779 (97j:90091)

[Ter08]
G. Terrone, Singular perturbation and homogenization problems in control theory, differential games and fully nonlinear partial differential equations, Ph.D. thesis, University of Padova, 2008.

[Tru88]
N.S. Trudinger, Comparison principles and pointwise estimates for viscosity solutions of nonlinear elliptic equations, Rev. Mat. Iberoamericana 4 (1988), 453-468. MR 1048584 (91k:35084)

[Tru89]
-, On regularity and existence of viscosity solutions of nonlinear second order elliptic equations, Partial Differential Equations and the Calculus of Variations (H. Brézis, ed.), vol. 2, Birkhäuser, Boston, 1989, pp. 938-957. MR 1034037 (90m:35041)

[Vel97]
V. Veliov, A generalization of the Tikhonov theorem for singularly perturbed differential inclusions, J. Dynam. Control Systems 3 (1997), 291-319. MR 1472355 (98h:34034)

[Wal82]
P. Walters, An introduction to ergodic theory, Springer-Verlag, New York, 1982. MR 648108 (84e:28017)

[Xu90]
C.-J. Xu, Subelliptic variational problems, Bull. Soc. Math. France 118 (1990), 147-169. MR 1087376 (92b:49008)

[YZ98]
G.G. Yin and Q. Zhang, Continuous-time Markov chains and applications. A singular perturbation approach, Springer-Verlag, New York, 1998. MR 1488963 (2000a:60142)

Similar Articles:

Retrieve articles in Memoirs of the American Mathematical Society with MSC (2000): 35Bxx, 35Kxx, 93C70, 49N70, 49L25, 60J60, 91A23, 93E20

Retrieve articles in all Journals with MSC (2000): 35Bxx, 35Kxx, 93C70, 49N70, 49L25, 60J60, 91A23, 93E20

Additional Information:

Olivier Alvarez
Affiliation: UMR 60-85, Université de Rouen, 76821 Mont-Saint Aignan cedex, France
Email: Olivier.Alvarez@univ-rouen.fr

Martino Bardi
Affiliation: Dipartimento di Matematica Pura ed Applicata, Università di Padova, via Trieste 63, 35121 Padova, Italy
Email: bardi@math.unipd.it

DOI: 10.1090/S0065-9266-09-00588-2
PII: S 0065-9266(09)00588-2
Keywords: Singular perturbations, differential games, viscosity solutions, dimension reduction, stochastic games, nonlinear parabolic equations, Hamilton-Jacobi equations, ergodic control, stabilization, homogenization, averaging, non-resonance, optimal control, controlled diffusion processes, oscillating initial data.
Received by editor(s): January 11, 2007
Posted: November 5, 2009
Additional Notes: The first author was supported in part by the French Ministry of Education, project ACI-JC 1025 ``Dislocation dynamics''. Affiliation at time of publication: UMR 60-85, Université de Rouen, 76821 Mont-Saint Aignan cedex, France. email: Olivier.Alvarez@univ-rouen.fr
The second author was supported in part by the M.I.U.R., project ``Viscosity, metric, and control theoretic methods for nonlinear partial differential equations''. Affiliation at time of publication: Dipartimento di Matematica Pura ed Applicata, Università di Padova, via Trieste 63, 35121 Padova, Italy. email: bardi@math.unipd.it
Copyright of article: Copyright 2009, American Mathematical Society




AMS and Social Media LinkedIn Facebook Podcasts Twitter YouTube RSS Feeds Blogs Wikipedia