A variational characterization of the optimal exit rate for controlled diffusions
Authors:
Ari Arapostathis and Vivek S. Borkar
Journal:
Theor. Probability and Math. Statist. 102 (2020), 5-19
MSC (2020):
Primary 60J25, 49R05, 60J60
DOI:
https://doi.org/10.1090/tpms/1125
Published electronically:
March 29, 2021
Full-text PDF
Abstract |
References |
Similar Articles |
Additional Information
Abstract: The main result in this paper is a variational formula for the exit rate from a bounded domain for a diffusion process in terms of the stationary law of the diffusion constrained to remain in this domain forever. Related results on the geometric ergodicity of the controlled $Q$-process are also presented.
References
- V. Anantharam and V. S. Borkar, A variational formula for risk-sensitive reward, SIAM J. Control Optim. 55 (2017), no. 2, 961–988. MR 3629428, DOI https://doi.org/10.1137/151002630
- A. Arapostathis and A. Biswas, Risk-sensitive control for a class of diffusions with jumps, ArXiv e-prints 1910.05004 (2019), available at https://arxiv.org/abs/1910.05004.
- Ari Arapostathis and Anup Biswas, A variational formula for risk-sensitive control of diffusions in $\Bbb R^d$, SIAM J. Control Optim. 58 (2020), no. 1, 85–103. MR 4048004, DOI https://doi.org/10.1137/18M1218704
- Ari Arapostathis, Anup Biswas, Vivek S. Borkar, and K. Suresh Kumar, A variational characterization of the risk-sensitive average reward for controlled diffusions on $\Bbb R^d$, SIAM J. Control Optim. 58 (2020), no. 6, 3785–3813. MR 4188837, DOI https://doi.org/10.1137/20M1329202
- A. Arapostathis and V. S. Borkar, ‘Controlled’ versions of the Collatz–Wielandt and Donsker–Varadhan formulae. In: Joshua V., Varadhan S., Vishnevsky V. (eds), Applied Probability and Stochastic Processes, pp. 199–214. Springer, Singapore, 2020.
- A. Arapostathis, H. Hmedi, and G. Pang, On uniform exponential ergodicity of Markovian multiclass many-server queues in the Halfin–Whitt regime, Math. Oper. Res. (2021), to appear, available at https://arxiv.org/abs/1812.03528.
- Ari Arapostathis and Anup Biswas, Infinite horizon risk-sensitive control of diffusions without any blanket stability assumptions, Stochastic Process. Appl. 128 (2018), no. 5, 1485–1524. MR 3780687, DOI https://doi.org/10.1016/j.spa.2017.08.001
- Ari Arapostathis, Anup Biswas, and Subhamay Saha, Strict monotonicity of principal eigenvalues of elliptic operators in $\Bbb {R}^d$ and risk-sensitive control, J. Math. Pures Appl. (9) 124 (2019), 169–219 (English, with English and French summaries). MR 3926044, DOI https://doi.org/10.1016/j.matpur.2018.05.008
- Ari Arapostathis, Vivek S. Borkar, and Mrinal K. Ghosh, Ergodic control of diffusion processes, Encyclopedia of Mathematics and its Applications, vol. 143, Cambridge University Press, Cambridge, 2012. MR 2884272
- Ari Arapostathis, Vivek S. Borkar, and K. Suresh Kumar, Risk-sensitive control and an abstract Collatz-Wielandt formula, J. Theoret. Probab. 29 (2016), no. 4, 1458–1484. MR 3571250, DOI https://doi.org/10.1007/s10959-015-0616-x
- Getachew K. Befekadu and Panos J. Antsaklis, On the asymptotic estimates for exit probabilities and minimum exit rates of diffusion processes pertaining to a chain of distributed control systems, SIAM J. Control Optim. 53 (2015), no. 4, 2297–2318. MR 3376771, DOI https://doi.org/10.1137/140990322
- V. E. Beneš, Existence of optimal strategies based on specified information, for a class of stochastic decision problems, SIAM J. Control 8 (1970), 179–188. MR 0265043
- H. Berestycki, L. Nirenberg, and S. R. S. Varadhan, The principal eigenvalue and maximum principle for second-order elliptic operators in general domains, Comm. Pure Appl. Math. 47 (1994), no. 1, 47–92. MR 1258192, DOI https://doi.org/10.1002/cpa.3160470105
- Anup Biswas and Vivek S. Borkar, On a controlled eigenvalue problem, Systems Control Lett. 59 (2010), no. 11, 734–735. MR 2767905, DOI https://doi.org/10.1016/j.sysconle.2010.08.009
- Nicolas Champagnat and Denis Villemonais, Exponential convergence to quasi-stationary distribution and $Q$-process, Probab. Theory Related Fields 164 (2016), no. 1-2, 243–283. MR 3449390, DOI https://doi.org/10.1007/s00440-014-0611-7
- Nicolas Champagnat and Denis Villemonais, Uniform convergence to the $Q$-process, Electron. Commun. Probab. 22 (2017), Paper No. 33, 7. MR 3663104, DOI https://doi.org/10.1214/17-ECP63
- L. Collatz, Einschliessungssatz für die charakteristischen Zahlen von Matrizen, Math. Z. 48 (1942), 221–226 (German). MR 8590, DOI https://doi.org/10.1007/BF01180013
- Pierre Collet, Servet Martínez, and Jaime San Martín, Quasi-stationary distributions, Probability and its Applications (New York), Springer, Heidelberg, 2013. Markov chains, diffusions and dynamical systems. MR 2986807
- Amir Dembo and Ofer Zeitouni, Large deviations techniques and applications, Stochastic Modelling and Applied Probability, vol. 38, Springer-Verlag, Berlin, 2010. Corrected reprint of the second (1998) edition. MR 2571413
- Monroe D. Donsker and S. R. S. Varadhan, On a variational formula for the principal eigenvalue for operators with maximum principle, Proc. Nat. Acad. Sci. U.S.A. 72 (1975), 780–783. MR 361998, DOI https://doi.org/10.1073/pnas.72.3.780
- D. Down, S. P. Meyn, and R. L. Tweedie, Exponential and uniform ergodicity of Markov processes, Ann. Probab. 23 (1995), no. 4, 1671–1691. MR 1379163
- Pedro Echeverría, A criterion for invariant measures of Markov processes, Z. Wahrsch. Verw. Gebiete 61 (1982), no. 1, 1–16. MR 671239, DOI https://doi.org/10.1007/BF00537221
- Stewart N. Ethier and Thomas G. Kurtz, Markov processes, Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics, John Wiley & Sons, Inc., New York, 1986. Characterization and convergence. MR 838085
- Wendell H. Fleming, Exit probabilities and optimal stochastic control, Appl. Math. Optim. 4 (1977/78), no. 4, 329–346. MR 512217, DOI https://doi.org/10.1007/BF01442148
- W. H. Fleming and M. R. James, Asymptotic series and exit time probabilities, Ann. Probab. 20 (1992), no. 3, 1369–1384. MR 1175266
- Wendell H. Fleming and Panagiotis E. Souganidis, A PDE approach to asymptotic estimates for optimal exit probabilities, Stochastic differential systems (Marseille-Luminy, 1984) Lect. Notes Control Inf. Sci., vol. 69, Springer, Berlin, 1985, pp. 281–285. MR 798331, DOI https://doi.org/10.1007/BFb0005083
- Wendell H. Fleming and Chun Ping Tsai, Optimal exit probabilities and differential games, Appl. Math. Optim. 7 (1981), no. 3, 253–282. MR 635802, DOI https://doi.org/10.1007/BF01442120
- Guang Lu Gong, Min Ping Qian, and Zhong Xin Zhao, Killed diffusions and their conditioning, Probab. Theory Related Fields 80 (1988), no. 1, 151–167. MR 970476, DOI https://doi.org/10.1007/BF00348757
- István Gyöngy and Nicolai Krylov, Existence of strong solutions for Itô’s stochastic equations via approximations, Probab. Theory Related Fields 105 (1996), no. 2, 143–158. MR 1392450, DOI https://doi.org/10.1007/BF01203833
- Naoyuki Ichihara, Large time asymptotic problems for optimal stochastic control with superlinear cost, Stochastic Process. Appl. 122 (2012), no. 4, 1248–1275. MR 2914752, DOI https://doi.org/10.1016/j.spa.2011.12.005
- Robert Knobloch and Lothar Partzsch, Uniform conditional ergodicity and intrinsic ultracontractivity, Potential Anal. 33 (2010), no. 2, 107–136. MR 2658978, DOI https://doi.org/10.1007/s11118-009-9161-5
- Sylvie Méléard and Denis Villemonais, Quasi-stationary distributions and population processes, Probab. Surv. 9 (2012), 340–410. MR 2994898, DOI https://doi.org/10.1214/11-PS191
- Ross G. Pinsky, On the convergence of diffusion processes conditioned to remain in a bounded region for large time to limiting positive recurrent diffusion processes, Ann. Probab. 13 (1985), no. 2, 363–378. MR 781410
- Helmut Wielandt, Unzerlegbare, nicht negative Matrizen, Math. Z. 52 (1950), 642–648 (German). MR 35265, DOI https://doi.org/10.1007/BF02230720
References
- V. Anantharam and V. S. Borkar, A variational formula for risk-sensitive reward, SIAM J. Control Optim. 55 (2017), no. 2, 961–988. MR 3629428
- A. Arapostathis and A. Biswas, Risk-sensitive control for a class of diffusions with jumps, ArXiv e-prints 1910.05004 (2019), available at https://arxiv.org/abs/1910.05004.
- Ari Arapostathis and Anup Biswas, A variational formula for risk-sensitive control of diffusions in $\mathbb {R}^d$, SIAM J. Control Optim. 58 (2020), no. 1, 85–103. MR 4048004
- Ari Arapostathis, Anup Biswas, Vivek S. Borkar, and K. Suresh Kumar, A variational characterization of the risk-sensitive average reward for controlled diffusions on $\Bbb R^d$, SIAM J. Control Optim. 58 (2020), no. 6, 3785–3813. MR 4188837
- A. Arapostathis and V. S. Borkar, ‘Controlled’ versions of the Collatz–Wielandt and Donsker–Varadhan formulae. In: Joshua V., Varadhan S., Vishnevsky V. (eds), Applied Probability and Stochastic Processes, pp. 199–214. Springer, Singapore, 2020.
- A. Arapostathis, H. Hmedi, and G. Pang, On uniform exponential ergodicity of Markovian multiclass many-server queues in the Halfin–Whitt regime, Math. Oper. Res. (2021), to appear, available at https://arxiv.org/abs/1812.03528.
- Ari Arapostathis and Anup Biswas, Infinite horizon risk-sensitive control of diffusions without any blanket stability assumptions, Stochastic Process. Appl. 128 (2018), no. 5, 1485–1524. MR 3780687
- Ari Arapostathis, Anup Biswas, and Subhamay Saha, Strict monotonicity of principal eigenvalues of elliptic operators in $\mathbb {R}^d$ and risk-sensitive control, J. Math. Pures Appl. (9) 124 (2019), 169–219. MR 3926044
- Ari Arapostathis, Vivek S. Borkar, and Mrinal K. Ghosh, Ergodic control of diffusion processes, Encyclopedia of Mathematics and its Applications, vol. 143, Cambridge University Press, Cambridge, 2012. MR 2884272
- Ari Arapostathis, Vivek S. Borkar, and K. Suresh Kumar, Risk-sensitive control and an abstract Collatz-Wielandt formula, J. Theoret. Probab. 29 (2016), no. 4, 1458–1484. MR 3571250
- Getachew K. Befekadu and Panos J. Antsaklis, On the asymptotic estimates for exit probabilities and minimum exit rates of diffusion processes pertaining to a chain of distributed control systems, SIAM J. Control Optim. 53 (2015), no. 4, 2297–2318. MR 3376771
- V. E. Beneš, Existence of optimal strategies based on specified information, for a class of stochastic decision problems, SIAM J. Control 8 (1970), 179–188. MR 0265043
- H. Berestycki, L. Nirenberg, and S. R. S. Varadhan, The principal eigenvalue and maximum principle for second-order elliptic operators in general domains, Comm. Pure Appl. Math. 47 (1994), no. 1, 47–92. MR 1258192
- Anup Biswas and Vivek S. Borkar, On a controlled eigenvalue problem, Systems Control Lett. 59 (2010), no. 11, 734–735. MR 2767905
- Nicolas Champagnat and Denis Villemonais, Exponential convergence to quasi-stationary distribution and $Q$-process, Probab. Theory Related Fields 164 (2016), no. 1-2, 243–283. MR 3449390
- Nicolas Champagnat and Denis Villemonais, Uniform convergence to the $Q$-process, Electron. Commun. Probab. 22 (2017), Paper No. 33, 7. MR 3663104
- L. Collatz, Einschliessungssatz für die charakteristischen Zahlen von Matrizen, Math. Z. 48 (1942), 221–226 (German). MR 8590
- Pierre Collet, Servet Martínez, and Jaime San Martín, Quasi-stationary distributions: Markov chains, diffusions and dynamical systems, Probability and its Applications, Springer, Heidelberg, 2013. MR 2986807
- Amir Dembo and Ofer Zeitouni, Large deviations techniques and applications, Stochastic Modelling and Applied Probability, vol. 38, Springer-Verlag, Berlin, 2010. Corrected reprint of the second (1998) edition. MR 2571413
- Monroe D. Donsker and S. R. S. Varadhan, On a variational formula for the principal eigenvalue for operators with maximum principle, Proc. Nat. Acad. Sci. U.S.A. 72 (1975), 780–783. MR 361998
- D. Down, S. P. Meyn, and R. L. Tweedie, Exponential and uniform ergodicity of Markov processes, Ann. Probab. 23 (1995), no. 4, 1671–1691. MR 1379163
- Pedro Echeverría, A criterion for invariant measures of Markov processes, Z. Wahrsch. Verw. Gebiete 61 (1982), no. 1, 1–16. MR 671239
- Stewart N. Ethier and Thomas G. Kurtz, Markov processes: Characterization and convergence, Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, Inc., New York, 1986. MR 838085
- Wendell H. Fleming, Exit probabilities and optimal stochastic control, Appl. Math. Optim. 4 (1977/78), no. 4, 329–346. MR 512217
- W. H. Fleming and M. R. James, Asymptotic series and exit time probabilities, Ann. Probab. 20 (1992), no. 3, 1369–1384. MR 1175266
- Wendell H. Fleming and Panagiotis E. Souganidis, A PDE approach to asymptotic estimates for optimal exit probabilities, Stochastic differential systems (Marseille-Luminy, 1984) Lect. Notes Control Inf. Sci., vol. 69, Springer, Berlin, 1985, pp. 281–285. MR 798331
- Wendell H. Fleming and Chun Ping Tsai, Optimal exit probabilities and differential games, Appl. Math. Optim. 7 (1981), no. 3, 253–282. MR 635802
- Guang Lu Gong, Min Ping Qian, and Zhong Xin Zhao, Killed diffusions and their conditioning, Probab. Theory Related Fields 80 (1988), no. 1, 151–167. MR 970476
- István Gyöngy and Nicolai Krylov, Existence of strong solutions for Itô’s stochastic equations via approximations, Probab. Theory Related Fields 105 (1996), no. 2, 143–158. MR 1392450
- Naoyuki Ichihara, Large time asymptotic problems for optimal stochastic control with superlinear cost, Stochastic Process. Appl. 122 (2012), no. 4, 1248–1275. MR 2914752
- Robert Knobloch and Lothar Partzsch, Uniform conditional ergodicity and intrinsic ultracontractivity, Potential Anal. 33 (2010), no. 2, 107–136. MR 2658978
- Sylvie Méléard and Denis Villemonais, Quasi-stationary distributions and population processes, Probab. Surv. 9 (2012), 340–410. MR 2994898
- Ross G. Pinsky, On the convergence of diffusion processes conditioned to remain in a bounded region for large time to limiting positive recurrent diffusion processes, Ann. Probab. 13 (1985), no. 2, 363–378. MR 781410
- Helmut Wielandt, Unzerlegbare, nicht negative Matrizen, Math. Z. 52 (1950), 642–648 (German). MR 35265
Similar Articles
Retrieve articles in Theory of Probability and Mathematical Statistics
with MSC (2020):
60J25,
49R05,
60J60
Retrieve articles in all journals
with MSC (2020):
60J25,
49R05,
60J60
Additional Information
Ari Arapostathis
Affiliation:
Department of Electrical and Computer Engineering, The University of Texas at Austin, 2501 Speedway, EER 7.824, Austin, Texas 78712
MR Author ID:
26760
Email:
ari@utexas.edu
Vivek S. Borkar
Affiliation:
Department of Electrical Engineering, Indian Institute of Technology, Powai, Mumbai 400076, India
Email:
borkar@ee.iitb.ac.in
Keywords:
Killed diffusions,
exit rate,
principal eigenvalue,
$Q$-process,
quasi-stationarity.
Received by editor(s):
November 7, 2019
Published electronically:
March 29, 2021
Additional Notes:
The work of the first author was supported in part by the Army Research Office through grant W911NF-17-1-001, in part by the National Science Foundation through grant DMS-1715210, and in part by the Office of Naval Research through grant N00014-16-1-2956 and was approved for public release under DCN# 43-6053-19. The work of the second author was supported by a J. C. Bose Fellowship.
Article copyright:
© Copyright 2020
Taras Shevchenko National University of Kyiv
