Abstract
This paper addresses the long-term average cost control of continuous time Markov processes. A survey of problems and methods contained in various works is given for continuous control, optimal stopping, and impulse control.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bellman R.: J. Math. Mech. 6 (1957), 679–684.
Bensoussan A.: Stochastic Control by Functional Analysis Methods, North-Holland, Amsterdam, 1982.
Bensoussan A. and Lions J. L.: Applications des inéquations variationnelles en contrôle stochastique, Dunod, Paris, 1978.
Bensoussan A. and Lions J. L.: Contrôle impulsionnel et inéquations quasi-variationnelles, Dunod, Paris, 1982.
Bensoussan A. and Lions J. L.: ‘Nouvelles formulations de problèmes de contrôle impulsionnel et applications’, CRAS, Paris, 1973, pp. 1189–1192.
Bensoussan A. and Lions J. L.: in Theory of Nonlinear Operators, Akademic Verlag, Berlin, 1978.
Bismut J. M. and Skalli B.: Z. Wahz. Verw. G. 39 (1977), 301–313.
Blackwell D. Ann. Math. Stat. 33 (1962), 719–726.
Doshi B. T.: Stoch. Proc. Appl. 4 (1976), 55–77.
Dynkin E. B.: Markov Processes, Vols. I, II, Springer-Verlag, Heidelberg, 1965.
Dynkin E. B.: Dokl. Acad. Nauk USSR, 150 (1963), 238–240.
Dynkin E. B. and Yuskevich A. A.: Controlled Markov Processes, Springer-Verlag, Berlin, 1979.
El Karoui, N.: Mèthodes probabilistes en contrôle stochastique, to be published.
Fayolle G. and Robin M.: in Modeling and Performance Evaluation of Computer Systems, North-Holland, Amsterdam, 1976.
Federgruen A., Hordijk A. and Tijms H. C.: Stoch. Proc. Appl. 9 (1979), 223–235.
Fleming W. and Rishel R.: Optimal Deterministic and Stochastic Control, Springer-Verlag, Berlin, 1975.
Has'minskii, R. Z.: Stochastic Stability of Differential Equations, Sijthoff & Noordhoff, 1980.
Hordijk, A.: ‘Dynamic Programming and Markov Potential Theory’, Mathematical Center Tr. No. 51, Amsterdam, 1974.
Howard R. A.: Dynamic Programming and Markov Processes, MIT Press. Cambridge, Mass, 1960.
Kakumanu P. K.: Siam J. Control 10 (1972), 210–220.
Karatzas, J.: ‘A Class of Singular Stochastic Control Problem’, to appear.
Kogan Y. A.: Theor. Proba. Applic. 14 (1969), 496–502.
Lanery, E.: ‘Etude asymptotique des systèmes markoviens à commande’, Revue inf. Recherche opérationnelle lère annee, No. 5, pp. 3–56.
Lasry, J. M.: ‘Contrôle stochastique ergodique’, Thèse Université de Paris IX, 1974.
Lepeltier, J. P. and Marchal, B.: ‘Théorie générale du contrôle impulsionnel’, Thèse, 1980.
Lippman S. A.: Ann. Math. Stat. 42 (1971), 1717–1726.
Lorenzani, M.: ‘Optimal Stopping for Markov Processes’, Note. Rend. Cont. Acad. Sci., Rome, 1981.
Mandl P.: Analytic Treatment of One-Dimensional Markov Processes, Springer-Verlag, New York, 1968.
Makowski, A.: ‘Local Optimality Conditions for Optimal Stopping’, CSR-81-23/1981, Dept of Elect. Eng. University of Maryland.
Menaldi, J. L.: ‘On the Degenerate variational Inequality with Neumann Boundary Conditions, Siam J. Control Opt., to be published.
Menaldi J. L.: Siam J. Control Opt. 18 (1980), 697–721.
Miller B.: J. Appl. Prob. 18 (1981), 361–377.
Prabhu N. and Stidham S.: in Mathematical Methods in Queueing Theory, (Lecture Notes in Economics and Mathematical Systems, 98) Springer-Verlag, Heidelberg, 1974.
Puterman, M.: ‘On the Optimal Control of Diffusion Processes’, TR No. 24, Stanford University, 1972.
Robin, M.: ‘Contrôle impulsionnel des processus de Markov’, Thesis, University of Paris IX, 1978.
Robin M.: Siam J. Control Opt. 19 (1981), 333–358.
Robin M.: in System Modeling and Optimization. (Lecture Notes in Control and Information Sciences, 38), Springer-Verlag, Heidelberg, 1982.
Robin, M.: ‘Asymptotics in Quasi-Variational Inequalities and Ergodic Control Problems’, to be published, Systems and Control Letters 1983.
Rosberg Z., Varayia P. and Walrand J.: IEEE Tr. Auto-Control, AC 27 (1982), 600–610.
Ross S. M.: Ann. Math. Stat. 42 (1971), 1767–1768.
Ross S. M.: J. Appl. Prob. 7 (1970), 649–656.
Ross S. M.: Applied Probability Models with Optimization Applications, Holden Day, San Francisco, 1970.
Schweitzer P.: ‘A Non-Expansive Mapping Approach to Undiscounted Semi-Markovian Decision Processes’, WP. 8109, 1981, Graduate School of Management, University of Rochester, U.S.A.
Stroock D. and Varadhan S.: Comm. Pure Appl. Math. 24 (1971), 147–225.
Sulem, A.: to be published.
Stettner, L.: ‘On Impulsive Control with Long-Run Average Cost Criterion’, to appear in Studia Mathematica.
Shiryaev A. N.: Optimal Stopping Rules, Springer-Verlag, New York, 1978.
Tarres, R.: ‘Comportement asymptotique d'un problème de contrôle stochastique’, Cahier de math. de la Décision No. 8215, Université de Paris IX, 1982.
Vanderduyn-Shouten F. A. and Hordijk A.: ‘On the Existence of Average Optimal Policies in Markov Decision Drift Processes with General State and Action Space’, Vrije Universiteit, Amsterdam, TR No. 73, 1981.
Wijngaard J.: Math. Oper. Res. 2 (1977), 91–102.
Yu Chung Liao- ‘On Optimal Control of a Brownian Motion and Approximation of Queueing Processes’, PhD Thesis, Brown University, R.I. 1982.
Yuskevich A. A.: Theor. Proba. Applic. 26 (1981); 796–802.
Zabczyk J.: in Stochastic Control Theory and Stochastic Differential Systems (Lecture Notes in Control and Information Sciences 16), Springer-Verlag, Heidelberg, 1979.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Robin, M. Long-term average cost control problems for continuous time Markov processes: A survey. Acta Appl Math 1, 281–299 (1983). https://doi.org/10.1007/BF00046603
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF00046603