Controlled semi-Markov fields with graph-structured compact state space
Authors:
H. Daduna, P. S. Knopov and R. K. Chorney
Translated by:
The authors
Original publication:
Teoriya Imovirnostei ta Matematichna Statistika, tom 69 (2003).
Journal:
Theor. Probability and Math. Statist. 69 (2004), 39-53
MSC (2000):
Primary 60K15, 60K35, 90C40
DOI:
https://doi.org/10.1090/S0094-9000-05-00612-5
Published electronically:
February 7, 2005
MathSciNet review:
2110903
Full-text PDF Free Access
Abstract | References | Similar Articles | Additional Information
Abstract: We introduce locally acting distributed decision makers in the theory of semi-Markov decisions for systems for which both the domain and the action space are general and compact. Such decision makers are characterized by making decisions on the basis of the information gathered at their local neighborhood only. The state transient function of the system also is of a local structure. We consider general holding times of the systems and this results in semi-Markov properties in time. The neighborhood structure of the systems resembles in space the Markov property of spatial processes. Under some regularity assumptions, we reduce the optimal problems within the set of local strategies to the corresponding problems for deterministic Markov strategies.
- 1. Erhan Çinlar, Introduction to stochastic processes, Prentice-Hall, Inc., Englewood Cliffs, N.J., 1975. MR 0380912
- 2. Ralph L. Disney and Peter C. Kiessler, Traffic processes in queueing networks, Johns Hopkins Series in the Mathematical Sciences, vol. 4, Johns Hopkins University Press, Baltimore, MD, 1987. A Markov renewal approach. MR 896033
- 3. G. Daduna, P. S. Knopov, and R. K. Chorneĭ, Local control of Markov processes of interaction on a graph with a compact set of states, Kibernet. Sistem. Anal. 3 (2001), 62–77, 189 (Russian, with English and Ukrainian summaries); English transl., Cybernet. Systems Anal. 37 (2001), no. 3, 348–360. MR 1872136, https://doi.org/10.1023/A:1011985609994
- 4. L. G. Gubenko and È. S. Štatland, Controllable semi-Markov processes, Kibernetika (Kiev) 2 (1972), 26–29 (Russian, with English summary). MR 0323423
- 5. L. G. Gubenko and È. S. Štatland, On discrete time Markov decision processes, Teor. Verojatnost. i Mat. Statist. 7 (1972), 51–64, 163 (Russian, with English summary). MR 0334957
- 6. K. Hinderer, Foundations of non-stationary dynamic programming with discrete time parameter, Lecture Notes in Operations Research and Mathematical Systems, Vol. 33, Springer-Verlag, Berlin-New York, 1970. MR 0267890
- 7. Ronald A. Howard, Research in semi-Markovian decision structures, J. Operations Res. Soc. Japan 6 (1963/64), 163–199. MR 172699
- 8. William S. Jewell, Markov-renewal programming. I. Formulation, finite return models, Operations Res. 11 (1963), 938–948. MR 163374, https://doi.org/10.1287/opre.11.6.938
- 9. William S. Jewell, Markov-renewal programming. II. Infinite return models, example, Operations Res. 11 (1963), 949–971. MR 163375, https://doi.org/10.1287/opre.11.6.949
- 10. M. Kitaev, Elimination of randomization in semi-Markov decision, Optimization 18 (1987), no. 3, 439–446. MR 882524, https://doi.org/10.1080/02331938708843259
- 11. Mikhail Yu. Kitaev and Vladimir V. Rykov, Controlled queueing systems, CRC Press, Boca Raton, FL, 1995. MR 1413045
- 12. K. Kuratowski, Topology. Vol. II, New edition, revised and augmented. Translated from the French by A. Kirkor, Academic Press, New York-London; Państwowe Wydawnictwo Naukowe Polish Scientific Publishers, Warsaw, 1968. MR 0259835
- 13. Fernando Luque-Vásquez and Ma. Teresa Robles Alcaraz, Controlled semi-Markov models with discounted unbounded costs, Bol. Soc. Mat. Mexicana (2) 39 (1994), no. 1-2, 51–68. MR 1338681, https://doi.org/10.1016/0167-6687(94)90691-2
- 14. R. T. Rockafellar, Measurable dependence of convex sets and functions on parameters, J. Math. Anal. Appl. 28 (1969), 4–25. MR 247019, https://doi.org/10.1016/0022-247X(69)90104-8
- 15. Sheldon M. Ross, Average cost semi-Markov decision processes, J. Appl. Probability 7 (1970), 649–656. MR 303635, https://doi.org/10.1017/s0021900200110617
- 16. N. B. Vasilyev, Bernoulli and Markov stationary measures in discrete local interactions, Developments in statistics, Vol. 1, Academic Press, New York, 1978, pp. 99–112. MR 505437
- 17. Oscar Vega-Amaya, Average optimality in semi-Markov control models on Borel spaces: unbounded cost and controls, Bol. Soc. Mat. Mexicana (2) 38 (1993), no. 1-2, 47–60. MR 1313106
- 18. K. Wakuta, Arbitrary state semi-Markov decision processes with unbounded rewards, Optimization 18 (1987), no. 3, 447–454. MR 882525, https://doi.org/10.1080/02331938708843260
Retrieve articles in Theory of Probability and Mathematical Statistics with MSC (2000): 60K15, 60K35, 90C40
Retrieve articles in all journals with MSC (2000): 60K15, 60K35, 90C40
Additional Information
H. Daduna
Affiliation:
Universität Hamburg, Fakultät Mathematik, Bundesstrasse 55, D–20146 Hamburg, Germany
Email:
amos801@math.uni-Hamburg.de
P. S. Knopov
Affiliation:
Glushkov Institute for Cybernetics, National Academy of Sciences of Ukraine, Academician Glushkov Avenue 40, Kiev–187, 03680, Ukraine
Email:
knopov1@yahoo.com
R. K. Chorney
Affiliation:
Inter-Regional Academy of Personnel Management, Department of Mathematics, Frometivs’ka Street 2, Kiev–39, 03039, Ukraine
DOI:
https://doi.org/10.1090/S0094-9000-05-00612-5
Keywords:
Semi-Markov processes,
Markov renewal processes,
optimal control,
average asymptotic reward,
renewal reward processes,
random fields,
local strategies
Received by editor(s):
January 24, 2003
Published electronically:
February 7, 2005
Additional Notes:
The research of the second author was partially supported by a grant from Deutsche Forschungsgemeinschaft at Hamburg University
Article copyright:
© Copyright 2005
American Mathematical Society
