Remote Access Transactions of the American Mathematical Society
Green Open Access

Transactions of the American Mathematical Society

ISSN 1088-6850(online) ISSN 0002-9947(print)



On the existence of good Markov strategies

Author: Theodore Preston Hill
Journal: Trans. Amer. Math. Soc. 247 (1979), 157-176
MSC: Primary 60G40
MathSciNet review: 517690
Full-text PDF

Abstract | References | Similar Articles | Additional Information

Abstract: In contrast to the known fact that there are gambling problems based on a finite state space for which no stationary family of strategies is at all good, in every such problem there always exist $ \varepsilon $-optimal Markov families (in which the strategy depends only on the current state and time) and also $ \varepsilon $-optimal tracking families (in which the strategy depends only on the current state and the number of times that state has been previously visited). More generally, this result holds for all finite state gambling problems with a payoff which is shift and permutation invariant.

References [Enhancements On Off] (What's this?)

  • [1] D. Blackwell, Discounted dynamic programming, Ann. Math. Statist. 36 (1965), 226-235. MR 0173536 (30:3749)
  • [2] J. L. Doob, Stochastic processes, Wiley, New York, 1953. MR 0058896 (15:445b)
  • [3] L. E. Dubins and L. J. Savage, Inequalities for stochastic processes, Dover, New York, 1976. MR 0410875 (53:14618)
  • [4] L. E. Dubins and W. D. Sudderth, An example in which stationary strategies are not adequate, Ann. Probability 3 (1975), 722-725. MR 0378098 (51:14267)
  • [5] -, Persistently $ \varepsilon $-optimal strategies, Math. Operations Research 2 (1977), 125-134. MR 0456570 (56:14794)
  • [6] W. Feller, An introduction to probability theory and its applications, vol. 1, Wiley, New York, 1965.
  • [7] P. A. Meyer, Probability and potentials, Blaisdell, Waltham, Mass., 1966. MR 0205288 (34:5119)
  • [8] D. Ornstein, On the existence of stationary optimal strategies, Proc. Amer. Math. Soc. 20 (1969), 563-569. MR 0253756 (40:6970)
  • [9] R. Purves and W. D. Sudderth, Some finitely additive probability, Ann. Probability 4 (1976), 259-276. MR 0402888 (53:6702)
  • [10] W. D. Sudderth, On the existence of good stationary strategies, Trans. Amer. Math. Soc. 135 (1969), 399-415. MR 0233595 (38:1916)
  • [11] -, On measurable gambling problems, Ann. Math. Statist. 42 (1971), 260-269. MR 0282404 (43:8116)

Similar Articles

Retrieve articles in Transactions of the American Mathematical Society with MSC: 60G40

Retrieve articles in all journals with MSC: 60G40

Additional Information

Keywords: Markov chain, gambling theory, strategy, stationary strategy, Markov strategy, stochastic process, dynamic programming, optimization, decision theory, control theory
Article copyright: © Copyright 1979 American Mathematical Society

American Mathematical Society