On the existence of good Markov strategies

Author:
Theodore Preston Hill

Journal:
Trans. Amer. Math. Soc. **247** (1979), 157-176

MSC:
Primary 60G40

MathSciNet review:
517690

Full-text PDF Free Access

Abstract | References | Similar Articles | Additional Information

Abstract: In contrast to the known fact that there are gambling problems based on a finite state space for which no stationary family of strategies is at all good, in every such problem there always exist -optimal Markov families (in which the strategy depends only on the current state and time) and also -optimal tracking families (in which the strategy depends only on the current state and the number of times that state has been previously visited). More generally, this result holds for all finite state gambling problems with a payoff which is shift and permutation invariant.

**[1]**David Blackwell,*Discounted dynamic programming*, Ann. Math. Statist.**36**(1965), 226–235. MR**0173536****[2]**J. L. Doob,*Stochastic processes*, John Wiley & Sons, Inc., New York; Chapman & Hall, Limited, London, 1953. MR**0058896****[3]**Lester E. Dubins and Leonard J. Savage,*Inequalities for stochastic processes (how to gamble if you must)*, Dover Publications, Inc., New York, 1976. Corrected republication of the 1965 edition. MR**0410875****[4]**Lester E. Dubins and William D. Sudderth,*An example in which stationary strategies are not adequate*, Ann. Probability**3**(1975), no. 4, 722–725. MR**0378098****[5]**Lester E. Dubins and William D. Sudderth,*Persistently 𝜖-optimal strategies*, Math. Oper. Res.**2**(1977), no. 2, 125–134. MR**0456570****[6]**W. Feller,*An introduction to probability theory and its applications*, vol. 1, Wiley, New York, 1965.**[7]**Paul-A. Meyer,*Probability and potentials*, Blaisdell Publishing Co. Ginn and Co., Waltham, Mass.-Toronto, Ont.-London, 1966. MR**0205288****[8]**Donald Ornstein,*On the existence of stationary optimal strategies*, Proc. Amer. Math. Soc.**20**(1969), 563–569. MR**0253756**, 10.1090/S0002-9939-1969-0253756-8**[9]**Roger A. Purves and William D. Sudderth,*Some finitely additive probability*, Ann. Probability**4**(1976), no. 2, 259–276. MR**0402888****[10]**William D. Sudderth,*On the existence of good stationary strategies*, Trans. Amer. Math. Soc.**135**(1969), 399–414. MR**0233595**, 10.1090/S0002-9947-1969-0233595-9**[11]**William D. Sudderth,*On measurable gambling problems*, Ann. Math. Statist.**42**(1971), 260–269. MR**0282404**

Retrieve articles in *Transactions of the American Mathematical Society*
with MSC:
60G40

Retrieve articles in all journals with MSC: 60G40

Additional Information

DOI:
https://doi.org/10.1090/S0002-9947-1979-0517690-9

Keywords:
Markov chain,
gambling theory,
strategy,
stationary strategy,
Markov strategy,
stochastic process,
dynamic programming,
optimization,
decision theory,
control theory

Article copyright:
© Copyright 1979
American Mathematical Society