On the existence of good Markov strategies

Author:
Theodore Preston Hill

Journal:
Trans. Amer. Math. Soc. **247** (1979), 157-176

MSC:
Primary 60G40

DOI:
https://doi.org/10.1090/S0002-9947-1979-0517690-9

MathSciNet review:
517690

Full-text PDF

Abstract | References | Similar Articles | Additional Information

Abstract: In contrast to the known fact that there are gambling problems based on a finite state space for which no stationary family of strategies is at all good, in every such problem there always exist -optimal Markov families (in which the strategy depends only on the current state and time) and also -optimal tracking families (in which the strategy depends only on the current state and the number of times that state has been previously visited). More generally, this result holds for all finite state gambling problems with a payoff which is shift and permutation invariant.

**[1]**D. Blackwell,*Discounted dynamic programming*, Ann. Math. Statist.**36**(1965), 226-235. MR**0173536 (30:3749)****[2]**J. L. Doob,*Stochastic processes*, Wiley, New York, 1953. MR**0058896 (15:445b)****[3]**L. E. Dubins and L. J. Savage,*Inequalities for stochastic processes*, Dover, New York, 1976. MR**0410875 (53:14618)****[4]**L. E. Dubins and W. D. Sudderth,*An example in which stationary strategies are not adequate*, Ann. Probability**3**(1975), 722-725. MR**0378098 (51:14267)****[5]**-,*Persistently -optimal strategies*, Math. Operations Research**2**(1977), 125-134. MR**0456570 (56:14794)****[6]**W. Feller,*An introduction to probability theory and its applications*, vol. 1, Wiley, New York, 1965.**[7]**P. A. Meyer,*Probability and potentials*, Blaisdell, Waltham, Mass., 1966. MR**0205288 (34:5119)****[8]**D. Ornstein,*On the existence of stationary optimal strategies*, Proc. Amer. Math. Soc.**20**(1969), 563-569. MR**0253756 (40:6970)****[9]**R. Purves and W. D. Sudderth,*Some finitely additive probability*, Ann. Probability**4**(1976), 259-276. MR**0402888 (53:6702)****[10]**W. D. Sudderth,*On the existence of good stationary strategies*, Trans. Amer. Math. Soc.**135**(1969), 399-415. MR**0233595 (38:1916)****[11]**-,*On measurable gambling problems*, Ann. Math. Statist.**42**(1971), 260-269. MR**0282404 (43:8116)**

Retrieve articles in *Transactions of the American Mathematical Society*
with MSC:
60G40

Retrieve articles in all journals with MSC: 60G40

Additional Information

DOI:
https://doi.org/10.1090/S0002-9947-1979-0517690-9

Keywords:
Markov chain,
gambling theory,
strategy,
stationary strategy,
Markov strategy,
stochastic process,
dynamic programming,
optimization,
decision theory,
control theory

Article copyright:
© Copyright 1979
American Mathematical Society