Abstract
Bounded variation stochastic control may be defined to include any stochastic control problem in which one restricts the cumulative displacement of the state caused by control to be of bounded variation on finite time intervals. In classical control problems, this cumulative displacement is the integral of the control process (or some function of it), and so is absolutely continuous. In impulse control (see Bensoussan & Lions (1978)), this cumulative displacement has jumps, between which it is either constant or absolutely continuous. Bounded variation control admits both these possibilities and also the possibility that the displacement of the state caused by the optimal control is singularly continuous, at least with positive probability over some interval of time. Problems which exhibit this feature will be called singular, and these are the objects of interest of the present paper.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
F.M. Baldursson,“Singular stochastic control and optimal stopping”, Stochastic, to. appear (1986).
J.A. Bather, “A continuous time inventory model”, J. Appl. Prob. 3, 538–549 (1966).
J.A. Bather, “A diffusion model for the control of a dam”, J. Appl. Prob. 5, 55–71 (1968).
J.A. Bather and H. Chernoff, “Sequential decisions in the control of a spaceship”, Proc. Fifth Berkeley Symposium on Mathematical Statistics and Probability 3, 181–207 (1967a).
J.A. Bather and H. Chernoff, “Sequential decisions in the control of a spaceship (finite fuel)”, J. Appl. Prob. 4, 584–604 (1967b).
V.E. Benes, L.A. Shepp and H.S. Witsenhausen, “Some solvable stochastic control problems”, Stochastics 4, 181–207 (1980).
A. Bensoussan and J.L. Lions, Applications des inéquations varia-tionnelles en contrôle stochastique, Dunod, Paris (1978).
M.I. Borodowski, A.D. Bratus’ and F.L. Chernous’ko, “Optimal impulse correction under random perturbation”, J. Appl. Math. Mech. 39, 797–805 (1975).
A.S. Bratus’, “Solutions of certain optimal correction problems with error of execution of the control action”, J. App. Math. Mech. 38, 433–440 (1974).
A.S. Bratus’ and F.L. Chernous’ko, “Numerical solution of optimal correction problems under random perturbations”, (English translation), Pergamon Press, J. USSR Comput. Mat. mat. Phys. 14, 1 (1974).
H. Chernoff, “Optimal stochastic control”, Sankhya, Ser. A, 30, 221–252 (1968).
F.L. Chernous’ko, “Optimum correction under active disturbances”, J. Appl. Math. Mech. 32, 203–208 (1968).
F.L. Chernous’ko, “Self-similar solutions of the Gellman equation for optimal correction of random disturbances”, J. Appl. Math. Mech. 35, 333–342 (1971).
P.-L. Chow, J.-L. Menaldi, and M. Robin, “Additive control of stochastic linear systems with finite horizons”, SIAM J. Control and Optimization 23, 858–899 (1985).
E.G. Coffman, Jr. and M.I. Reiman, “Diffusion approximations for computer/communication systems”, in G. Iazeolla, P.G. Courtois and A. Hordijk (eds.), Mathematical Computer Performance and Reliability, North Holland, Amsterdam, 33–53 (1984).
R.J. Elliott, Stochastic Calculus and Applications, Springer, New York.
M.J. Faddy, “Optimal control of finite dams: continuous output procedure”, Adv. Appl. Prob. 6, 689–710 (1974).
J.M. Harrison, Brownian Motion and Stochastic Flow Systems, Wiley, New York (1985a).
J.M. Harrison, “Brownian networks as approximate models of multi-class networks of queues”, preprint prepared for the Institute of Mathematical Analysis Workshop, June 1986.
J.M. Harrison, T.M. Sellke and A.J. Taylor, “Impulse control of Brownian motion”, Math. Operations Research 8, 454–466 (1983).
J.M. Harrison and M.I. Taksar, “Instantaneous control of a Brownian motion”, Math. Operations Research 8, 439–453 (1983).
J.M. Harrison and A.J. Taylor, “Optimal control of a Brownian storage system”, Stoch. Proc. Appl. 6, 179–194 (1978).
J.M. Harrison and R.J. Williams, “Reflected Brownian motion in a polyhedral domain: stationary distributions with exponential densities”, preprint (1985).
A. Heinricher, “A singular stochastic control problem arising from a deterministic problem with non-Lipschitzian minimizers”, Ph.D. Dissertation, Dept. of Mathematics, Carnegie Mellon University (1986).
V.A. Iaroshevskii and S.V. Petukhov, “Optimal one-parametric correction of the trajectories of spacecrafts”, Kosmicheskie Issledovaniia 8, 4 (1970).
D.E. Okhotsimskii, D.E. Riasin, and N.N. Chentsov, “Optimal strategies in corrections”, Dokl Akad. Nauk SSSR 175, 1 (1967).
D. Iglehart and W. Whitt, “Multiple channel queues in heavy traffic”, I and II, Adv. App. Prob. 2, 150–177 and 355–364 (1970).
S.D. Jacka, “A finite fuel stochastic control problem”, Stochastics 10, 103–113 (1983).
I. Karatzas, “The monotone follower problem in stochastic decision theory”, App. Math. Optim. 7, 175–189 (1981).
I. Karatzas, “A class of singular stochastic control problems”, Adv. Appl. Prob. 15, 225–254 (1983).
I. Karatzas, “Probabilistic aspects of finite-fuel stochastic control”, Proc. Natl. Acad. Sci. USA 82, 5579–5581 (1985).
I. Karatzas and S.E. Shreve, “Connections between optimal stopping and singular stochastic control I. Monotone follower problems”, SIAM J. Control and Optimization 22, 856–877 (1984).
I. Karatzas and S.E. Shreve, “Connections between optimal stopping and singular stochastic control II. Reflected follower problems”, SIAM J. Control and Optimization 23, 433–451 (1985).
I. Karatzas and S.E. Shreve, “Equivalent models for finite-fuel stochastic control”, Stochastics, to appear (1986).
J.P. Lehoczky and S.E. Shreve, “Absolutely continuous and singular stochastic control”, Stochastics 17, 91–109 (1986).
A.J. Lemoine, “Networks of queues-A survey of weak convergence results”, Management Sci. 24, 1175–1193 (1978).
D. Ludwig, “Optimal harvesting of randomly fluctuating resource I. Application of perturbation methods”, SIAM J. Appl. Math. 37, 166–184 (1979).
J.L. Menaldi and M. Robin, “On some cheap control problems for diffusion processes”, Trans. Amer. Math. Soc. 278, 771–802 (1983). See also C.R. Acad. Sc. Paris, Serie I, 294 (1982) 541–544.
P.A. Meyer, Ed., Lecture Notes in Mathematics 511, Séminaire de Probabilités X, Université de Strasbourg, Springer, New York (1976).
P. van Moerbeke, On optimal stopping and free boundary problems“, Arch. Rational Mech. Anal. 60, 101–148 (1976).
S.R. Pliska, “A diffusion model for the optimal operation of a reservoir system”, J. Appl Prob. 12, 859–863 (1975).
M.L. Puterman, “A diffusion process model for a storage system”, TIMS Studies in Management Sciences 1, 859–863.
M. Reiman, “Open queuing networks in heavy traffic, Math. Operations Research 9, 441–458 (1984).
A.N. Shiryaev, Optimal Stopping Rules, Springer, New York (1978).
S.E. Shreve, J.P. Lehoczky and D.P. Gaver, “Optimal consumption for general diffusions with absorbing and reflecting barriers”, SIAM J. Control and Optimization 22, 55–75 (1984).
M. Taksar, “Average optimal singular control and a related stopping problem”, Math Operations Research 10, 63–81 (1985).
S.R.S. Varadhan and R.J. Williams, “Brownian motion in a wedge with oblique reflection”, Comm. Pure Appl. Math. 38, 405–443 (1985).
W. Whitt, “Heavy traffic theorems for queues: A survey”, in Mathematical Methods in Queueing Theory, A.B. Clarke, ed., Springer, New York (1974).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1988 Springer-Verlag New York Inc.
About this paper
Cite this paper
Shreve, S.E. (1988). An Introduction to Singular Stochastic Control. In: Fleming, W., Lions, PL. (eds) Stochastic Differential Systems, Stochastic Control Theory and Applications. The IMA Volumes in Mathematics and Its Applications, vol 10. Springer, New York, NY. https://doi.org/10.1007/978-1-4613-8762-6_30
Download citation
DOI: https://doi.org/10.1007/978-1-4613-8762-6_30
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4613-8764-0
Online ISBN: 978-1-4613-8762-6
eBook Packages: Springer Book Archive