Abstract
The aim of this paper is to study the convergence properties of the gradient projection method and to apply these results to algorithms for linearly constrained problems. The main convergence result is obtained by defining a projected gradient, and proving that the gradient projection method forces the sequence of projected gradients to zero. A consequence of this result is that if the gradient projection method converges to a nondegenerate point of a linearly constrained problem, then the active and binding constraints are identified in a finite number of iterations. As an application of our theory, we develop quadratic programming algorithms that iteratively explore a subspace defined by the active constraints. These algorithms are able to drop and add many constraints from the active set, and can either compute an accurate minimizer by a direct method, or an approximate minimizer by an iterative method of the conjugate gradient type. Thus, these algorithms are attractive for large scale problems. We show that it is possible to develop a finite terminating quadratic programming algorithm without non-degeneracy assumptions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
P. Bertsekas, “On the Goldstein-Levitin-Polyak gradient projection method,”IEEE Transactions on Automatic Control 21 (1976) 174–184.
D.P. Bertsekas, “Projected Newton methods for optimization problems with simple constraints,”SIAM Journal on Control and Optimization 20 (1982) 221–246.
J.V. Burke and J.J. Moré, “On the identification of active constraints,” Argonne National Laboratory, Mathematics and Computer Science Division Report ANL/MCS-TM-82, (Argonne, IL, 1986).
R.S. Dembo and U. Tulowitzki, “On the minimization of quadratic functions subject to box constraints,” Working Paper Series B#71, School of Organization and Management, Yale University, (New Haven, CT, 1983).
R.S. Dembo and U. Tulowitzki, “Local convergence analysis for successive inexact quadratic programming methods,” Working Paper Series B#78, School of Organization and Management, Yale University, (New Haven, CT, 1984).
R.S. Dembo and U. Tulowitzki, “Sequential truncated quadratic programming methods,”, in: P.T. Boggs, R.H. Byrd and R.B. Schnabel, eds.,Numerical Optimization 1984 (Society of Industrial and Applied Mathematics, Philadelphia, 1985) pp. 83–101.
J.C. Dunn, “Global and asymptotic convergence rate estimates for a class of projected gradient processes,”SIAM Journal on Control and Optimization 19 (1981), 368–400.
J.C. Dunn, “On the convergence of projected gradient processes to singular critical points,”Journal of Optimization Theory and Applications (1986) to appear.
R. Fletcher,Practical Methods of Optimization Volume 2: Constrained Optimization (John Wiley & Sons, New York, 1981).
E.M. Gafni and D.P. Bertsekas, “Convergence of a gradient projection method,” Massachusetts Institute of Technology, Laboratory for Information and Decision Systems Report LIDS-P-1201 (Cambridge, Massachusetts, 1982).
E.M. Gafni and D.P. Bertsekas, “Two-metric projection methods for constrained optimization,”SIAM Journal on Control and Optimization 22 (1984) 936–964.
P.E. Gill, W. Murray and M.H. Wright,Practical Optimization (Academic Press, New York, 1981).
A.A. Goldstein, “Convex programming in Hilbert space,”Bulletin of the American Mathematical Society 70 (1964) 709–710.
E.S. Levitin and B.T. Polyak, “Constrained minimization problems”,USSR Computational Mathematics and Mathematical Physics 6 (1966), 1–50.
G.P. McCormick and R.A. Tapia, “The gradient projection method under mild differentiability conditions,”SIAM Journal on Control 10 (1972) 93–98.
D.P. O'Leary, “A generalized conjugate gradient algorithm for solving a class of quadratic programming problems,”Linear Algebra and its Applications 34 (1980) 371–399.
R.R. Phelps “The gradient projection method using Curry's steplength,”SIAM Journal on Control and Optimization 24 (1986) 692–699.
B.T. Polyak, “The conjugate gradient method in extremal problems,”USSR Computational Mathematics and Mathematical Physics 9 (1969) 94–112.
E.H. Zarantonello, “Projections on convex sets in Hilbert space and spectral theory,” in: E.H. Zarantonello, ed.,Contributions to Nonlinear Functional Analysis (Academic Press, New York, 1971).
Author information
Authors and Affiliations
Additional information
Work supported in part by the Applied Mathematical Sciences subprogram of the Office of Energy Research of the U.S. Department of Energy under Contract W-31-109-Eng-38.
Work supported in part by the Applied Mathematical Sciences subprogram of the Office of Energy Research of the U.S. Department of Energy under Contract W-31-109-Eng-38.
Rights and permissions
About this article
Cite this article
Calamai, P.H., Moré, J.J. Projected gradient methods for linearly constrained problems. Mathematical Programming 39, 93–116 (1987). https://doi.org/10.1007/BF02592073
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02592073