Abstract
More than twenty years ago, Murray and Lootsma showed that Hessian matrices of the logarithmic barrier function become increasingly ill-conditioned at points on the barrier trajectory as the solution is approached. This paper explores some further characteristics of the barrier Hessian. We first show that, except in two special cases, the barrier Hessian is ill-conditioned in an entire region near the solution. At points in a more restricted region (including the barrier trajectory itself), this ill-conditioning displays a special structure connected with subspaces defined by the Jacobian of the active constraints. We then indicate how a Cholesky factorization with diagonal pivoting can be used to detect numerical rank-deficiency in the barrier Hessian, and to provide information about the underlying subspaces without making an explicit prediction of the active constraints. Using this subspace information, a close approximation to the Newton direction can be calculated by solving linear systems whose condition reflects that of the original problem.
Similar content being viewed by others
References
T.F. Coleman and D.C. Sorensen, “A note on the computation of an orthonormal basis for the null space of a matrix,”Mathematical Programming 29 (1984) 234–242.
J.E. Dennis, Jr. and R.B. Schnabel,Numerical Methods for Unconstrained Optimization and Nonlinear Equations (Prentice-Hall, Englewood Cliffs, New Jersey, 1983).
A.V. Fiacco, “Barrier methods for nonlinear programming,” in:Operations Research Support Methodology (A.G. Holzman, ed.), (Marcel Dekker, New York, 1979) 377–440.
A.V. Fiacco and G.P. McCormick,Nonlinear Programming: Sequential Unconstrained Minimization Techniques (John Wiley and Sons, New York, 1968), (Republished by SIAM, Philadelphia, 1990).
R. Fletcher,Practical Methods of Optimization (second edition), (John Wiley and Sons, Chichester, 1987).
R. Fletcher and A.P. McCann, “Acceleration techniques for nonlinear programming,” in:Optimization (R. Fletcher, ed.), (Academic Press, New York, 1969) 203–213.
P.E. Gill, W. Murray and M.H. Wright,Practical Optimization (Academic Press, London and New York, 1981).
P.E. Gill, W. Murray and M.H. Wright,Numerical Linear Algebra and Optimization, Volume 1 (Addison-Wesley, Redwood City, California, 1991).
G.H. Golub and C.F. Van Loan,Matrix Computations (second edition), (Johns Hopkins University Press, Baltimore, Maryland, 1989).
C.C. Gonzaga, “Path following methods for linear programming,”SIAM Review 34 (1992) 167–224.
N.J. Higham, “Analysis of the Cholesky decomposition of a semi-definite matrix,” in:Reliable Numerical Computation (M.G. Cox and S. Hammarling, eds.), (Clarendon Press, Oxford, 1990) 161–185.
N.K. Karmarkar, “A new polynomial time algorithm for linear programming,”Combinatorica 4 (1984) 373–395.
F.A. Lootsma, Hessian matrices of penalty functions for solving constrained optimization problems,Philips Research Reports 24 (1969) 322–331.
W. Murray, “Analytical expressions for the eigenvalues and eigenvectors of the Hessian matrices of barrier and penalty functions,”Journal on Optimization Theory and Applications 7 (1971) 189–196.
W. Murray and M.H. Wright, Projected Lagrangian methods based on the trajectories of penalty and barrier functions, Report SOL 78-23, Department of Operations Research, Stanford University, Stanford, California, 1978.
S.G. Nash and A. Sofer, “A barrier method for large-scale constrained optimization,”ORSA Journal on Computing 5 (1993) 40–53.
C.R. Papadimitriou and K. Steiglitz,Combinatorial Optimization: Algorithms and Complexity (Prentice-Hall, Englewood Cliffs, New Jersey, 1982).
R.B. Schnabel and E. Eskow, “A new modified Cholesky factorization,”SIAM Journal on Scientific and Statistical Computing 11 (1990) 1136–1158.
G.W. Stewart and J. Sun,Matrix Perturbation Theory (Academic Press, Boston, 1990).
J.H. Wilkinson,The Algebraic Eigenvalue Problem (The Clarendon Press, Oxford, 1965).
M.H. Wright,Numerical Methods for Nonlinearly Constrained Optimization, Ph. D. thesis, Stanford University, California, 1976.
M.H. Wright, Interior methods for constrained optimization, in:Acta Numerica 1992 (A. Iserles, ed.), (Cambridge University Press, New York, 1992) 341–407.
M.H. Wright, Why a pure primal Newton barrier step may be infeasible, Numerical Analysis Manuscript 93-02, AT&T Bell Laboratories, Murray Hill, New Jersey; to appear inSIAM Journal on Optimization (1994).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Wright, M.H. Some properties of the Hessian of the logarithmic barrier function. Mathematical Programming 67, 265–295 (1994). https://doi.org/10.1007/BF01582224
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01582224