Abstract
New algorithms are presented for approximating the minimum of the sum of squares ofM real and differentiable functions over anN-dimensional space. These algorithms update estimates for the location of a minimum after each one of the functions and its first derivatives are evaluated, in contrast with other least-square algorithms which evaluate allM functions and their derivatives at one point before using any of this information to make an update. These new algorithms give estimates which fluctuate about a minimum rather than converging to it. For many least-square problems, they give an adequate approximation for the solution more quickly than do other algorithms.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Raphson, J.,Analysis Aequationum Universalis, London, England, 1690.
Gauss, K. F.,Theoria Motus Corporum Coelestium, Werke 7, pp. 240–254, Königlichen Gesellschaft der Wissenschaften, Goettingen, Germany, 1809.
Levenberg, K.,A Method for the Solution of Certain Non-linear Problems in Least Squares, Quarterly of Applied Mathematics, Vol. 2, pp. 164–168, 1944.
Marquardt, D. W.,An Algorithm for Least Squares Estimation, SIAM Journal on Applied Mathematics, Vol. 11, pp. 431–441, 1963.
Parker, F. D.,Inverses of Vandermonde Matrices, American Mathematical Monthly, Vol. 71, pp. 410–411, 1964.
Box, M. J.,A Comparison of Several Current Optimization Methods, Computer Journal, Vol. 9, pp. 67–77, 1966.
Brown, K., andDennis, J.,Derivative-Free Analogues of the Levenberg-Marquardt and Gauss Algorithms for Non-linear Least Squares Approximation, IBM Philadelphia Scientific Center, Technical Report No. 320-2994, 1970.
Fletcher, R., andPowell, M. J. D.,A Rapidly Convergent Descent Method for Minimization, Computer Journal, Vol. 6, pp. 163–168, 1963.
Brown, K., andDennis, J.,New Computational Algorithms for Minimizing a Sum of Squares of Non-linear Functions, Yale University, Department of Computer Science, Research Report No. 71-6, 1971.
Cheswick, J., Davidon, W., andSchoen, C.,Application of a New Least Squares Method to Structure Refinement, Acta Crystallographica (to appear).
Author information
Authors and Affiliations
Additional information
Communicated by D. R. Fulkerson
It is a pleasure to thank J. Chesick of Haverford College for suggesting and implementing the application of this algorithm to x-ray crystallography.
Rights and permissions
About this article
Cite this article
Davidon, W.C. New least-square algorithms. J Optim Theory Appl 18, 187–197 (1976). https://doi.org/10.1007/BF00935703
Issue Date:
DOI: https://doi.org/10.1007/BF00935703