Abstract
LetF(x,y) be a function of the vector variablesx∈R n andy∈R m. One possible scheme for minimizingF(x,y) is to successively alternate minimizations in one vector variable while holding the other fixed. Local convergence analysis is done for this vector (grouped variable) version of coordinate descent, and assuming certain regularity conditions, it is shown that such an approach is locally convergent to a minimizer and that the rate of convergence in each vector variable is linear. Examples where the algorithm is useful in clustering and mixture density decomposition are given, and global convergence properties are briefly discussed.
Similar content being viewed by others
References
Zangwill, W.,Nonlinear Programming: A Unified Approach, Prentice-Hall, Englewood Cliffs, New Jersey, 1969.
Stewart, G. W.,Introduction to Matrix Computations, Academic Press, Orlando, Florida, 1973.
Bezdek, J. C.,Pattern Recognition with Fuzzy Objective Function Algorithms, Plenum Press, New York, New York, 1981.
Hathaway, R. J.,Another Interpretation of the EM Algorithm, Statistics and Probability Letters, Vol. 4, pp. 53–56, 1986.
Redner, R. A., andWalker, H. F.,Mixture Densities, Maximum Likelihood, and the EM Algorithm, SIAM Review, Vol. 26, pp. 195–239, 1984.
Dennis, J. E., Jr., andSteihaug, T.,On the Successive Projections Approach to Least-Squares Problems, Technical Report No. 83-18, Department of Mathematical Sciences, Rice University, Houston, Texas, 1983.
Author information
Authors and Affiliations
Additional information
Communicated by R. A. Tapia
This research was supported in part by NSF Grant No. IST-84-07860. The authors are indebted to Professor R. A. Tapia for his help in improving this paper.
Rights and permissions
About this article
Cite this article
Bezdek, J.C., Hathaway, R.J., Howard, R.E. et al. Local convergence analysis of a grouped variable version of coordinate descent. J Optim Theory Appl 54, 471–477 (1987). https://doi.org/10.1007/BF00940196
Issue Date:
DOI: https://doi.org/10.1007/BF00940196