Skip to main content
SpringerLink
Log in

A Note on the Alternating Direction Method of Multipliers

  • Published:
Journal of Optimization Theory and Applications Aims and scope Submit manuscript

Abstract

We consider the linearly constrained separable convex programming, whose objective function is separable into m individual convex functions without coupled variables. The alternating direction method of multipliers has been well studied in the literature for the special case m=2, while it remains open whether its convergence can be extended to the general case m≥3. This note shows the global convergence of this extension when the involved functions are further assumed to be strongly convex.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite element approximations. Comput. Math. Appl. 2, 17–40 (1976)

    Article  MATH  Google Scholar 

  2. Gabay, D.: Applications of the method of multipliers to variational inequalities. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems, pp. 299–331. North-Holland, Amsterdam (1983)

    Chapter  Google Scholar 

  3. Chen, G., Teboulle, M.: A proximal-based decomposition method for convex minimization problems. Math. Program. 64, 81–101 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  4. Eckstein, J., Bertsekas, D.P.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  5. Fukushima, M.: Application of the alternating directions method of multipliers to separable convex programming problems. Comput. Optim. Appl. 2, 93–111 (1992)

    Article  MathSciNet  Google Scholar 

  6. He, B.S., Liao, L.Z., Han, D.R., Yang, H.: A new inexact alternating direction method for monotone variational inequalities. Math. Program. 92, 103–118 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  7. Kontogiorgis, S., Meyer, R.R.: A variable-penalty alternating directions method for convex optimization. Math. Program. 83, 29–53 (1998)

    MathSciNet  MATH  Google Scholar 

  8. Tseng, P.: Applications of a splitting algorithm to decomposition in convex programming and variational inequalities. SIAM J. Control Optim. 29, 119–138 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  9. Chan, R.H., Yang, J.F., Yuan, X.M.: Alternating direction method for image inpainting in wavelet domain. SIAM J. Imaging Sci. 4, 807–826 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  10. Chen, C.H., He, B.S., Yuan, X.M.: Matrix completion via alternating direction method. IMA J. Numer. Anal. 32, 227–245 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  11. Esser, E.: Applications of Lagrangian-based alternating direction methods and connections to split Bregman. UCLA CAM Report 09-31 (2009)

  12. He, B.S., Xu, M.H., Yuan, X.M.: Solving large-scale least squares covariance matrix problems by alternating direction methods. SIAM J. Matrix Anal. Appl. 32, 136–152 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  13. Ng, M.K., Weiss, P.A., Yuan, X.M.: Solving constrained total-variation problems via alternating direction methods. SIAM J. Sci. Comput. 32, 2710–2736 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  14. Sun, J., Zhang, S.: A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs. Eur. J. Oper. Res. 207, 1210–1220 (2010)

    Article  MATH  Google Scholar 

  15. Tao, M., Yuan, X.M.: Recovering low-rank and sparse components of matrices from incomplete and noisy observations. SIAM J. Optim. 21, 57–81 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  16. Yang, J.F., Zhang, Y.: Alternating direction algorithms for ℓ 1 problems in compressive sensing. SIAM J. Sci. Comput. 33, 250–278 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  17. Yuan, X.M., Yang, J.F.: Sparse and low-rank matrix decomposition via alternating direction method. Pac. J. Optim. (to appear)

  18. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Faund. Trends Mach. Learn. 3, 1–122 (2010)

    Article  MATH  Google Scholar 

  19. Bardsley, J.M., Knepper, S., Nagy, J.: Structured linear algebra problems in adaptive optics imaging. Adv. Comput. Math. 5(2–4), 103–117 (2011)

    Article  MathSciNet  Google Scholar 

  20. Kiwiel, K.C., Rosa, C.H., Ruszczyński, A.: Proximal decomposition via alternating linearization. SIAM J. Optim. 9, 668–689 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  21. Setzer, S., Steidl, G., Tebuber, T.: Deblurring Poissonian images by split Bregman techniques. J. Vis. Commun. Image Represent. 21, 193–199 (2010)

    Article  Google Scholar 

  22. Peng, Y.G., Ganesh, A., Wright, J., Xu, W.L., Ma, Y.: Robust alignment by sparse and low-rank decomposition for linearly correlated images. IEEE Trans. Pattern Anal. Mach. Intell. (to appear)

  23. He, B.S., Tao, M., Yuan, X.M.: Alternating direction method with Gaussian back substitution for separable convex programming. SIAM J. Optim. (to appear)

  24. He, B.S., Tao, M., Xu, M.H., Yuan, X.M.: Alternating directions based contraction method for generally separable linearly constrained convex programming problems. Optimization (to appear)

  25. Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)

    MATH  Google Scholar 

  26. Facchinei, F., Pang, J.S.: Finite-Dimensional Variational Inequalities and Complementarity Problems, vols. I and II. Springer, New York (2003)

    Google Scholar 

Download references

Acknowledgements

The research was supported by the National Natural Science Foundation of China (NSFC) grants 11071122, 11171159, Doctoral Found of Ministry of Education of China 20103207110002, and the Hong Kong General Research Grant: HKBU203311.

Author information

Authors and Affiliations

  1. School of Mathematical Science and Key Laboratory for NSLSCS of Jiangsu Province, Nanjing Normal University, Nanjing, 210046, P.R. China

    Deren Han

  2. Department of Mathematics, Hong Kong Baptist University, Kowloon, Hong Kong, China

    Xiaoming Yuan

Authors
  1. Deren Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoming Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoming Yuan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Han, D., Yuan, X. A Note on the Alternating Direction Method of Multipliers. J Optim Theory Appl 155, 227–238 (2012). https://doi.org/10.1007/s10957-012-0003-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10957-012-0003-z

Keywords

Search

Navigation