An efficient algorithm for the $\ell _{p}$ norm based metric nearness problem
HTML articles powered by AMS MathViewer
- by Peipei Tang, Bo Jiang and Chengjing Wang;
- Math. Comp. 94 (2025), 2495-2532
- DOI: https://doi.org/10.1090/mcom/4026
- Published electronically: November 8, 2024
- PDF | Request permission
Abstract:
Given a dissimilarity matrix, the metric nearness problem is to find the nearest matrix of distances that satisfy the triangle inequalities. This problem has wide applications, such as sensor networks, image processing, and so on. But it is of great challenge even to obtain a moderately accurate solution due to the $O(n^{3})$ metric constraints and the nonsmooth objective function which is usually a weighted $\ell _{p}$ norm based distance. In this paper, we propose a delayed constraint generation method with each subproblem solved by the semismooth Newton based proximal augmented Lagrangian method (PALM) for the metric nearness problem. Due to the high memory requirement for the storage of the matrix related to the metric constraints, we take advantage of the special structure of the matrix and do not need to store the corresponding constraint matrix. A pleasing aspect of our algorithm is that we can solve these problems involving up to $10^{8}$ variables and $10^{13}$ constraints. Numerical experiments demonstrate the efficiency of our algorithm.
Concerning the theory, firstly, under a mild condition, we establish a calmness condition which is very essential for the analysis of local convergence rate of PALM. Secondly, we prove the equivalence between the dual nondegeneracy condition and nonsingularity of the generalized Jacobian for the inner subproblem of PALM. Thirdly, for the $\ell _{1}$ or $\ell _{\infty }$ norm based problem, without the strict complementarity condition, we also prove the equivalence between the dual nondegeneracy condition and the uniqueness of the primal solution.
References
- Nikhil Bansal, Avrim Blum, and Shuchi Chawla, Correlation clustering, Mach. Learn. 56 (2004), no. 1-3, 89–113. MR 3363423, DOI 10.1023/B:MACH.0000033116.57574.95
- D. Batra, R. Sukthankar, and T. Chen, Semi-supervised clustering via learnt codeword distances, Proceedings of the British Machine Vision Conference 2008, Leeds, September 2008.
- H. H. Bauschke and J. M. Borwein, On the convergence of von Neumann’s alternating projection algorithm for two sets, Set-Valued Anal. 1 (1993), no. 2, 185–212. MR 1239403, DOI 10.1007/BF01027691
- Roger Behling, Yunier Bello-Cruz, and Luiz-Rafael Santos, Infeasibility and error bound imply finite convergence of alternating projections, SIAM J. Optim. 31 (2021), no. 4, 2863–2892. MR 4336330, DOI 10.1137/20M1358669
- Roger Behling and Alfredo Iusem, The effect of calmness on the solution set of systems of nonlinear equations, Math. Program. 137 (2013), no. 1-2, Ser. A, 155–165. MR 3010423, DOI 10.1007/s10107-011-0486-7
- D. Bertsimas and J. N. Tsitsiklis, Introduction to Linear Optimization, vol. 6, Athena Scientific, Belmont, MA, 1997.
- J. Frédéric Bonnans and Alexander Shapiro, Perturbation analysis of optimization problems, Springer Series in Operations Research, Springer-Verlag, New York, 2000. MR 1756264, DOI 10.1007/978-1-4612-1394-9
- Justin Brickell, Inderjit S. Dhillon, Suvrit Sra, and Joel A. Tropp, The metric nearness problem, SIAM J. Matrix Anal. Appl. 30 (2008), no. 1, 375–396. MR 2399586, DOI 10.1137/060653391
- F. H. Clarke, Optimization and nonsmooth analysis, 2nd ed., Classics in Applied Mathematics, vol. 5, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1990. MR 1058436, DOI 10.1137/1.9781611971309
- Laurent Condat, Fast projection onto the simplex and the $l_1$ ball, Math. Program. 158 (2016), no. 1-2, Ser. A, 575–585. MR 3511394, DOI 10.1007/s10107-015-0946-6
- Timothy A. Davis and Yifan Hu, The University of Florida sparse matrix collection, ACM Trans. Math. Software 38 (2011), no. 1, Art. 1, 25. MR 2865011, DOI 10.1145/2049662.2049663
- I. S. Dhillon, S. Sra, and J. A. Tropp, Triangle fixing algorithms for the metric nearness problem, 17th International Conference on Neural Information Processing Systems (NIPS’04), MIT Press, Cambridge, MA, USA, 2004, pp. 361–368, 2004.
- A. L. Dontchev and R. T. Rockafellar, Implicit Functions and Solution Mappings: A View from Variational Analysis, Springer, New York, 2009.
- D. Drusvyatskiy, A. D. Ioffe, and A. S. Lewis, Transversality and alternating projections for nonconvex sets, Found. Comput. Math. 15 (2015), no. 6, 1637–1651. MR 3413631, DOI 10.1007/s10208-015-9279-3
- Richard L. Dykstra, An algorithm for restricted least squares regression, J. Amer. Statist. Assoc. 78 (1983), no. 384, 837–842. MR 727568, DOI 10.1080/01621459.1983.10477029
- Jonathan Eckstein and Dimitri P. Bertsekas, On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators, Math. Programming 55 (1992), no. 3, Ser. A, 293–318. MR 1168183, DOI 10.1007/BF01581204
- René Escalante and Marcos Raydan, Alternating projection methods, Fundamentals of Algorithms, vol. 8, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2011. MR 2849884, DOI 10.1137/1.9781611971941
- Francisco Facchinei and Jong-Shi Pang, Finite-dimensional variational inequalities and complementarity problems. Vol. II, Springer Series in Operations Research, Springer-Verlag, New York, 2003. MR 1955649
- D. Gabay and B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite element approximation, Comput. Math. Appl. 2 (1976), 17–40.
- M. Gabidolla, A. Iskakov, M. F. Demirci, and A. Yazici, On approximating metric nearness through deep learning, Artificial Intelligence and Soft Computing, ICAISC 2019, Lecture Notes in Computer Science, vol. 11508, Springer, Cham, 2019, pp. 62–72.
- C. Gentile, Distributed sensor location through linear programming with triangle inequality constraints, IEEE Trans. Wirel. Commun. 6 (2007), no. 7, 2572–2581.
- Roland Glowinski, On alternating direction methods of multipliers: a historical perspective, Modeling, simulation and optimization for science and technology, Comput. Methods Appl. Sci., vol. 34, Springer, Dordrecht, 2014, pp. 59–82. MR 3330832, DOI 10.1007/978-94-017-9054-3_{4}
- R. Glowinski and A. Marrocco, Sur l’approximation, par éléments finis d’ordre un, et la résolution, par pénalisation-dualité, d’une classe de problèmes de Dirichlet non linéaires, Rev. Française Automat. Informat. Recherche Opérationnelle Sér. Rouge Anal. Numér. 9 (1975), no. R-2, 41–76 (French, with English summary). MR 388811, DOI 10.1051/m2an/197509R200411
- J. Han and D. Sun, Newton and quasi-Newton methods for normal maps with polyhedral sets, J. Optim. Theory Appl. 94 (1997), no. 3, 659–676. MR 1470887, DOI 10.1023/A:1022653001160
- Magnus R. Hestenes, Multiplier and gradient methods, J. Optim. Theory Appl. 4 (1969), 303–320. MR 271809, DOI 10.1007/BF00927673
- N. J. Higham, Matrix nearness problems and applications, Applications of matrix theory (Bradford, 1988) Inst. Math. Appl. Conf. Ser. New Ser., vol. 22, Oxford Univ. Press, New York, 1989, pp. 1–27. MR 1041063, DOI 10.1093/imamat/22.1.1
- J.-B. Hiriart-Urruty, J.-J. Strodiot, and V.H. Nguyen, Generalized Hessian matrix and second-order optimality conditions for problems with ${C}^{1,1}$ data, Applied Mathematics and Optimization 11 (1984), 43–56.
- Alexey F. Izmailov, Alexey S. Kurennoy, and Mikhail V. Solodov, A note on upper Lipschitz stability, error bounds, and critical multipliers for Lipschitz-continuous KKT systems, Math. Program. 142 (2013), no. 1-2, Ser. A, 591–604. MR 3127087, DOI 10.1007/s10107-012-0586-z
- Alexander Y. Kruger, About intrinsic transversality of pairs of sets, Set-Valued Var. Anal. 26 (2018), no. 1, 111–142. MR 3769339, DOI 10.1007/s11228-017-0446-3
- Alexander Y. Kruger, D. Russell Luke, and Nguyen H. Thao, Set regularities and feasibility problems, Math. Program. 168 (2018), no. 1-2, Ser. B, 279–311. MR 3767748, DOI 10.1007/s10107-016-1039-x
- Alexander Y. Kruger and Nguyen H. Thao, Regularity of collections of sets and convergence of inexact alternating projections, J. Convex Anal. 23 (2016), no. 3, 823–847. MR 3533177
- J. B. Kruskal and M. Wish, Multidimensional Scaling, no. 07-011, Quantitative Applications in the Social Sciences, Sage Publications, 1978.
- J. Leskovec and A. Krevl, SNAP datasets: Stanford large network dataset collection, 2014, http://snap.stanford.edu/data.
- Xudong Li, Defeng Sun, and Kim-Chuan Toh, An asymptotically superlinearly convergent semismooth Newton augmented Lagrangian method for linear programming, SIAM J. Optim. 30 (2020), no. 3, 2410–2440. MR 4146384, DOI 10.1137/19M1251795
- Xudong Li, Defeng Sun, and Kim-Chuan Toh, On the efficient computation of a generalized Jacobian of the projector over the Birkhoff polytope, Math. Program. 179 (2020), no. 1-2, Ser. A, 419–446. MR 4050144, DOI 10.1007/s10107-018-1342-9
- Meixia Lin, Defeng Sun, and Kim-Chuan Toh, An augmented Lagrangian method with constraint generation for shape-constrained convex regression problems, Math. Program. Comput. 14 (2022), no. 2, 223–270. MR 4421904, DOI 10.1007/s12532-021-00210-0
- O. L. Mangasarian, Normal solutions of linear programs, Math. Programming Stud. 22 (1984), 206–216. Mathematical programming at Oberwolfach, II (Oberwolfach, 1983). MR 774243, DOI 10.1007/bfb0121017
- Fanwen Meng, Defeng Sun, and Gongyun Zhao, Semismoothness of solutions to generalized equations and the Moreau-Yosida regularization, Math. Program. 104 (2005), no. 2-3, Ser. B, 561–581. MR 2179251, DOI 10.1007/s10107-005-0629-9
- R. H. Pearce, Towards a general formulation of lazy constraints, Ph.D. Thesis, The University of Queensland, 2019.
- M. J. D. Powell, A method for nonlinear constraints in minimization problems, Optimization (Sympos., Univ. Keele, Keele, 1968) Academic Press, London-New York, 1969, pp. 283–298. MR 272403
- S. M. Robinson, Some continuity properties of polyhedral multifunctions, Mathematical Programming at Oberwolfach, Springer Berlin Heidelberg, Berlin, Heidelberg, 1981, pp. 206–214.
- R. Tyrrell Rockafellar, Convex analysis, Princeton Mathematical Series, No. 28, Princeton University Press, Princeton, NJ, 1970. MR 274683, DOI 10.1515/9781400873173
- R. T. Rockafellar, Augmented Lagrangians and applications of the proximal point algorithm in convex programming, Math. Oper. Res. 1 (1976), no. 2, 97–116. MR 418919, DOI 10.1287/moor.1.2.97
- R. Tyrrell Rockafellar, Monotone operators and the proximal point algorithm, SIAM J. Control Optim. 14 (1976), no. 5, 877–898. MR 410483, DOI 10.1137/0314056
- R. Tyrrell Rockafellar and Roger J.-B. Wets, Variational analysis, Grundlehren der mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 317, Springer-Verlag, Berlin, 1998. MR 1491362, DOI 10.1007/978-3-642-02431-3
- V. Roth, J. Laub, J. M. Buhmann, and K.-R. Müller, Going metric: denoising pairwise data, Proceedings of the 15th International Conference on Neural Information Processing Systems (Cambridge, MA, USA), NIPS’02, MIT Press, 2002, pp. 841–848.
- V. Roth, J. Laub, M. Kawanabe, and J. M. Buhmann, Optimal cluster preserving embedding of non-metric proximity data, IEEE Trans. Pattern Anal. Mach. Intell. 25 (2002), 1540–1551.
- C. Ruggles, N. Veldt, and D. F. Gleich, A parallel projection method for metric constrained optimization, 2020 Proceedings of the SIAM Workshop on Combinatorial Scientific Computing (CSC), 2020, pp. 43–53.
- M. V. Solodov and B. F. Svaiter, A hybrid approximate extragradient-proximal point algorithm using the enlargement of a maximal monotone operator, Set-Valued Anal. 7 (1999), no. 4, 323–345. MR 1756912, DOI 10.1023/A:1008777829180
- M. V. Solodov and B. F. Svaiter, An inexact hybrid generalized proximal point algorithm and some new results on the theory of Bregman functions, Math. Oper. Res. 25 (2000), no. 2, 214–230. MR 1853949, DOI 10.1287/moor.25.2.214.12222
- Rishi Sonthalia and Anna C. Gilbert, Project and forget: solving large-scale metric constrained problems, J. Mach. Learn. Res. 23 (2022), Paper No. [326], 54. MR 4577765
- Nate Veldt, David F. Gleich, Anthony Wirth, and James Saunderson, Metric-constrained optimization for graph clustering algorithms, SIAM J. Math. Data Sci. 1 (2019), no. 2, 333–355. MR 3958772, DOI 10.1137/18M1217152
- S. N. Vitaladevuni and R. Basri, Co-clustering of image segments using convex optimization applied to em neuronal reconstruction, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2010, pp. 2203–2210.
- E. P. Xing, A.Y. Ng, M.I. Jordan, and S. Russell, Distance metric learning, with application to clustering with side-information, Proceedings of the 15th International Conference on Neural Information Processing Systems, 2002, pp. 521–528.
Bibliographic Information
- Peipei Tang
- Affiliation: City Brain Institute, School of Computer and Computing Science, Hangzhou City University, Hangzhou 310015, People’s Republic ofChina
- ORCID: 0000-0003-4137-8029
- Email: tangpp@hzcu.edu.cn
- Bo Jiang
- Affiliation: School of Computer Science, Zhejiang University, No. 38, Zheda Road, Hangzhou 310027, People’s Republic of China
- ORCID: 0009-0009-8002-7302
- Email: 22021105@zju.edu.cn
- Chengjing Wang
- Affiliation: School of Mathematics, Southwest Jiaotong University, No. 999, Xian Road, West Park, High-tech Zone, Chengdu 611756, People’s Republic of China
- MR Author ID: 787623
- Email: renascencewang@hotmail.com
- Received by editor(s): May 1, 2023
- Received by editor(s) in revised form: July 29, 2024
- Published electronically: November 8, 2024
- Additional Notes: The work of the first author was supported by the National Science and Technology Major Project (2022ZD0119103), and by the Zhejiang Provincial Natural Science Foundation of China under Grant LY24F020013, LHZSD24F020001 and the Scientific Research Foundation of Zhejiang University City College under Grant X-202112. The work of the third author was supported by the National Natural Science Foundation of China under Grant U21A20169, and by the Zhejiang Provincial Natural Science Foundation of China under Grant LTGY23H240002. The authors were supported by advanced computing resources provided by the Supercomputing Center of Hangzhou City University.
The third author is the corresponding author. - © Copyright 2024 American Mathematical Society
- Journal: Math. Comp. 94 (2025), 2495-2532
- MSC (2020): Primary 90C25, 65K05, 90C06, 49M27, 90C20
- DOI: https://doi.org/10.1090/mcom/4026