A priori generalization error analysis of two-layer neural networks for solving high dimensional Schrödinger eigenvalue problems

Lu, Jianfeng; Lu, Yulong

doi:10.1090/cams/5

A priori generalization error analysis of two-layer neural networks for solving high dimensional Schrödinger eigenvalue problems
HTML articles powered by AMS MathViewer

by Jianfeng Lu and Yulong Lu

Comm. Amer. Math. Soc. 2 (2022), 1-21

DOI: https://doi.org/10.1090/cams/5

Published electronically: January 31, 2022

HTML | PDF

Abstract:

This paper analyzes the generalization error of two-layer neural networks for computing the ground state of the Schrödinger operator on a $d$-dimensional hypercube with Neumann boundary condition. We prove that the convergence rate of the generalization error is independent of dimension $d$, under the a priori assumption that the ground state lies in a spectral Barron space. We verify such assumption by proving a new regularity estimate for the ground state in the spectral Barron space. The latter is achieved by a fixed point argument based on the Krein-Rutman theorem.

References

Herbert Amann, Fixed point equations and nonlinear eigenvalue problems in ordered Banach spaces, SIAM Rev. 18 (1976), no. 4, 620–709. MR 415432, DOI 10.1137/1018114
Ben Andrews and Julie Clutterbuck, Proof of the fundamental gap conjecture, J. Amer. Math. Soc. 24 (2011), no. 3, 899–916. MR 2784332, DOI 10.1090/S0894-0347-2011-00699-1
Ben Andrews, Julie Clutterbuck, and Daniel Hauer, The fundamental gap for a one-dimensional Schrödinger operator with Robin boundary conditions, Proc. Amer. Math. Soc. 149 (2021), no. 4, 1481–1493. MR 4242306, DOI 10.1090/proc/15140
Francis Bach, Breaking the curse of dimensionality with convex neutral networks, J. Mach. Learn. Res. 18 (2017), Paper No. 19, 53. MR 3634886
Andrew R. Barron, Universal approximation bounds for superpositions of a sigmoidal function, IEEE Trans. Inform. Theory 39 (1993), no. 3, 930–945. MR 1237720, DOI 10.1109/18.256500
Richard F. Bass and Pei Hsu, Some potential theory for reflecting Brownian motion in Hölder and Lipschitz domains, Ann. Probab. 19 (1991), no. 2, 486–508. MR 1106272
Cai, Zi and Jinguo Liu, 2018. Approximating quantum many-body wave functions using artificial neural networks. Phys. Rev. B 97, 035116.
Caragea, Andrei, Philipp Petersen, and Felix Voigtlaender, 2020. Neural network approximation and estimation of classifiers with classification boundary in a barron class. arXiv:2011.09363
Giuseppe Carleo and Matthias Troyer, Solving the quantum many-body problem with artificial neural networks, Science 355 (2017), no. 6325, 602–606. MR 3642415, DOI 10.1126/science.aag2302
Chen, Fan, Jianguo Huang, Chunmei Wang, and Haizhao Yang, 2020. Friedrichs learning: weak solutions of partial differential equations via deep learning. arXiv:2012.08023.
Chen, Zhiang, Jianfeng Lu, and Yulong Lu, 2021. On the representation of solutions to elliptic PDEs in Barron spaces. Adv. Neural Info. Processing Syst. 34.
Choo, Kenney, Antonio Mezzacapo, and Giuseppe Carleo. Fermionic neural-network states for ab-initio electronic structure. Nat. Commun. 11, no. 1, 1–7.
R. M. Dudley, The sizes of compact subsets of Hilbert space and continuity of Gaussian processes, J. Functional Analysis 1 (1967), 290–330. MR 0220340, DOI 10.1016/0022-1236(67)90017-1
E. Weinan, Chao Ma, Stephan Wojtowytsch, and Lei Wu, 2020. Towards a mathematical understanding of neural network-based machine learning: what we know and what we don’t. arXiv:2009.10713.
E. Weinan, Chao Ma, and Lei Wu, 2019.. Barron spaces and the compositional function spaces for neural network models. arXiv:1906.08039.
E. Weinan and Stephan Wojtowytsch, 2020. Representation formulas and pointwise properties for barron functions. arXiv:2006.05982.
E. Weinan and Stephan Wojtowytsch, 2020. Some observations on partial differential equations in Barron and multi-layer spaces, arXiv:2012.01484.
Weinan E and Bing Yu, The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems, Commun. Math. Stat. 6 (2018), no. 1, 1–12. MR 3767958, DOI 10.1007/s40304-018-0127-z
Gao, Xun and Lu-Ming. Duan, 2017. Efficient representation of quantum many-body states with deep neural networks, Nat. Commun. 8, 662, 2017.
James Glimm and Arthur Jaffe, Quantum physics, 2nd ed., Springer-Verlag, New York, 1987. A functional integral point of view. MR 887102, DOI 10.1007/978-1-4612-4728-9
Pierre Grisvard, Elliptic problems in nonsmooth domains, Classics in Applied Mathematics, vol. 69, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2011. Reprint of the 1985 original [ MR0775683]; With a foreword by Susanne C. Brenner. MR 3396210, DOI 10.1137/1.9781611972030.ch1
Yiqi Gu, Haizhao Yang, and Chao Zhou, SelectNet: self-paced learning for high-dimensional partial differential equations, J. Comput. Phys. 441 (2021), Paper No. 110444, 18. MR 4265654, DOI 10.1016/j.jcp.2021.110444
Jiequn Han, Arnulf Jentzen, and Weinan E, Solving high-dimensional partial differential equations using deep learning, Proc. Natl. Acad. Sci. USA 115 (2018), no. 34, 8505–8510. MR 3847747, DOI 10.1073/pnas.1718942115
Jiequn Han, Jianfeng Lu, and Mo Zhou, Solving high-dimensional eigenvalue problems using deep neural networks: a diffusion Monte Carlo like approach, J. Comput. Phys. 423 (2020), 109792, 13. MR 4156932, DOI 10.1016/j.jcp.2020.109792
Jiequn Han, Linfeng Zhang, and Weinan E, Solving many-electron Schrödinger equation using deep neural networks, J. Comput. Phys. 399 (2019), 108929, 8. MR 4013818, DOI 10.1016/j.jcp.2019.108929
Hermann, Jan, Zeno Schätzle, and Frank Noé, 2020. Deep-neural-network solution of the electronic Schrödinger equation. Nature Chemistry 12, no. 10, 891–897.
Hong, Qingguo, Jonathan W Siegel, and Jinchao Xu, 2021. A priori analysis of stable neural network solutions to numerical PDEs. arXiv:2104.02903.
Kerner, Joachim, 2021. A lower bound on the spectral gap of one-dimensional Schrödinger operators. arXiv:2102.03816.
Yuehaw Khoo, Jianfeng Lu, and Lexing Ying, Solving for high-dimensional committor functions using artificial neural networks, Res. Math. Sci. 6 (2019), no. 1, Paper No. 1, 13. MR 3887217, DOI 10.1007/s40687-018-0160-2
Jason M. Klusowski and Andrew R. Barron, Approximation by combinations of ReLU and squared ReLU ridge functions with $\ell ^1$ and $\ell ^0$ controls, IEEE Trans. Inform. Theory 64 (2018), no. 12, 7649–7656. MR 3883687, DOI 10.1109/tit.2018.2874447
M. G. Kreĭn and M. A. Rutman, Linear operators leaving invariant a cone in a Banach space, Amer. Math. Soc. Translation 1950 (1950), no. 26, 128. MR 0038008
LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton, 2015. Deep learning. Nature, 521, no. 7553, 436–444.
Lu, Yulong, Jianfeng Lu, and Min Wang, 2021. A priori generalization analysis of the deep ritz method for solving high dimensional elliptic partial differential equations. In Conference on Learning Theory, PMLR, pp. 3196–3241.
Luo, Tao and Haizhao Yang, 2020. Two-layer neural networks for partial differential equations: Optimization and generalization theory. arXiv:2006.15733.
Mishra, Siddhartha and Roberto Molinaro, Estimates on the generalization error of physics informed neural networks (PINNs) for approximating PDEs, 2020. arXiv:2006.16144.
Pfau, David, James S Spencer, Alexander GDG Matthews, and W Matthew C Foulkes, 2020. Ab initio solution of the many-electron schrödinger equation with deep neural networks. Phys. Rev. Res. 2, no. 3, 033429.
M. Raissi, P. Perdikaris, and G. E. Karniadakis, Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys. 378 (2019), 686–707. MR 3881695, DOI 10.1016/j.jcp.2018.10.045
schmidhuber, Jürgen, 2015. Deep learning in neural networks: an overview. Neural Netw. 61 85–117.
Ohad Shamir and Shai Shalev-Shwartz, Matrix completion with the trace norm: learning, bounding, and transducing, J. Mach. Learn. Res. 15 (2014), 3401–3423. MR 3277164
Yeonjong Shin, Jérôme Darbon, and George Em Karniadakis, On the convergence of physics informed neural networks for linear second-order elliptic and parabolic type PDEs, Commun. Comput. Phys. 28 (2020), no. 5, 2042–2074. MR 4188529, DOI 10.4208/cicp.oa-2020-0193
Shin, Yeonjong, Zhongqiang Zhang, and George Em Karniadakis. Error estimates of residual minimization using neural networks for linear PDEs, arXiv:2010.08019, 2020.
Siegel, Jonathan W and Jinchao Xu, 2020. Approximation rates for neural networks with general activation functions. Neural Netw.
Jonathan W. Siegel and Jinchao Xu, High-order approximation rates for shallow neural networks with cosine and $\textrm {ReLU}^k$ activation functions, Appl. Comput. Harmon. Anal. 58 (2022), 1–26. MR 4357282, DOI 10.1016/j.acha.2021.12.005
I. M. Singer, Bun Wong, Shing-Tung Yau, and Stephen S.-T. Yau, An estimate of the gap of the first two eigenvalues in the Schrödinger operator, Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4) 12 (1985), no. 2, 319–333. MR 829055
Justin Sirignano and Konstantinos Spiliopoulos, DGM: a deep learning algorithm for solving partial differential equations, J. Comput. Phys. 375 (2018), 1339–1364. MR 3874585, DOI 10.1016/j.jcp.2018.08.029
Wolf, Michael M. 2020. Mathematical Foundations of Supervised Learning, https://www-m5.ma.tum.de/foswiki/pub/M5/Allgemeines/MA4801_2020S/ML_notes_main.pdf. Last visited on 2020/12/5.
Jinchao Xu, Finite neuron method and convergence analysis, Commun. Comput. Phys. 28 (2020), no. 5, 1707–1745. MR 4188519, DOI 10.4208/cicp.oa-2020-0191
Yaohua Zang, Gang Bao, Xiaojing Ye, and Haomin Zhou, Weak adversarial networks for high-dimensional partial differential equations, J. Comput. Phys. 411 (2020), 109409, 14. MR 4079373, DOI 10.1016/j.jcp.2020.109409