Deep backward schemes for high-dimensional nonlinear PDEs
HTML articles powered by AMS MathViewer
- by Côme Huré, Huyên Pham and Xavier Warin HTML | PDF
- Math. Comp. 89 (2020), 1547-1579 Request permission
Abstract:
We propose new machine learning schemes for solving high-dimensional nonlinear partial differential equations (PDEs). Relying on the classical backward stochastic differential equation (BSDE) representation of PDEs, our algorithms estimate simultaneously the solution and its gradient by deep neural networks. These approximations are performed at each time step from the minimization of loss functions defined recursively by backward induction. The methodology is extended to variational inequalities arising in optimal stopping problems. We analyze the convergence of the deep learning schemes and provide error estimates in terms of the universal approximation of neural networks. Numerical results show that our algorithms give very good results till dimension 50 (and certainly above), for both PDEs and variational inequalities problems. For the PDEs resolution, our results are very similar to those obtained by the recent method in [Commun. Math. Stat. 5 (2017), pp. 349–380] when the latter converges to the right solution or does not diverge. Numerical tests indicate that the proposed methods are not stuck in poor local minima as it can be the case with the algorithm designed in the work previously mentioned, and no divergence is experienced. The only limitation seems to be due to the inability of the considered deep neural networks to represent a solution with a too complex structure in high dimension.References
- Rainer Avikainen, On irregular functionals of SDEs and the Euler scheme, Finance Stoch. 13 (2009), no. 3, 381–401. MR 2519837, DOI 10.1007/s00780-009-0099-7
- Vlad Bally and Gilles Pagès, Error analysis of the optimal quantization algorithm for obstacle problems, Stochastic Process. Appl. 106 (2003), no. 1, 1–40. MR 1983041
- Christian Beck, Weinan E, and Arnulf Jentzen, Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations, J. Nonlinear Sci. 29 (2019), no. 4, 1563–1619. MR 3993178, DOI 10.1007/s00332-018-9525-3
- Sebastian Becker, Patrick Cheridito, and Arnulf Jentzen, Deep optimal stopping, J. Mach. Learn. Res. 20 (2019), Paper No. 74, 25. MR 3960928
- Bruno Bouchard and Jean-François Chassagneux, Discrete-time approximation for continuously and discretely reflected BSDEs, Stochastic Process. Appl. 118 (2008), no. 12, 2269–2293. MR 2474351, DOI 10.1016/j.spa.2007.12.007
- Bruno Bouchard and Nizar Touzi, Discrete-time approximation and Monte-Carlo simulation of backward stochastic differential equations, Stochastic Process. Appl. 111 (2004), no. 2, 175–206. MR 2056536, DOI 10.1016/j.spa.2004.01.001
- Bruno Bouchard and Xavier Warin, Monte-Carlo valuation of American options: facts and new algorithms to improve existing methods, Numerical methods in finance, Springer Proc. Math., vol. 12, Springer, Heidelberg, 2012, pp. 215–255. MR 3287773, DOI 10.1007/978-3-642-25746-9_{7}
- Quentin Chan-Wai-Nam, Joseph Mikael, and Xavier Warin, Machine learning for semi linear PDEs, J. Sci. Comput. 79 (2019), no. 3, 1667–1712. MR 3946472, DOI 10.1007/s10915-019-00908-3
- Weinan E, Jiequn Han, and Arnulf Jentzen, Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations, Commun. Math. Stat. 5 (2017), no. 4, 349–380. MR 3736669, DOI 10.1007/s40304-017-0117-6
- Weinan E, Martin Hutzenthaler, Arnulf Jentzen, and Thomas Kruse, On multilevel Picard numerical approximations for high-dimensional nonlinear parabolic partial differential equations and high-dimensional nonlinear backward stochastic differential equations, J. Sci. Comput. 79 (2019), no. 3, 1534–1571. MR 3946468, DOI 10.1007/s10915-018-00903-0
- N. El Karoui, C. Kapoudjian, E. Pardoux, S. Peng, and M. C. Quenez, Reflected solutions of backward SDE’s, and related obstacle problems for PDE’s, Ann. Probab. 25 (1997), no. 2, 702–737. MR 1434123, DOI 10.1214/aop/1024404416
- Emmanuel Gobet, Jean-Philippe Lemor, and Xavier Warin, A regression-based Monte Carlo method to solve backward stochastic differential equations, Ann. Appl. Probab. 15 (2005), no. 3, 2172–2202. MR 2152657, DOI 10.1214/105051605000000412
- J. Han, A. Jentzen, and W. E, Solving high-dimensional partial differential equations using deep learning, Proceedings of the National Academy of Sciences 115 (2018), no. 34, 8505–8510.
- J. Han and J. Long, Convergence of the deep bsde method for coupled fbsdes. arXiv:1811.01165 (2018).
- Pierre Henry-Labordère, Nadia Oudjane, Xiaolu Tan, Nizar Touzi, and Xavier Warin, Branching diffusion representation of semilinear PDEs and Monte Carlo approximation, Ann. Inst. Henri Poincaré Probab. Stat. 55 (2019), no. 1, 184–210. MR 3901645, DOI 10.1214/17-aihp880
- K. Hornik, M. Stinchcombe, and H. White, Multilayer feedforward networks are universal approximators, Neural Networks 2 (1989), no. 5, 359–366.
- Halbert White, Artificial neural networks, Blackwell Publishers, Oxford, 1992. Approximation and learning theory; With contributions by A. R. Gallant, K. Hornik, M. Stinchcombe and J. Wooldridge. MR 1203316
- M. Hutzenthaler, A. Jentzen, T. Kruse, and T.A. Nguyen, A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations. ArXiv:1901.10854 (2019).
- M. Hutzenthaler, A. Jentzen, T. Kruse, T.A. Nguyen, and P. von Wurstemberger, Overcoming the curse of dimensionality in the numerical approximation of semilinear parabolic partial differential equations, arXiv preprint arXiv:1807.01212 (2018).
- Patrick Jaillet, Damien Lamberton, and Bernard Lapeyre, Variational inequalities and the pricing of American options, Acta Appl. Math. 21 (1990), no. 3, 263–289. MR 1096582, DOI 10.1007/BF00047211
- Jean-Philippe Lemor, Emmanuel Gobet, and Xavier Warin, Rate of convergence of an empirical regression method for solving generalized backward stochastic differential equations, Bernoulli 12 (2006), no. 5, 889–916. MR 2265667, DOI 10.3150/bj/1161614951
- É. Pardoux and S. G. Peng, Adapted solution of a backward stochastic differential equation, Systems Control Lett. 14 (1990), no. 1, 55–61. MR 1037747, DOI 10.1016/0167-6911(90)90082-6
- Justin Sirignano and Konstantinos Spiliopoulos, DGM: a deep learning algorithm for solving partial differential equations, J. Comput. Phys. 375 (2018), 1339–1364. MR 3874585, DOI 10.1016/j.jcp.2018.08.029
- Xavier Warin, Nesting Monte Carlo for high-dimensional non-linear PDEs, Monte Carlo Methods Appl. 24 (2018), no. 4, 225–247. MR 3891748, DOI 10.1515/mcma-2018-2020
- Xavier Warin, Nesting Monte Carlo for high-dimensional non-linear PDEs, Monte Carlo Methods Appl. 24 (2018), no. 4, 225–247. MR 3891748, DOI 10.1515/mcma-2018-2020
- Jianfeng Zhang, A numerical scheme for BSDEs, Ann. Appl. Probab. 14 (2004), no. 1, 459–488. MR 2023027, DOI 10.1214/aoap/1075828058
Additional Information
- Côme Huré
- Affiliation: Laboratoire de Probabilites Statistique et Modelisation, Paris-Diderot University, 75013 Paris, France
- Email: hure@lpsm.paris
- Huyên Pham
- Affiliation: Laboratoire de Probabilites Statistique et Modelisation, Paris-Diderot University, 75013 Paris, France; and Center for Research in Economics and Statistics–ENSAE, Paris, France; and Finance et Marché de l’Energie, Paris, France
- Email: pham@lpsm.paris
- Xavier Warin
- Affiliation: EDF R&D, site de Clamart, 1 avenue du General De Gaulle, 92141 Clamart, France; and Finance et Marché de l’Energie, Paris, France
- Email: xavier.warin@edf.fr
- Received by editor(s): February 6, 2019
- Received by editor(s) in revised form: August 29, 2019, and November 8, 2019
- Published electronically: January 31, 2020
- Additional Notes: This work was supported by FiME, Laboratoire de Finance des Marchés de l’Energie, and the “Finance and Sustainable Development” EDF - CACIB Chair.
- © Copyright 2020 American Mathematical Society
- Journal: Math. Comp. 89 (2020), 1547-1579
- MSC (2010): Primary 60H35, 65C20, 65M12
- DOI: https://doi.org/10.1090/mcom/3514
- MathSciNet review: 4081911