Remote Access St. Petersburg Mathematical Journal

St. Petersburg Mathematical Journal

ISSN 1547-7371(online) ISSN 1061-0022(print)

Request Permissions   Purchase Content 
 
 

 

Approximation by sums of ridge functions with fixed directions


Author: V. E. Ismailov
Translated by: S. Kislyakov
Original publication: Algebra i Analiz, tom 28 (2016), nomer 6.
Journal: St. Petersburg Math. J. 28 (2017), 741-772
MSC (2010): Primary 41A30
DOI: https://doi.org/10.1090/spmj/1471
Published electronically: October 2, 2017
Full-text PDF

Abstract | References | Similar Articles | Additional Information

Abstract: The paper contains a survey of some results about approximation of functions of several variables by sums of ridge functions with fixed directions. Also, some new theorems are proved, both for uniform approximation and for approximation in $ L_{2}$. These theorems generalize some results by the author known previously. The paper is finished by the study of the role of ridge functions in a problem of approximation by neural networks.


References [Enhancements On Off] (What's this?)

  • 1. V. I. Arnol'd, On the representation of continuous functions of three variables by superpositions of continuous functions of two variables, Mat. Sb. 48 (1959), 3-74. (Russian) MR 0121453
  • 2. M.-B. A. Babaev, The approximation of polynomials of two variables by functions of the form $ Q(x+\psi )y$, Dokl. Akad. Nauk SSSR 193 (1970), no. 5, 967-969; English transl., Soviet. Math. Dokl. 11 (1970), 1034-1036. MR 0280915
  • 3. -, Sharp estimates for the approximation of functions of several variables by sums of functions of a lesser number of variables, Mat. Zametki 12 (1972), no. 1, 105-114; English transl., Math. Notes 12 (1972), 495-500. MR 0326243
  • 4. D. Braess and A. Pinkus, Interpolation by ridge functions, J. Approx. Theory 73 (1993), no. 2, 218-236. MR 1216487
  • 5. R. C. Buck, On approximation theory and functional equations, J. Approx. Theory 5 (1972), 228-237. MR 0377363
  • 6. M. D. Buhmann and A. Pinkus, Identifying linear combinations of ridge functions, Adv. Appl. Math. 22 (1999), no. 1, 103-118. MR 1657745
  • 7. E. J. Candes, Ridgelets: estimating with ridge functions, Ann. Statist. 31 (2003), no. 5, 1561-1599. MR 2012826
  • 8. -, Ridgelets: theory and applications, Ph.D. Thesis, Depart. Statist., Stanford Univ., Stanford, 1998.
  • 9. T. Chen and H. Chen, Approximation of continuous functionals by neural networks with application to dynamic systems, IEEE Trans. Neural Networks 4 (1993), 910-918.
  • 10. E. W. Cheney and W. A. Light, Approximation theory in tensor product spaces, Lecture Notes in Math., vol. 1169, Springer-Verlag, Berlin, 1985. MR 817984
  • 11. C. K. Chui and X. Li, Approximation by ridge functions and neural networks with one hidden layer, J. Approx. Theory 70 (1992), no. 2, 131-141. MR 1172015
  • 12. R. C. Cowsik, A. Klopotowski, and M. G. Nadkarni, When is $ f(x,y)=u(x)+v(y)$?, Proc. Indian Acad. Sci. Math. Sci. 109 (1999), no. 1, 57-64. MR 1687171
  • 13. G. Cybenko, Approximation by superpositions of a sigmoidal function, Math. Control Signals Systems 2 (1989), no. 4, 303-314. MR 1015670
  • 14. S. Demko, A superposition theorem for bounded continuous functions, Proc. Amer. Math. Soc. 66 (1977), no. 1, 75-78. MR 0457651
  • 15. R. A. DeVore, K. I. Oskolkov, and P. P. Petrushev, Approximation by feedforward neural networks, Ann. Numer. Math. 4 (1997), no. 1, 261-287. MR 1422683
  • 16. P. Diaconis and M. Shahshahani, On nonlinear functions of linear combinations, SIAM J. Sci. Statist. Comput. 5 (1984), no. 1, 175-191. MR 731890
  • 17. S. P. Diliberto and E. G. Straus, On the approximation of a function of several variables by the sum of functions of fewer variables, Pacific J. Math. 1 (1951), 195-210. MR 0043882
  • 18. D. L. Donoho and I. M. Johnstone, Projection-based approximation and a duality method with kernel methods, Ann. Statist. 17 (1989), no. 1, 58-106. MR 981438
  • 19. N. Dyn, W. A. Light, and E. W. Cheney, Interpolation by piecewise-linear radial basis functions, J. Approx. Theory. 59 (1989), no. 2, 202-223. MR 1022117
  • 20. B. L. Fridman, An improvement in the smoothness of the functions in A. N. Kolmogorov's theorem on superposition, Dokl. Akad. Nauk SSSR 177 (1967), no. 5, 1019-1022; English transl., Soviet Math. Dokl. 8 (1967), no. 6, 1550-1553. MR 0225066
  • 21. J. H. Friedman and J. W. Tukey, A projection pursuit algorithm for exploratory data analysis, IEEE Trans. Comput. 23 (1974), 881-890.
  • 22. J. H. Friedman and W. Stuetzle, Projection pursuit regression, J. Amer. Statist. Assoc. 76 (1981), no. 376, 817-823. MR 650892
  • 23. A. L. Garkavi, V. A. Medvedev, and S. Ya. Khavinson, On the existence of a best uniform approximation of a function in two variables by sums $ \varphi (x)+\psi (y)$, Sibirsk. Mat. Zh. 36 (1995), no. 4, 819-827; English transl., Sib. Math. J. 36 (1995), no. 4, 707-713. MR 1367249
  • 24. M. von Golitschek and W. A. Light, Approximation by solutions of the planar wave equation, SIAM J. Numer. Anal. 29 (1992), no. 3, 816-830. MR 1163358
  • 25. M. Golomb, Approximation by functions of fewer variables. On numerical approximation, Proc. Symp. (Madison, 1958), Univ. Wisconsin Press, Madison, 1959, pp. 275-327. MR 0102168
  • 26. G. Gripenberg, Approximation by neural networks with a bounded number of nodes at each level, J. Approx. Theory 122 (2003), no. 2, 260-266. MR 1988304
  • 27. K. Hornik, Approximation capabilities of multilayer feedforward networks, Neural Networks 4 (1991), 251-257.
  • 28. P. J. Huber, Projection pursuit, Ann. Statist. 13 (1985), no. 2, 435-475. MR 790553
  • 29. V. E. Ismailov, Approximation by neural networks with weights varying on a finite set of directions, J. Anal. Appl. 389 (2012), no. 1, 72-83. MR 2876482
  • 30. -, A note on the representation of continuous functions by linear superpositions, Expo. Math. 30 (2012), no. 1, 96-101. MR 2899658
  • 31. -, On the theorem of M. Golomb, Proc. Indian Acad. Sci. Math. Sci. 119 (2009), no. 1, 45-52. MR 2508488
  • 32. -, On the representation by linear superpositions, J. Approx. Theory 151 (2008), no. 2, 113-125. MR 2407861
  • 33. -, A note on the best $ L_{2}$ approximation by ridge functions, Appl. Math. E-Notes 7 (2007), 71-76 (electronic). MR 2295689
  • 34. -, Characterization of an extremal sum of ridge functions, J. Comput. Appl. Math. 205 (2007), no. 1, 105-115. MR 2324828
  • 35. -, On error formulas for approximation by sums of univariate functions, Int. J. Math. Math. Sci. 2006, Article ID 65620. MR 2219166 (2007c:41026)
  • 36. -, On methods for computing the exact value of the best approximation by sums of functions of one variable, Sibirsk. Mat. Zh. 47 (2006), no. 5, 1076-1082; English transl., Sib. Math. J. 47 (2006), no. 5, 883-888. MR 2266517
  • 37. Y. Ito, Nonlinearity creates linear independence, Adv. Comput. Math. 5 (1996), no. 2-3, 189-203. MR 1399380
  • 38. -, Approximation of functions on a compact set by finite sums of a sigmoid function without scaling, Neural Networks 4 (1991), no. 6, 817-826.
  • 39. -, Approximation of continuous functions on $ \mathbb{R}^{d}$ by linear combinations of shifted rotations of a sigmoid function with and without scaling, Neural Networks 5 (1992), 105-115.
  • 40. F. John, Plane waves and spherical means applied to partial differential equations, Intersci. Publ., New York, 1955. MR 0075429
  • 41. I. Kazantsev, Tomographic reconstruction from arbitrary directions using ridge functions, Inverse Problems 14 (1998), no. 3, 635-645. MR 1630007
  • 42. I. Kazantsev and I. Lemahieu, Reconstruction of elongated structures using ridge functions and natural pixels, Inverse Problems 16 (2000), no. 6, 505-517.
  • 43. S. Ya. Khavinson, Best approximation by linear superpositions (approximate nomography), Transl. Math. Monogr., vol. 159. Amer. Math. Soc., Providence, RI, 1997. MR 1421322
  • 44. -, A Chebyshev theorem for the approximation of a function of two variables by sums $ \varphi (x)+\psi (y)$, Izv. Akad. Nauk SSSR Ser. Mat. 33 (1969), no. 3, 650-666; English transl., Math. USSR-Izv. 3 (1969), no. 3, 617-632. MR 0262746
  • 45. A. Klopotowski, M. G. Nadkarni, and K. P. S. Bhaskara Rao, When is $ f(x_{1},x_{2},\dots ,x_{n})=u_{1}(x_{1})+u_{2}(x_{2}) +\dots +u_{n}(x_{n})$?, Proc. Indian Acad. Sci. Math. Sci. 113 (2003), no. 1, 77-86. MR 1971557
  • 46. A. N. Kolmogorov, On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition, Dokl. Akad. Nauk SSSR 114 (1957), no. 5, 953-956. (Russian) MR 0111809
  • 47. M. Leshno, V. Ya. Lin, A. Pinkus, and S. Schocken, Multilayer feedforward networks with a non-polynomial activation function can approximate any function, Neural Networks 6 (1993), 861-867.
  • 48. X. Li, Interpolation by ridge polynomials and its application in neural networks, J. Comput. Appl. Math. 144 (2002), no. 1-2, 197-209. MR 1909992
  • 49. W. A. Light and E. W. Cheney, On the approximation of a bivariate function by the sum of univariate functions, J. Approx. Theory 29 (1980), no. 4, 305-323. MR 598725
  • 50. W. A. Light, Ridge functions, sigmoidal functions and neural networks, Approximation Theory VII (Austin, TX, 1992), Acad. Press, Boston, MA, 1993, pp. 163-206. MR 1212573
  • 51. V. Ya. Lin and A. Pinkus, Fundamentality of ridge functions, J. Approx. Theory 75 (1993), no. 3, 295-311. MR 1250542
  • 52. B. F. Logan and L. A. Shepp, Optimal reconstruction of a function from its projections, Duke Math. J. 42 (1975), no. 4, 645-659. MR 0397240
  • 53. G. G. Lorentz, Metric entropy, widths, and superpositions of functions, Amer. Math. Monthly 69 (1962), 469-485. MR 0141926
  • 54. V. Maiorov and A. Pinkus, Lower bounds for approximation by MLP neural networks, Neurocomputing 25 (1999), 81-91.
  • 55. V. Maiorov, Approximation by neural networks and learning theory, J. Complexity 22 (2006), no. 1, 102-117. MR 2198503
  • 56. -, On best approximation by ridge functions, J. Approx. Theory 99 (1999), no. 1, 68-94. MR 1696577
  • 57. R. B. Marr, On the reconstruction of a function on a circular domain from a sampling of its line integrals, J. Math. Anal. Appl. 45 (1974), 357-374. MR 0336156
  • 58. D. E. Marshall and A. G. O'Farrell, Approximation by a sum of two algebras. The lightning bolt principle, J. Funct. Anal. 52 (1983), no. 3, 353-368. MR 712586
  • 59. -, Uniform approximation by real functions, Fund. Math. 104 (1979), no. 3, 203-211. MR 559174
  • 60. F. Natterer, The mathematics of computerized tomography, J. Wiley, Ltd., Chichester, 1986. MR 856916
  • 61. K. I. Oskolkov, Ridge approximation, Fourier-Chebyshev analysis and optimal quadrature formulas, Tr. Mat. Inst. Steklova 219 (1997), 269-285; English transl., Proc. Steklov Inst. Math. 219 (1997), 265-280. MR 1642280
  • 62. P. A. Ostrand, Dimension of metric spaces and Hilbert's problem $ 13$, Bull. Amer. Math. Soc. 71 (1965), 619-622. MR 0177391
  • 63. P. P. Petrushev, Approximation by ridge functions and neural networks, SIAM J. Math. Anal. 30 (1998), no. 1, 155-189. MR 1646689
  • 64. A. Pinkus, Approximating by ridge functions, Surface Fitting and Multiresolution Methods, Vanderbilt Univ. Press, Nashville, 1997, pp. 279-292. MR 1660030
  • 65. -, Approximation theory of the MLP model in neural networks, Acta Numer. 8 (1999), 143-195. MR 1819645
  • 66. W. Rudin, Functional analysis, McGraw-Hill Ser. Higher Math., McGraw-Hill Book Co., New York, 1973. MR 0365062
  • 67. M. Sanguineti, Universal approximation by ridge computational models and neural networks: a survey, Open Appl. Math. J. 2 (2008), no. 1, 31-58. MR 2399691
  • 68. L. Schwartz, Théorie générale des fonctions moyenne-périodiques, Ann. of Math.(2) 48 (1947), 857-928. MR 0023948
  • 69. Y. Shin and J. Ghosh, Ridge polynomial networks, IEEE Trans. Neural Networks 6 (1995), 610-622.
  • 70. D. A. Sprecher, An improvement in the superposition theorem of Kolmogorov, J. Math. Anal. Appl. 38 (1972), 208-213. MR 0302838 (46:1981)
  • 71. J. P. Sproston and D. Strauss, Sums of subalgebras of $ \mathrm {C(X)}$, J. London Math. Soc.(2) 45 (1992), no. 2, 265-278. MR 1171554
  • 72. Y. Sternfeld, Dimension, superposition of functions and separation of points, in compact metric spaces, Israel J. Math. 50 (1985), no. 1-2, 13-53. MR 788068
  • 73. -, Uniform separation of points and measures and representation by sums of algebras, Israel J. Math. 55 (1986), no. 3, 350-362. MR 876401
  • 74. -, Uniformly separating families of functions, Israel J. Math. 29 (1978), no. 1, 61-91. MR 0487991
  • 75. M. Stinchcombe and H. White, Approximating and learning unknown mappings using multilayer feedforward networks with bounded weights, Proc. Internat. Joint Conf. Neural Networks, vol. 3, IEEE, New York, 1990, pp. 7-16.
  • 76. V. N. Temlyakov, On approximation by ridge functions, Preprint. Depart. Math., Univ. South Carolina, 1996.
  • 77. A. G. Vitushkin and G. M. Henkin, Linear superpositions of functions, Uspekhi Mat. Nauk 22 (1967), no. 1, 77-124; English transl., Russian Math. Surveys 22 (1967), no. 1, 77-125. MR 0237729
  • 78. B. A. Vostrečov and M. A. Kreines, Approximation of continuous functions by superpositions of plane waves, Dokl. Akad. Nauk SSSR 140 (1961), no. 6, 1237-1240; English transl., Soviet Math. Dokl. 2 (1961), no. 6, 1326-1329. MR 0131106
  • 79. W. Wu, G. Feng, and X. Li, Training multilayer perceptrons via minimization of sum of ridge functions, Adv. Comput. Math. 17 (2002), no. 4, 331-347. MR 1916983
  • 80. T. F. Xie and F. L. Cao, The ridge function representation of polynomials and an application to neural networks, Acta Math. Sin. (Engl. Ser.) 27 (2011), no. 11, 2169-2176. MR 2843194

Similar Articles

Retrieve articles in St. Petersburg Mathematical Journal with MSC (2010): 41A30

Retrieve articles in all journals with MSC (2010): 41A30


Additional Information

V. E. Ismailov
Affiliation: Institute of Mathematics and Mechanics, National Academy of Sciences of Azerbaijan
Email: vugaris@mail.ru

DOI: https://doi.org/10.1090/spmj/1471
Keywords: Ridge function, best approximation, superposition, cycle, path, neural network
Received by editor(s): February 8, 2015
Published electronically: October 2, 2017
Additional Notes: Supported by the Development of Science Foundation of the Republic of Azerbaijan (grant EIF-2013-9(15)-46/11/1)
Article copyright: © Copyright 2017 American Mathematical Society

American Mathematical Society