Remote Access Proceedings of the American Mathematical Society
Green Open Access

Proceedings of the American Mathematical Society

ISSN 1088-6826(online) ISSN 0002-9939(print)

 
 

 

Equivalence of approximation by convolutional neural networks and fully-connected networks


Authors: Philipp Petersen and Felix Voigtlaender
Journal: Proc. Amer. Math. Soc. 148 (2020), 1567-1581
MSC (2010): Primary 41A25; Secondary 44A35, 41A46
DOI: https://doi.org/10.1090/proc/14789
Published electronically: December 6, 2019
Full-text PDF

Abstract | References | Similar Articles | Additional Information

Abstract: Convolutional neural networks are the most widely used type of neural networks in applications. In mathematical analysis, however, mostly fully-connected networks are studied. In this paper, we establish a connection between both network architectures. Using this connection, we show that all upper and lower bounds concerning approximation rates of fully-connected neural networks for functions $ f \in \mathcal {C}$--for an arbitrary function class $ \mathcal {C}$--translate to essentially the same bounds concerning approximation rates of convolutional neural networks for functions $ f \in \mathcal {C}^{\mathrm {equi}}$, with the class $ \mathcal {C}^{\mathrm {equi}}$ consisting of all translation equivariant functions whose first coordinate belongs to $ \mathcal {C}$. All presented results consider exclusively the case of convolutional neural networks without any pooling operation and with circular convolutions, i.e., not based on zero-padding.


References [Enhancements On Off] (What's this?)


Similar Articles

Retrieve articles in Proceedings of the American Mathematical Society with MSC (2010): 41A25, 44A35, 41A46

Retrieve articles in all journals with MSC (2010): 41A25, 44A35, 41A46


Additional Information

Philipp Petersen
Affiliation: Mathematical Institute, University of Oxford, Andrew Wiles Building, Oxford, United Kingdom
Email: Philipp.Petersen@maths.ox.ac.uk

Felix Voigtlaender
Affiliation: Lehrstuhl Wissenschaftliches Rechnen, Katholische Universität Eichstätt–Ingolstadt, Ostenstraße 26, 85072 Eichstätt, Germany
Email: felix@voigtlaender.xyz

DOI: https://doi.org/10.1090/proc/14789
Keywords: Neural networks, convolutional neural networks, function approximation, rate of convergence
Received by editor(s): September 4, 2018
Received by editor(s) in revised form: March 5, 2019, and August 5, 2019
Published electronically: December 6, 2019
Additional Notes: The first author was supported by a DFG research fellowship.
Both authors contributed equally to this work
Communicated by: Yuan Xu
Article copyright: © Copyright 2019 American Mathematical Society