Mathematical Neural Networks
Metadatos
Mostrar el registro completo del ítemAutor
García Cabello, JuliaEditorial
MDPI
Materia
Artificial neural network Universal approximation Activation function Injectivity
Fecha
2022-02-17Referencia bibliográfica
García Cabello, J. Mathematical Neural Networks. Axioms 2022, 11, 80. https://doi.org/10.3390/axioms11020080
Patrocinador
Spanish Government PID2019-103880RB-I00; Junta de Andalucía P12.SEJ.2463 SEJ340; "Maria de Maeztu" Excellence Unit IMAG - MCIN/AEI CEX2020001105-MResumen
ANNs succeed in several tasks for real scenarios due to their high learning abilities.
This paper focuses on theoretical aspects of ANNs to enhance the capacity of implementing those
modifications that make ANNs absorb the defining features of each scenario. This work may be also
encompassed within the trend devoted to providing mathematical explanations of ANN performance,
with special attention to activation functions. The base algorithm has been mathematically decoded
to analyse the required features of activation functions regarding their impact on the training process
and on the applicability of the Universal Approximation Theorem. Particularly, significant new
results to identify those activation functions which undergo some usual failings (gradient preserving)
are presented here. This is the first paper—to the best of the author’s knowledge—that stresses the
role of injectivity for activation functions, which has received scant attention in literature but has great
incidence on the ANN performance. In this line, a characterization of injective activation functions
has been provided related to monotonic functions which satisfy the classical contractive condition
as a particular case of Lipschitz functions. A summary table on these is also provided, targeted at
documenting how to select the best activation function for each situation.