Replacing pooling functions in Convolutional Neural Networks by linear combinations of increasing functions
Metadata
Show full item recordEditorial
Elsevier
Materia
Convolutional Neural Networks Pooling function Order statistic Generalized Sugeno integral
Date
2022-05-06Referencia bibliográfica
Iosu Rodriguez-Martinez... [et al.]. Replacing pooling functions in Convolutional Neural Networks by linear combinations of increasing functions, Neural Networks, Volume 152, 2022, Pages 380-393, ISSN 0893-6080, [https://doi.org/10.1016/j.neunet.2022.04.028]
Sponsorship
Tracasa Instrumental (iTRACASA), Spain; Gobierno de Navarra-Departamento de Universidad, Innovacion y Transformacion Digital, Spain; Spanish Ministry of Science, Spain PID2019-108392GB-I00; Andalusian Excellence project, Spain PID2019-108392GB-I00; Conselho Nacional de Desenvolvimento Cientifico e Tecnologico (CNPQ) PC095-096; Fundacao de Amparo a Ciencia e Tecnologia do Estado do Rio Grande do Sul (FAPERGS) P18-FR-4961 301618/2019-4 19/2551-000 1279-9Abstract
Traditionally, Convolutional Neural Networks make use of the maximum or arithmetic mean in
order to reduce the features extracted by convolutional layers in a downsampling process known
as pooling. However, there is no strong argument to settle upon one of the two functions and, in
practice, this selection turns to be problem dependent. Further, both of these options ignore possible
dependencies among the data. We believe that a combination of both of these functions, as well
as of additional ones which may retain different information, can benefit the feature extraction
process. In this work, we replace traditional pooling by several alternative functions. In particular, we
consider linear combinations of order statistics and generalizations of the Sugeno integral, extending
the latter’s domain to the whole real line and setting the theoretical base for their application. We
present an alternative pooling layer based on this strategy which we name ‘‘CombPool’’ layer. We
replace the pooling layers of three different architectures of increasing complexity by CombPool
layers, and empirically prove over multiple datasets that linear combinations outperform traditional
pooling functions in most cases. Further, combinations with either the Sugeno integral or one of its
generalizations usually yield the best results, proving a strong candidate to apply in most architectures.