ExplainLFS: Explaining neural architectures for similarity learning from local perturbations in the latent feature space
Metadatos
Mostrar el registro completo del ítemAutor
Bello García, Marilyn; Costa, Pablo; Nápoles, Gonzalo; Mesejo Santiago, Pablo; Cordón García, ÓscarEditorial
Elsevier
Materia
Similarity learning networks Face recognition Image retrieval
Fecha
2024-04-05Referencia bibliográfica
Bello, Marilyn, et al. ExplainLFS: Explaining neural architectures for similarity learning from local perturbations in the latent feature space. Information Fusion 108 (2024) 102407 [10.1016/j.inffus.2024.102407]
Patrocinador
R&D project CONFIA (PID2021-122916NB-I00), funded by MICIU/AEI/10.13039/501100011033/ and FEDER, EU; Funding for open access charges is covered by Universidad de Granada / CBUAResumen
Despite the increasing development in recent years of explainability techniques for deep neural networks, only
some are dedicated to explaining the decisions made by neural networks for similarity learning. While existing
approaches can explain classification models, their adaptation to generate visual similarity explanations is not
trivial. Neural architectures devoted to this task learn an embedding that maps similar examples to nearby
vectors and non-similar examples to distant vectors in the feature space. In this paper, we propose a post-hoc
agnostic technique that explains the inference of such architectures on a pair of images. The proposed method
establishes a relation between the most important features of the abstract feature space and the input feature
space (pixels) of an image. For this purpose, we employ a relevance assignment and a perturbation process
based on the most influential latent features in the inference. Then, a reconstruction process of the images of the
pair is carried out from the perturbed embedding vectors. This process relates the latent features to the original
input features. The results indicate that our method produces ‘‘continuous’’ and ‘‘selective’’ explanations. A
sharp drop in the value of the function (summarized by a low value of the area under the curve) indicates its
superiority over other explainability approaches when identifying features relevant to similarity learning. In
addition, we demonstrate that our technique is agnostic to the specific type of similarity model, e.g., we show
its applicability in two similarity learning tasks: face recognition and image retrieval.