Optimal Input Representation in Neural Systems at the Edge of Chaos
Metadatos
Afficher la notice complèteEditorial
MDPI
Materia
Information processing Input representation Neural networks Criticality hypothesis Edge of chaos Reservoir computing
Date
2021Referencia bibliográfica
Morales, G.B.; Muñoz, M.A. Optimal Input Representation in Neural Systems at the Edge of Chaos. Biology 2021, 10, 702. https:// doi.org/10.3390/biology10080702
Patrocinador
The Spanish Ministry and Agencia Estatal de investigación (AEI) through grant FIS2017-84256-P (European Regional Development Fund); “Consejería de Conocimiento, Investigación Universidad, Junta de Andalucía” and European Regional Development Fund, Project Ref. A-FQM-175-UGR18 and Project Ref. P20-00173Résumé
Shedding light on how biological systems represent, process and store information in noisy
environments is a key and challenging goal. A stimulating, though controversial, hypothesis poses
that operating in dynamical regimes near the edge of a phase transition, i.e., at criticality or the “edge
of chaos”, can provide information-processing living systems with important operational advantages,
creating, e.g., an optimal trade-off between robustness and flexibility. Here, we elaborate on a recent
theoretical result, which establishes that the spectrum of covariance matrices of neural networks
representing complex inputs in a robust way needs to decay as a power-law of the rank, with an
exponent close to unity, a result that has been indeed experimentally verified in neurons of the mouse
visual cortex. Aimed at understanding and mimicking these results, we construct an artificial neural
network and train it to classify images. We find that the best performance in such a task is obtained
when the network operates near the critical point, at which the eigenspectrum of the covariance
matrix follows the very same statistics as actual neurons do. Thus, we conclude that operating near
criticality can also have—besides the usually alleged virtues—the advantage of allowing for flexible,
robust and efficient input representations.