In some neuronal networks in the brain which are thought to operate as associative memories, a sparse coding of information can enhance the storage capacity. The extent to which this statement is valid in general is discussed here, by considering some simple formal models of associative memory which include different neurobiological constraints. In nets of linear neurons, trained with either a Hebbian (purely incremental) or a Stanton and Sejnowski learning rule, sparse coding increases the number of independent associations that can be stored. When neurons are nonlinear, for a diversity of learning rules, sparse coding may result in an increase in the number of patterns that can be discriminated. The analysis is then used to help interpret recent evidence on the encoding of information in the taste and visual systems, as obtained from recordings in primates. Following the taste pathway, it is found that the breadth of tuning of individual neurons becomes progressively finer, consistent with the idea that sparser representations become advantageous as the taste information is eventually associated with that coming from other sensory modalities. In the visual system, considering a population of neurons in the temporal cortex that respond preferentially to faces, it is argued that their breadth of tuning represents a compromise between fully distributed encoding, and a grandmother cell type of encoding, which would result in a given neuron responding only to an individual face. © 1990 Informa UK Ltd All rights reserved: reproduction in whole or part not permitted.

The relative advantages of sparse versus distributed encoding for associative neuronal networks in the brain / Rolls, Edmund T; Treves, Alessandro. - In: NETWORK. - ISSN 0954-898X. - 1:4(1990), pp. 407-421. [10.1088/0954-898X_1_4_002]

The relative advantages of sparse versus distributed encoding for associative neuronal networks in the brain

Rolls, Edmund T;Treves, Alessandro
1990

Abstract

In some neuronal networks in the brain which are thought to operate as associative memories, a sparse coding of information can enhance the storage capacity. The extent to which this statement is valid in general is discussed here, by considering some simple formal models of associative memory which include different neurobiological constraints. In nets of linear neurons, trained with either a Hebbian (purely incremental) or a Stanton and Sejnowski learning rule, sparse coding increases the number of independent associations that can be stored. When neurons are nonlinear, for a diversity of learning rules, sparse coding may result in an increase in the number of patterns that can be discriminated. The analysis is then used to help interpret recent evidence on the encoding of information in the taste and visual systems, as obtained from recordings in primates. Following the taste pathway, it is found that the breadth of tuning of individual neurons becomes progressively finer, consistent with the idea that sparser representations become advantageous as the taste information is eventually associated with that coming from other sensory modalities. In the visual system, considering a population of neurons in the temporal cortex that respond preferentially to faces, it is argued that their breadth of tuning represents a compromise between fully distributed encoding, and a grandmother cell type of encoding, which would result in a given neuron responding only to an individual face. © 1990 Informa UK Ltd All rights reserved: reproduction in whole or part not permitted.
1
4
407
421
https://www.tandfonline.com/doi/abs/10.1088/0954-898X_1_4_002
Rolls, Edmund T; Treves, Alessandro
File in questo prodotto:
File Dimensione Formato  
Rol+90.pdf

non disponibili

Tipologia: Versione Editoriale (PDF)
Licenza: Non specificato
Dimensione 1.03 MB
Formato Adobe PDF
1.03 MB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/20.500.11767/85744
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 134
  • ???jsp.display-item.citation.isi??? 125
social impact