Conference Paper 2019

Tailored dynamic range using an ensemble of networks

The dynamic range quantifies the range of inputs that a neural network can discriminate. It is maximized at a non-equilibrium phase transition. However, besides the actual size of the dynamic range, it is crucial that the interval of discriminable inputs covers the relevant inputs. We show analytically for a generic spiking model that {\textendash} while the dynamic range indeed is maximal at criticality {\textendash} the discriminable intervals are virtually indistinguishable from each other in the vicinity of the phase transition. We identify the constrained discriminable interval to be a result of coalescence (the simultaneous activation of the same unit from multiple sources). In our model, we can compensate coalescence by implementing adaptive synaptic weights and thereby obtain specific discriminable intervals that can be tuned by changing the distance to criticality. This enables us to optimally address particular tasks by constructing tailored ensembles of coalescence-compensated networks, e.g., discriminating very broad or bimodal input distributions, with implications for machine learning approaches such as reservoir computing networks.

Author(s): Zierenberg, J and Wilting, J and Priesemann, V and Levina, A
Book Title: DPG-Frühjahrstagung 2019
Year: 2019
Bibtex Type: Conference Paper (inproceedings)
Address: Regensburg, Germany
Electronic Archiving: grant_archive
Note: DPG-Frühjahrstagung 2019

BibTex

@inproceedings{item_3054505,
  title = {{Tailored dynamic range using an ensemble of networks}},
  booktitle = {{DPG-Fr{\"u}hjahrstagung 2019}},
  abstract = {The dynamic range quantifies the range of inputs that a neural network can discriminate. It is maximized at a non-equilibrium phase transition. However, besides the actual size of the dynamic range, it is crucial that the interval of discriminable inputs covers the relevant inputs. We show analytically for a generic spiking model that {\textendash} while the dynamic range indeed is maximal at criticality {\textendash} the discriminable intervals are virtually indistinguishable from each other in the vicinity of the phase transition. We identify the constrained discriminable interval to be a result of coalescence (the simultaneous activation of the same unit from multiple sources). In our model, we can compensate coalescence by implementing adaptive synaptic weights and thereby obtain specific discriminable intervals that can be tuned by changing the distance to criticality. This enables us to optimally address particular tasks by constructing tailored ensembles of coalescence-compensated networks, e.g., discriminating very broad or bimodal input distributions, with implications for machine learning approaches such as reservoir computing networks.},
  address = {Regensburg, Germany},
  year = {2019},
  note = {DPG-Fr{\"u}hjahrstagung 2019},
  slug = {item_3054505},
  author = {Zierenberg, J and Wilting, J and Priesemann, V and Levina, A}
}