Article 2019

Models that learn how humans learn: The case of decision-making and its disorders

{Popular computational models of decision-making make specific assumptions about learning processes that may cause them to underfit observed behaviours. Here we suggest an alternative method using recurrent neural networks (RNNs) to generate a flexible family of models that have sufficient capacity to represent the complex learning and decision- making strategies used by humans. In this approach, an RNN is trained to predict the next action that a subject will take in a decision-making task and, in this way, learns to imitate the processes underlying subjects\textquoteright choices and their learning abilities. We demonstrate the benefits of this approach using a new dataset drawn from patients with either unipolar (n \textequals 34) or bipolar (n \textequals 33) depression and matched healthy controls (n \textequals 34) making decisions on a two-armed bandit task. The results indicate that this new approach is better than baseline reinforcement-learning methods in terms of overall performance and its capacity to predict subjects\textquoteright choices. We show that the model can be interpreted using off-policy simulations and thereby provides a novel clustering of subjects\textquoteright learning processes\textemdashsomething that often eludes traditional approaches to modelling and behavioural analysis.}

Author(s): Dezfouli, A and Griffiths, K and Ramos, F and Dayan, P and Balleine, BW
Journal: {PLoS Computational Biology}
Volume: 16
Number (issue): 6
Pages: 1--33
Year: 2019
Publisher: Public Library of Science
Project(s):
Bibtex Type: Article (article)
DOI: 10.1371/journal.pcbi.1006903
Address: San Francisco, CA
Electronic Archiving: grant_archive

BibTex

@article{item_3067504,
  title = {{Models that learn how humans learn: The case of decision-making and its disorders}},
  journal = {{PLoS Computational Biology}},
  abstract = {{Popular computational models of decision-making make specific assumptions about learning processes that may cause them to underfit observed behaviours. Here we suggest an alternative method using recurrent neural networks (RNNs) to generate a flexible family of models that have sufficient capacity to represent the complex learning and decision- making strategies used by humans. In this approach, an RNN is trained to predict the next action that a subject will take in a decision-making task and, in this way, learns to imitate the processes underlying subjects\textquoteright choices and their learning abilities. We demonstrate the benefits of this approach using a new dataset drawn from patients with either unipolar (n \textequals 34) or bipolar (n \textequals 33) depression and matched healthy controls (n \textequals 34) making decisions on a two-armed bandit task. The results indicate that this new approach is better than baseline reinforcement-learning methods in terms of overall performance and its capacity to predict subjects\textquoteright choices. We show that the model can be interpreted using off-policy simulations and thereby provides a novel clustering of subjects\textquoteright learning processes\textemdashsomething that often eludes traditional approaches to modelling and behavioural analysis.}},
  volume = {16},
  number = {6},
  pages = {1--33},
  publisher = {Public Library of Science},
  address = {San Francisco, CA},
  year = {2019},
  slug = {item_3067504},
  author = {Dezfouli, A and Griffiths, K and Ramos, F and Dayan, P and Balleine, BW}
}