Institute Homepage
Institute Homepage EN Sign In

Back

Conference Paper 2019

Feudal Multi-Agent Hierarchies for Cooperative Reinforcement Learning

{We investigate how reinforcement learning agents can learn to cooperate. Drawing inspiration from human societies, in which successful coordination of many individuals is often facilitated by hierarchical organisation, we introduce Feudal Multi-agent Hierarchies (FMH). In this framework, a \textquotesinglemanager\textquotesingle agent, which is tasked with maximising the environmentally-determined reward function, learns to communicate subgoals to multiple, simultaneously-operating, \textquotesingleworker\textquotesingle agents. Workers, which are rewarded for achieving managerial subgoals, take concurrent actions in the world. We outline the structure of FMH and demonstrate its potential for decentralised learning and control. We find that, given an adequate set of subgoals from which to choose, FMH performs, and particularly scales, substantially better than cooperative approaches that use a shared reward function.}

Author(s): Ahilan, S and Dayan, P
Book Title: Workshop on Structure & Priors in Reinforcement Learning (SPiRL 2019) at ICLR 2019
Pages: 1--11
Year: 2019
Bibtex Type: Conference Paper (inproceedings)
Address: New Orleans, LA, USA
Electronic Archiving: grant_archive

BibTex 

@inproceedings{item_3023944,
  title = {{Feudal Multi-Agent Hierarchies for Cooperative Reinforcement Learning}},
  booktitle = {{Workshop on Structure \& Priors in Reinforcement Learning (SPiRL 2019) at ICLR 2019}},
  abstract = {{We investigate how reinforcement learning agents can learn to cooperate. Drawing inspiration from human societies, in which successful coordination of many individuals is often facilitated by hierarchical organisation, we introduce Feudal Multi-agent Hierarchies (FMH). In this framework, a \textquotesinglemanager\textquotesingle agent, which is tasked with maximising the environmentally-determined reward function, learns to communicate subgoals to multiple, simultaneously-operating, \textquotesingleworker\textquotesingle agents. Workers, which are rewarded for achieving managerial subgoals, take concurrent actions in the world. We outline the structure of FMH and demonstrate its potential for decentralised learning and control. We find that, given an adequate set of subgoals from which to choose, FMH performs, and particularly scales, substantially better than cooperative approaches that use a shared reward function.}},
  pages = {1--11},
  address = {New Orleans, LA, USA},
  year = {2019},
  slug = {item_3023944},
  author = {Ahilan, S and Dayan, P}
}