ProtoGAN: Towards Few Shot Learning for Action Recognition

2019

Manual

ps

Few-shot learning (FSL) for action recognition is a challenging task of recognizing novel action categories which are represented by few instances in the training data. In a more generalized FSL setting (G-FSL), both seen as well as novel action categories need to be recognized. Conventional classifiers suffer due to inadequate data in FSL setting and inherent bias towards seen action categories in G-FSL setting. In this paper, we address this problem by proposing a novel ProtoGAN framework which synthesizes additional examples for novel categories by conditioning a conditional generative adversarial network with class prototype vectors. These class prototype vectors are learnt using a Class Prototype Transfer Network (CPTN) from examples of seen categories. Our synthesized examples for a novel class are semantically similar to real examples belonging to that class and is used to train a model exhibiting better generalization towards novel classes. We support our claim by performing extensive experiments on three datasets: UCF101, HMDB51 and Olympic-Sports. To the best of our knowledge, we are the first to report the results for G-FSL and provide a strong benchmark for future research. We also outperform the state-of-the-art method in FSL for all the aforementioned datasets.

@manual{protogan:iccvw,
  title = {ProtoGAN: Towards Few Shot Learning for Action Recognition},
  author = {Dwivedi, Sai Kumar and Gupta, Vikram and Mitra, Rahul and Ahmed, Shuaib and Jain, Arjun},
  booktitle = {Proc. International Conference on Computer Vision (ICCV) Workshops},
  month = oct,
  year = {2019},
  doi = {},
  month_numeric = {10}
}