Institute Homepage
Institute Homepage EN Sign In

Back

Autonomous Vision Conference Paper 2018

Semantic Visual Localization

Thumb ticker sm upper body tiny
Autonomous Vision, Perzeptive Systeme
Andreas Geiger
Guest Scientist
Teaser andreas

Robust visual localization under a wide range of viewing conditions is a fundamental problem in computer vision. Handling the difficult cases of this problem is not only very challenging but also of high practical relevance, eg, in the context of life-long localization for augmented reality or autonomous robots. In this paper, we propose a novel approach based on a joint 3D geometric and semantic understanding of the world, enabling it to succeed under conditions where previous approaches failed. Our method leverages a novel generative model for descriptor learning, trained on semantic scene completion as an auxiliary task. The resulting 3D descriptors are robust to missing observations by encoding high-level 3D geometric and semantic information. Experiments on several challenging large-scale localization datasets demonstrate reliable localization under extreme viewpoint, illumination, and geometry changes.

Author(s): Johannes Schönberger and Marc Pollefeys and Andreas Geiger and Torsten Sattler
Book Title: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Year: 2018
Publisher: IEEE Computer Society
Project(s):
Bibtex Type: Conference Paper (inproceedings)
Event Name: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2018
Event Place: Salt Lake City, USA
Electronic Archiving: grant_archive
Links:

BibTex 

@inproceedings{Schoenberger2018CVPR,
  title = {Semantic Visual Localization },
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  abstract = {Robust visual localization under a wide range of viewing conditions is a fundamental problem in computer vision. Handling the difficult cases of this problem is not only very challenging but also of high practical relevance, eg, in the context of life-long localization for augmented reality or autonomous robots. In this paper, we propose a novel approach based on a joint 3D geometric and semantic understanding of the world, enabling it to succeed under conditions where previous approaches failed. Our method leverages a novel generative model for descriptor learning, trained on semantic scene completion as an auxiliary task. The resulting 3D descriptors are robust to missing observations by encoding high-level 3D geometric and semantic information. Experiments on several challenging large-scale localization datasets demonstrate reliable localization under extreme viewpoint, illumination, and geometry changes.},
  publisher = {IEEE Computer Society},
  year = {2018},
  slug = {schoenberger2018cvpr},
  author = {Sch{\"o}nberger, Johannes and Pollefeys, Marc and Geiger, Andreas and Sattler, Torsten}
}