Institute Homepage
Institute Homepage DE Sign In

Back

Autonomous Vision Conference Paper 2018

Deep Marching Cubes: Learning Explicit Surface Representations

Thumb ticker sm screen shot 2016 10 03 at 7.50.00 pm
Autonomous Vision
Yiyi Liao
Thumb ticker sm 15622370 10154388287129965 2854177071828643222 n
Autonomous Vision
Simon Donne
Thumb ticker sm upper body tiny
Autonomous Vision, Perceiving Systems
Andreas Geiger
Guest Scientist
Yiyi paper teaser

Existing learning based solutions to 3D surface prediction cannot be trained end-to-end as they operate on intermediate representations (eg, TSDF) from which 3D surface meshes must be extracted in a post-processing step (eg, via the marching cubes algorithm). In this paper, we investigate the problem of end-to-end 3D surface prediction. We first demonstrate that the marching cubes algorithm is not differentiable and propose an alternative differentiable formulation which we insert as a final layer into a 3D convolutional neural network. We further propose a set of loss functions which allow for training our model with sparse point supervision. Our experiments demonstrate that the model allows for predicting sub-voxel accurate 3D shapes of arbitrary topology. Additionally, it learns to complete shapes and to separate an object's inside from its outside even in the presence of sparse and incomplete ground truth. We investigate the benefits of our approach on the task of inferring shapes from 3D point clouds. Our model is flexible and can be combined with a variety of shape encoder and shape inference techniques.

Author(s): Yiyi Liao and Simon Donne and Andreas Geiger
Book Title: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Year: 2018
Publisher: IEEE Computer Society
Project(s):
Bibtex Type: Conference Paper (inproceedings)
Event Name: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) 2018
Event Place: Salt Lake City, USA
Electronic Archiving: grant_archive
Links:

BibTex 

@inproceedings{Liao2018CVPR,
  title = {Deep Marching Cubes: Learning Explicit Surface Representations },
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  abstract = {Existing learning based solutions to 3D surface prediction cannot be trained end-to-end as they operate on intermediate representations (eg, TSDF) from which 3D surface meshes must be extracted in a post-processing step (eg, via the marching cubes algorithm). In this paper, we investigate the problem of end-to-end 3D surface prediction. We first demonstrate that the marching cubes algorithm is not differentiable and propose an alternative differentiable formulation which we insert as a final layer into a 3D convolutional neural network. We further propose a set of loss functions which allow for training our model with sparse point supervision. Our experiments demonstrate that the model allows for predicting sub-voxel accurate 3D shapes of arbitrary topology. Additionally, it learns to complete shapes and to separate an object's inside from its outside even in the presence of sparse and incomplete ground truth. We investigate the benefits of our approach on the task of inferring shapes from 3D point clouds. Our model is flexible and can be combined with a variety of shape encoder and shape inference techniques.},
  publisher = {IEEE Computer Society},
  year = {2018},
  slug = {liao2018cvpr},
  author = {Liao, Yiyi and Donne, Simon and Geiger, Andreas}
}