Institute Homepage
Institute Homepage DE Sign In

Back

Neural Capture and Synthesis Conference Paper 2023

DINER: Depth-aware Image-based Neural Radiance Fields

Thumb ticker sm me
Perceiving Systems
Malte Prinzler
  • Guest Scientist
Thumb ticker sm justus thies
Neural Capture and Synthesis, Perceiving Systems
Justus Thies
Max Planck Research Group Leader
Mpi diner teaser

We present Depth-aware Image-based NEural Radiance fields (DINER). Given a sparse set of RGB input views, we predict depth and feature maps to guide the reconstruction of a volumetric scene representation that allows us to render 3D objects under novel views. Specifically, we propose novel techniques to incorporate depth information into feature fusion and efficient scene sampling. In comparison to the previous state of the art, DINER achieves higher synthesis quality and can process input views with greater disparity. This allows us to capture scenes more completely without changing capturing hardware requirements and ultimately enables larger viewpoint changes during novel view synthesis. We evaluate our method by synthesizing novel views, both for human heads and for general objects, and observe significantly improved qualitative results and increased perceptual metrics compared to the previous state of the art. The code is publicly available for research purposes.

Author(s): Malte Prinzler and Otmar Hilliges and Justus Thies
Book Title: IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR)
Year: 2023
Bibtex Type: Conference Paper (inproceedings)
Event Name: CVPR 2023
Event Place: Vancouver
State: Accepted
URL: https://malteprinzler.github.io/projects/diner/diner.html
Degree Type: PhD
Digital: True
Electronic Archiving: grant_archive
Links:

BibTex 

@inproceedings{prinzler2023diner,
  title = {DINER: Depth-aware Image-based Neural Radiance Fields},
  booktitle = {IEEE/CVF Conf.~on Computer Vision and Pattern Recognition (CVPR)},
  abstract = {We present Depth-aware Image-based NEural Radiance fields (DINER). Given a sparse set of RGB input views, we predict depth and feature maps to guide the reconstruction of a volumetric scene representation that allows us to render 3D objects under novel views. Specifically, we propose novel techniques to incorporate depth information into feature fusion and efficient scene sampling. In comparison to the previous state of the art, DINER achieves higher synthesis quality and can process input views with greater disparity. This allows us to capture scenes more completely without changing capturing hardware requirements and ultimately enables larger viewpoint changes during novel view synthesis. We evaluate our method by synthesizing novel views, both for human heads and for general objects, and observe significantly improved qualitative results and increased perceptual metrics compared to the previous state of the art. The code is publicly available for research purposes.},
  degree_type = {PhD},
  year = {2023},
  slug = {prinzler2023diner},
  author = {Prinzler, Malte and Hilliges, Otmar and Thies, Justus},
  url = {https://malteprinzler.github.io/projects/diner/diner.html}
}