Autonomous Vision Conference Paper 2021

KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs

Kilonerf

NeRF synthesizes novel views of a scene with unprecedented quality by fitting a neural radiance field to RGB images. However, NeRF requires querying a deep Multi-Layer Perceptron (MLP) millions of times, leading to slow rendering times, even on modern GPUs. In this paper, we demonstrate that real-time rendering is possible by utilizing thousands of tiny MLPs instead of one single large MLP. In our setting, each individual MLP only needs to represent parts of the scene, thus smaller and faster-to-evaluate MLPs can be used. By combining this divide-and-conquer strategy with further optimizations, rendering is accelerated by three orders of magnitude compared to the original NeRF model without incurring high storage costs. Further, using teacher-student distillation for training, we show that this speed-up can be achieved without sacrificing visual quality.

Author(s): Christian Reiser and Songyou Peng and Yiyi Liao and Andreas Geiger
Book Title: 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Pages: 14315--14325
Year: 2021
Bibtex Type: Conference Paper (inproceedings)
DOI: 10.1109/ICCV48922.2021.01407
Event Name: IEEE/CVF International Conference on Computer Vision (ICCV 2021)
Event Place: Montreal
State: Published
URL: https://ieeexplore.ieee.org/document/9710464
Electronic Archiving: grant_archive
Links:

BibTex

@inproceedings{Reiser2021ICCV,
  title = {KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs},
  booktitle = {2021 IEEE/CVF International Conference on Computer Vision (ICCV) },
  abstract = {NeRF synthesizes novel views of a scene with unprecedented quality by fitting a neural radiance field to RGB images. However, NeRF requires querying a deep Multi-Layer Perceptron (MLP) millions of times, leading to slow rendering times, even on modern GPUs. In this paper, we demonstrate that real-time rendering is possible by utilizing thousands of tiny MLPs instead of one single large MLP. In our setting, each individual MLP only needs to represent parts of the scene, thus smaller and faster-to-evaluate MLPs can be used. By combining this divide-and-conquer strategy with further optimizations, rendering is accelerated by three orders of magnitude compared to the original NeRF model without incurring high storage costs. Further, using teacher-student distillation for training, we show that this speed-up can be achieved without sacrificing visual quality.},
  pages = {14315--14325 },
  year = {2021},
  slug = {reiser2021iccv},
  author = {Reiser, Christian and Peng, Songyou and Liao, Yiyi and Geiger, Andreas},
  url = {https://ieeexplore.ieee.org/document/9710464}
}