SDFStudio is a unified and modular framework for neural implicit surface reconstruction, built on top of the awesome nerfstudio project. We provide a unified implementation of three major implicit surface reconstruction methods: UniSurf, VolSDF, and NeuS. SDFStudio also supports various scene representions, such as MLPs, Tri-plane, and Multi-res. feature grids, and multiple point sampling strategies such as surface-guided sampling as in UniSurf, and Voxel-surface guided sampling from NeuralReconW. It further integrates recent advances in the area such as the utillization of monocular cues (MonoSDF), geometry regularization (UniSurf) and multi-view consistency (Geo-NeuS). Thanks to the unified and modular implementation, SDFStudio makes it easy to transfer ideas from one method to another. For example, Mono-NeuS applies the idea from MonoSDF to NeuS, and Geo-VolSDF applies the idea from Geo-NeuS to VolSDF.
In this example, we use a proposal network for sampling (inspired by nerfacto and mip-NeRF 360), a big MLP (8 layers with 512 hidden dimension) as scene representations, and train the model on 8 A100 GPUs for ~10 hours on the Heritage dataset.
Reconstruction result of BakedSDF on the mip-NeRF 360 dataset.
Reconstruction results of Bakedangelo on the Tanks and Temples dataset. Bakedangelo combines BakedSDF with numerical gridents and progressive training of Neuralangelo.
Reconstruction results of UniSurf, VolSDF and NeuS using a Multi-Res. Feature Grid representation on the DTU datasets.
Geo-NeuS improves reconstruction quality by enforcing multi-view photometric consistency during optimization. Thanks to our unified framework, we can apply the idea from Geo-NeuS to UniSurf and VolSDF.
Reconstruction result of MonoSDF on the Tanks and Temples dataset Courtroom scene.
We can also apply the idea of MonoSDF to NeuS, as shown here on the Tanks and Temples dataset Auditorium scene:
NeuS with empty space skipping based on nerfacc with monocular prior from MonoSDF, trained for ~17 minutes.
SDFStudio also supports RGB-D data to obtain high-quality 3D reconstruction. Here for example, we train NeuS on the synthetic rgbd dataset from neural rgbd surface reconstruction:
Nerfstudio's viewer can be used for interactive visualization of rendered RGB images, depths and normals.
@misc{Yu2022SDFStudio,
author = {Yu, Zehao and Chen, Anpei and Antic, Bozidar and Peng, Songyou and Bhattacharyya, Apratim
and Niemeyer, Michael and Tang, Siyu and Sattler, Torsten and Geiger, Andreas},
title = {SDFStudio: A Unified Framework for Surface Reconstruction},
year = {2022},
url = {https://github.com/autonomousvision/sdfstudio},
}