NeRF Implementations in PyTorch: Focus on Speed and Efficiency

The purpose of this repository is to gain a comprehensive understanding of NeRF and Volumetric Rendering, focusing on speed and efficiency of inference. The main objective is to adapt NeRF for real-time applications by experimenting with various accelerated models

Current Implementation: FastNeRF

• Implemented the factorized model.
• Currently, working on integrating caching mechanism from VDB: High-Resolution Sparse Volumes with Dynamic Topology

Future Implementation Plans

• KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs
• PlenOctrees for Real-time Rendering of Neural Radiance Fields
• Baking Neural Radiance Fields for Real-Time View Synthesis

Training

To train the different NeRF models, use the train.py script. The script will support multiple models.

python train.py --model nerf --config ./config/nerf_config.json

Original NeRF Implementation in PyTorch

The entire implementation was initially done in Jupyter Notebooks.

1. 1_camera.ipynb
2. 2_load_dataset.ipynb
3. 3_volumetric_rendering.ipynb
4. 4_voxel_reconstruction.ipynb
5. 5_train_nerf.ipynb

Visualizations

Visual results achieved using the NeRF implementation on a chair dataset:

• The result of Voxel Reconstruction:

• The result of NeRF Output:

• Training Visualization:

Comparing Inference Speed

TODO

References

• NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis
• scratchapixel