NeRFLiX: High-Quality Neural View Synthesis by Learning a Degradation-Driven Inter-viewpoint MiXer (CVPR2023)
This is the official implementation of NeRFLiX.
Our recent work NeRFLiX++, an improved version that is stronger, faster and for 4K NeRFs. We will release the source code later.
The project website is now available.
'''
LLFF /Tanks
# python test_llff_tensorRF/test_tankandtemple_tensorRF.py [ pretrained_weights | view_selection json file | nerf rendered test view |enhanced_result_path ]
I also give a simple script about how to enhance novel views by nerf model. The pre-trained model and some josn files are avaliable at google drive
step 1 : perpare a trained NeRF model on a 3D scene [training images and training poses]
step 2 : utilize this NeRF model to render novel views(images) given a set of novel camera poses
step 3 : run 'pose_matching.py' by given four necessary inputs:
(1) training images (2) training camera poses (3) novel images (4) novel camera poses
after that , you will get the view selection results saved in a josn file
step 4 : run the 'test_novelviews.py' , then the enhanced views will be placed in the result_path
Also you can train an in-the-wild scene using SOTA NeRF approaches and enjoy the photo-relastic results by NeRFLiX.
I put the training code in the train folder.
you should download the vimeo_septuplet and eight LLFF-T scenes(drop every eighth frame)
then simple run the following commend using single or multiple GPUs
python3 -m torch.distributed.launch --nproc_per_node=gpu_number --master_port=port_code $(dirname $0)/train.py
We also upload a video demo at YouTube.
Please note that the code in this repository is intended solely for research purposes and is not licensed for commercial use.
If our work is useful for your research, please consider citing:
@inproceedings{zhou2023nerflix,
title={NeRFLiX: High-Quality Neural View Synthesis by Learning a Degradation-Driven Inter-viewpoint MiXer},
author={Zhou, Kun and Li, Wenbo and Wang, Yi and Hu, Tao and Jiang, Nianjuan and Han, Xiaoguang and Lu, Jiangbo},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={12363--12374},
year={2023}
}
@article{zhou2023nerflix,
title={From NeRFLiX to NeRFLiX++: A General NeRF-Agnostic Restorer Paradigm},
author={Zhou, Kun and Li, Wenbo and Jiang, Nianjuan and Han, Xiaoguang and Lu, Jiangbo},
journal={arXiv preprint arXiv:2306.06388},
year={2023}
}