EnvGS: Modeling View-Dependent Appearance with Environment Gaussian

Project Page | Paper | arXiv

EnvGS: Modeling View-Dependent Appearance with Environment Gaussian
Tao Xie*, Xi Chen*, Zhen Xu, Yiman Xie, Yudong Jin, Yujun Shen, Sida Peng, Hujun Bao, Xiaowei Zhou
arXiv 2024

News:

• 24.01.07: The 2D Gaussian Tracer for EnvGS has also been open-sourced.

Pipeline

Code

Tracer

We have released the code for our 2D Gaussian ray tracer, built on the foundation of 2DGS and NVIDIA OptiX, tailored for differentiable optimization and rendering tasks. Key features of this tracer include:

• Differentiability: The tracer is entirely differentiable, encompassing the 2D Gaussian parameters and the input ray origins and directions, should the rays be optimized.
• Path Tracing: It supports path tracing with multiple bounces, which is beneficial for rendering complex materials and simulating intricate light transport phenomena.
• Customizable Rendering: The tracer allows for customized rendering, enabling you to incorporate additional precomputed parameters and outputs tailored to your specific requirements.

Please refer to the 2D Gaussian Tracer for more details.

teaser.mp4

EnvGS

Our complete EnvGS code is coming soon, stay tuned!

TODOs

• TODO: Release 2D Gaussian ray tracer.
• TODO: Release the training and evaluation code of EnvGS.

Acknowledgments

This work is implemented using our PyTorch framework, EasyVolcap, feel free to explore it.

• EasyVolcap: Accelerating Neural Volumetric Video Research

We would also like to acknowledge the following inspiring prior work:

• NeRF-Casting: Improved View-Dependent Appearance with Consistent Reflections
• 3D Gaussian Ray Tracing: Fast Tracing of Particle Scenes
• Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields
• 3D Gaussian Splatting for Real-Time Radiance Field Rendering
• 2DGS: 2D Gaussian Splatting for Geometrically Accurate Radiance Fields

Citation

If you find this code useful for your research, please use the following BibTeX entry.

@article{xie2024envgs,
  title={EnvGS: Modeling View-Dependent Appearance with Environment Gaussian},
  author={Xie, Tao and Chen, Xi and Xu, Zhen and Xie, Yiman and Jin, Yudong and Shen, Yujun and Peng, Sida and Bao, Hujun and Zhou, Xiaowei},
  journal={arXiv preprint arXiv:2412.15215},
  year={2024}
}