S4D: Streaming 4D Real-World Reconstruction with Gaussians and 3D Control Points

   

Bing He^1*;, Yunuo Chen^1*, Guo Lu¹ , Li Song ^1†, Wenjun Zhang<sup>1

1</sup>Shanghai Jiao Tong University

arXiv preprint, 2024

🔆 Introduction

S4D is a generalized streaming workflow for dynamic 3D scene reconstruction. The full process includes the 3D segmentation, 3D control points generation, object-wise motion manipulation, and residual compensation.

3D Control Point is a discrete 3D motion representation. Each control point characterizes local translations and rotations around its space. By introducing a finely designed ray coordinate system, partial parameters of the 3D control points can be obtained directly from the optical flow without training.

All Code will be released soon... 🔨 🚧

📜 Citation

If you find our work useful, please consider citing:

@article{he2024s4d,
  title={S4D: Streaming 4D Real-World Reconstruction with Gaussians and 3D Control Points},
  author={He, Bing and Chen, Yunuo and Lu, Guo and Song, Li and Zhang, Wenjun},
  journal={arXiv preprint arXiv:2408.13036},
  year={2024}
}