[CVPR 2026] UFO: Unifying Feed-Forward and Optimization-based Methods for Large Driving Scene Modeling

Kaiyuan Tan^1,2,*, Yingying Shen^1,*, Ziyue Zhu¹, Mingfei Tu¹, Haohui Zhu¹, Bing Wang¹, Guang Chen¹, Hangjun Ye^1,✉, Haiyang Sun^1,†

¹ Xiaomi EV ² UIUC

(*) Equal contribution. (†) Project leader. (✉)Corresponding Author.

Abstract

Dynamic driving scene reconstruction is critical for autonomous driving simulation and closed-loop learning. While recent feed-forward methods have shown promise for 3D reconstruction, they struggle with long-range driving sequences due to quadratic complexity in sequence length and challenges in modeling dynamic objects over extended durations. We propose UFO, a novel recurrent paradigm that combines the benefits of optimization-based and feed-forward methods for efficient long-range 4D reconstruction.Our approach maintains a 4D scene representation that is iteratively refined as new observations arrive, using a visibility-based filtering mechanism to select informative scene tokens and enable efficient processing of long sequences. For dynamic objects, we introduce an object pose-guided modeling approach that supports accurate long-range motion capture. Experiments on the Waymo Open Dataset demonstrate that our method significantly outperforms both per-scene optimization and existing feedforward methods across various sequence lengths. Notably, our approach can reconstruct 16-second driving logs within 0.5 second while maintaining superior visual quality and geometric accuracy.

Overview

Updates

• Release Paper
• Release v1 Models
• Release Full Models
• Release Inference Framework
• Release Training Framework

License

This project is released under the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) license. See LICENSE for the full text. The non-commercial restriction means the code and trained checkpoints are usable for research and evaluation purposes only; please contact the authors for commercial use.

Acknowledgments

Parts of this codebase are adapted/inspired from open-source projects we gratefully acknowledge:

• timm
• DiT
• GLIDE
• GaussianSTORM

Citation

@misc{tan2026ufounifyingfeedforwardoptimizationbased,
      title={UFO: Unifying Feed-Forward and Optimization-based Methods for Large Driving Scene Modeling}, 
      author={Kaiyuan Tan and Yingying Shen and Mingfei Tu and Haohui Zhu and Bing Wang and Guang Chen and Hangjun Ye and Haiyang Sun},
      year={2026},
      eprint={2602.20943},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2602.20943}, 
}