UMA: Ultra-detailed Human Avatars via Multi-level Surface Alignment

Heming Zhu · Guoxing Sun · Christian Theobalt · Marc Habermann‡

Project Page  |  arXiv  |  Dataset

---

News

• [2026-05-15] 🎉 Created the GitHub repository.
• [2026-05-15] 🚀 The UMA Dataset is released — see this link.

---

Dataset

Please find the Raw Data, Annotations, and Metadata that are used in the paper from this link. The dataset covers 5 subjects (Subject_0 .. Subject_4), each with long, dense multi-view video sequences performing everyday motions. Every sequence is paired with Foreground Segmentation and Per-frame Groundtruth 3D Reconstructions generated by NeuS2.

Access

1. Register first. Open the dataset page and click the registration link to request an account — the per-subject packs are behind login.
2. Once your account is approved, you can either:
   • Click individual files in the browser. Every pack (Subject_X_metadata.zip, Subject_X_<split>_fgseg.tar, Subject_X_<split>_neus2.tar.gz, Subject_X_<split>_video.tar.part_NN) is a direct download link on the page.
   • Use the provided script for bulk / unattended downloads. Grab dataset_downloader.py from the page and run it in a terminal — it logs in with your credentials, filters by SUBJECTS / MODALITIES, resumes interrupted downloads, and cleans up partial .tmp files on failure. See the Download with scripts section below.

Folder structure

Each subject ships 3 sequence packs per split (video + fgseg + neus2) plus a single metadata.zip. Extracting the fgseg and neus2 packs inside the corresponding Subject_X/training/ (or Subject_X/testing/) directory yields the layout below:

|--- Subject_X
    |---training                                # training split
    |   |--- stream000.mp4 ... stream039.mp4    # 40 multi-view RGB streams
    |   |--- foregroundSegmentation             # from Subject_X_training_fgseg.tar
    |   |       |--- stream000.mp4 ... stream039.mp4
    |   |--- recon_neus2                        # from Subject_X_training_neus2.tar.gz
    |           |--- 000000.ply                 # one NeuS2 reconstruction per frame
    |           |--- 000001.ply
    |           |--- ...
    |---testing
        |--- ...                                # same layout as training

Notes on the packs:

• Subject_X_<split>_video.tar.part_* → top-level training/ (or testing/), excluding foregroundSegmentation* / recon_neus2* / recon / recon_metashape (these are produced by sibling packs).
• Subject_X_<split>_fgseg.tar → top-level foregroundSegmentation/; the source variant (_recon / _refined / _sapien_1k) is normalized to a single name.
• Subject_X_<split>_neus2.tar.gz → top-level recon_neus2/, with per-frame <frame>/model.ply flattened to <frame>.ply.

Metadata (Subject_X_metadata.zip)

The per-subject metadata bundle (≤ 50 MB) ships DDC and SMPL-X characters side by side, plus the shared camera calibration. Details on loading and posing the DDC character can be found in DeepCharacters Pytorch.

Subject_X_metadata
|--- cameras.calib                          # camera intrinsics / extrinsics
|--- ddc_character
|   |--- ddc.character                      # character setting
|   |--- ddc.motion                         # motion file
|   |--- ddc.skeleton                       # template skeleton
|   |--- ddc.mtl                            # template material
|   |--- ddc.obj.mtl                        # template OBJ material
|   |--- ddc_graph.obj                      # embedded graph mesh
|   |--- ddc_graph.obj.laplacian            # graph Laplacian
|   |--- ddc_graph.obj.mtl                  # graph material
|   |--- base_trans.pkl                     # translation for normalization
|   |--- dof_convert_dict.pkl               # dof with/without hands
|   |--- segmentation.txt                   # template segmentation
|   |--- skeletoolToGTPose                  # training skeleton poses
|   |       |--- ...
|   |--- skeletoolToGTPose_test             # testing skeleton poses
|           |--- ...
|--- smplx_character
    |--- fitted_smplx_mesh.ply              # SMPL-X fitted mesh
    |--- smplx_shape.npz                    # SMPL-X shape parameters
    |--- smplx_train_merged.npz             # SMPL-X training poses
    |--- smplx_test_merged.npz              # SMPL-X testing poses

Download with scripts

A helper script dataset_downloader.py is provided on the dataset page. It logs in, lists all packs, and downloads only the subjects / modalities you select, with resume and partial-file cleanup.

# 1. Install dependencies
pip install requests beautifulsoup4 tqdm

# 2. Edit credentials and selection at the top of dataset_downloader.py
#    SUBJECTS   = [0, 1, 2, 3, 4]   # or []  for all
#    MODALITIES = ["metadata", "fgseg", "neus2", "video"]   # or []  for all

# 3. Run
python dataset_downloader.py

Multi-part video packs are concatenated and extracted with:

# video (uncompressed tar, multi-volume)
cat Subject_3_training_video.tar.part_* | tar -xf -

# neus2 (gzipped tar, single file)
tar -xzf Subject_3_training_neus2.tar.gz

# fgseg (uncompressed tar, single file)
tar -xf Subject_3_training_fgseg.tar

# metadata
unzip Subject_3_metadata.zip

---

TODO

• Release the UMA dataset.
• Release the demo code.
• Release the training code.

---

Citation

If you find our dataset and source code useful for your research, please consider citing our paper:

@article{zhu2025ultra,
  title={UMA: Ultra-detailed Human Avatars via Multi-level Surface Alignment},
  author={Zhu, Heming and Sun, Guoxing and Theobalt, Christian and Habermann, Marc},
  journal={arXiv preprint arXiv:2506.01802},
  year={2025}
}

---

Contact

For questions and clarifications, feel free to get in touch with: Heming Zhu: hezhu@mpi-inf.mpg.de Marc Habermann: mhaberma@mpi-inf.mpg.de

---

License

UMA is under the CC-BY-NC license. The license applies to the dataset, metadata, and any released pre-trained models.