IDArb: Intrinsic Decomposition for Arbitrary Number of Input Views and Illuminations

     

Zhibing Li¹, Tong Wu^{1 †}, Jing Tan¹, Mengchen Zhang^2,3, Jiaqi Wang³, Dahua Lin^{1,3 †
1}The Chinese University of Hong Kong ²Zhejiang University ³Shanghai AI Laboratory
†: Corresponding Authors

v2_1.mp4

• Release inference code and pretrained checkpoints.
• Release training code.

Install

Our environment has been tested on CUDA 11.8 with A100.

git clone git@github.com:Lizb6626/IDArb.git && cd IDArb
conda create -n idarb python==3.8 -y
conda activate idarb
conda install pytorch==2.2.1 torchvision==0.17.1 torchaudio==2.2.1 pytorch-cuda=11.8 -c pytorch -c nvidia
pip install -r requirements.txt

Inference

Single Image Intrinsic Decomposition

python main.py --data_dir example/single --output_dir output/single --input_type single

Multi-view Intrinsic Decomposition

For multi-view intrinsic decomposition, camera pose can be incorporated by enabling the --cam option.

## --num_views: number of input views

# Without camera pose information
python main.py --data_dir example/multi --output_dir output/multi --input_type multi --num_views 4

# With camera pose information
python main.py --data_dir example/multi --output_dir output/multi --input_type multi --num_views 4 --cam

Acknowledgement

This project relies on many amazing repositories. Thanks to the authors for sharing their code and data.

• Wonder3D
• GeoWizard
• EscherNet
• Stanford-ORB
• NeRD

Citation

@article{li2024idarb,
  author    = {Li, Zhibing and Wu, Tong and Tan, Jing and Zhang, Mengchen and Wang, Jiaqi and Lin, Dahua},
  title     = {IDArb: Intrinsic Decomposition for Arbitrary Number of Input Views and Illuminations},
  journal   = {arXiv preprint arXiv:2412.12083},
  year      = {2024},
}