BiNAR achieves pixel-level aligned RGB-IR bi-modal 3D scene reconstruction and rendering.
Clone this repository and set up the environment with the following command:
git clone git@github.com:jankin-wang/BiNAR.git
cd BiNAR
conda create -y -n binar python=3.8
conda activate binar
pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 -f https://download.pytorch.org/whl/torch_stable.html
conda install cudatoolkit-dev=11.3 -c conda-forge
pip install -r requirements.txt
pip install submodules/diff-gaussian-rasterization
pip install submodules/simple-knn/
Please download the raw data from PARID_Raw and place it in the ./dataset/PARID_Raw folder under the project directory.
The PARID (Pixel-Aligned RGB-IR Dataset) provides pixel-level aligned RGB and IR image pairs across both indoor and outdoor scenes. Each IR image retains real thermal information. If you need to recover the temperature information of each pixel in the scene, use the temperature range in the table below to perform inverse normalization.
| Scene | Type | Temperature Min (°C) | Temperature Max (°C) |
|---|---|---|---|
| Desktop | Indoor | 0 | 80 |
| UAV | Indoor | 14 | 34 |
| Kettles | Indoor | 7 | 33 |
| Computer | Indoor | 10 | 60 |
| Aircon | Indoor | 1 | 50 |
| Apples | Indoor | -5 | 30 |
| Bottles | Indoor | -6 | 30 |
| E-Bike | Outdoor | 5 | 24 |
| Car | Outdoor | 5 | 21 |
| Bicycle | Outdoor | 5 | 25 |
To start training, rendering and evaluating, simply use:
python scripts/run_joint.py
If you find our work useful in your research, please consider citing: