2Huaiyin Institute of Technology
We propose a novel depth-centric learning (DCL) framework that integrates the atmospheric scattering model (ASM) with the brightness consistency constraint (BCC) constraint. Our key idea is that both ASM and BCC rely on a shared depth estimation network. This network simultaneously exploits adjacent dehazed frames to enhance depth estimation via BCC and uses the refined depth cues to more effectively remove haze through ASM.
For more video demos, please refer to our project homepage.
- [16-12-2024] The "Chinese Interpretation" version of DCL has been added.
- [14-12-2024] Training and inference code is released. (this repository).
- [13-12-2024] We created the project homepage and the GitHub README.
- [10-12-2024] Accepted to AAAI 2025.
Our Depth-Centric Learning (DCL) framework integrates the atmospheric scattering model with a brightness consistency constraint via shared depth prediction.
git clone https://github.com/fanjunkai1/DCL.git
conda create -n DCL python=3.9.19
conda activate DCL
conda install pytorch==1.11.0 torchvision==0.12.0 torchaudio==0.11.0 cudatoolkit=11.3 -c pytorch
pip install -r requirements.txtThe original DVD (CVPR 2024) dataset (1920x1080 size) can be downloaded from the following link:
- GoProHazy dataset can be downloaded from Baidu Drive(hbih).
- DrivingHazy dataset can be downloaded from Baidu Drive(ei4j).
- InternetHazy dataset can be downloaded from Baidu Drive(p39a).
Google Drive users can download the GoProHazy, DrivingHazy, and InternetHazy datasets via this link
All videos in these datasets are initially recorded at a resolution of 1920×1080. After applying distortion correction and cropping based on the intrinsic parameters K of the GoPro 11 camera (calibrated by us), the resolutions of GoProHazy and DrivingHazy are 1600×512.
- Camera Calibration. Download the chessboard image set from here, and place the data in the
./calibrate folder, then run the following command:
python calibrate.py- Undistort and Crop. Use the calibration results to undistort and crop the 1920x1080 image to 1600x512, then save the new intrinsic parameters. The command is as follows:
python preprocess.pyThe data organization for the ./preprocess folder is shown below:
preprocess
|--gopro_preprocess
|--videos
|--test
|--clear_video
|--...
|--hazy_video
|--...
|--processed
|--...
|-- train
|--clear_video
|--1_clear_video.mp4
|--...
|--hazy_video
|--1_hazy_video.mp4
|--...
|--processed
|--clear_video
|--hazy_video
|--intrinsic.npy
Note: The preprocessed GoProHazy dataset can be downloaded here.
The data format in the ./split/gopro_fan folder and the corresponding explanation are as follows:
......
train_video/hazy_video/7_hazy_video 6 5 4 5
train_video/hazy_video/7_hazy_video 7 5 5 6
train_video/hazy_video/7_hazy_video 8 6 5 7
......| the folder of current frame t | t idx | t matched idx | t-1 matched idx | t+1 matched idx |
|---|---|---|---|---|
| train_video/hazy_video/7_hazy_video | 7 | 5 | 5 | 6 |
- Training model. Place the gopro_data (preprocessed GoProHazy) folder, downloaded from Google Drive, into the
./datafolder, and then execute the following command:
python train.py --model_name DCL- Training Visualization. The training and validation log files for DCL are saved in the train and val folders under
./logger/DCL. They can be visualized in TensorBoard using the following command:
cd DCL
tensorboard --logdir=./logger/DCLTo visualize locally from a remote server, run ssh -L 16006:127.0.0.1:6006 -p xxxxx root@serverIP to set up the tunnel, then use the command above. Finally, open http://localhost:16006/ in your browser for training visualization.
Move the trained model from ./logger/DCL/models to the ./models/DCL folder, or use our pre-trained model, which can be downloaded from Google Drive. Then, execute the following command:
python test_gopro_hazy.py --image_path ./data/gopro_data/test_video/hazy_video/24_hazy_videopython test_dense_fog.py --image_path /opt/data/common/SeeingThroughFog/SeeingThroughFogCompressedExtracted --dataset densefog --load_weights_folder ./models/DCL
python test_dense_fog.py --image_path /opt/data/common/SeeingThroughFog/SeeingThroughFogCompressedExtracted --dataset lightfog --load_weights_folder ./models/DCLThe output test results are saved in the ./outputs folder.
Our DCL achieved state-of-the-art performance on GoProHazy and DENSE-Fog datasets,
Visual Comparison (click to expand)
-
Visual comparison for video dehazing
-
Visual comparison for depth estimation
If you find the code helpful in your research or work, please cite the following paper(s).
@inproceedings{fan2025depth,
title={Depth-Centric Dehazing and Depth-Estimation from Real-World Hazy Driving Video},
author={Fan, Junkai and Wang, Kun and Yan, Zhiqiang and Chen, Xiang and Gao, Shangbing and Li, Jun and Yang, Jian},
booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
pages={xxxxx--xxxxx},
year={2025}
}
@inproceedings{fan2024driving,
title={Driving-Video Dehazing with Non-Aligned Regularization for Safety Assistance},
author={Fan, Junkai and Weng, Jiangwei and Wang, Kun and Yang, Yijun and Qian, Jianjun and Li, Jun and Yang, Jian},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={26109--26119},
year={2024}
}This code is based on the Monodepth2 and DVD. Thank them for their outstanding work.
If you have any questions or suggestions, please contact junkai.fan@njust.edu.cn
This work is licensed under the Apache License, Version 2.0 (as defined in the LICENSE).
By downloading and using the code and model you agree to the terms in the LICENSE.