HDRI-SR

Joint High Dynamic Range Imaging and Super-Resolution from a Single Image

Jae Woong Soh, Jae Sung Park, and Nam Ik Cho

[Paper] [Arxiv]

Environments

• Ubuntu 18.04
• Tensorflow 1.8
• CUDA 9.0 & cuDNN 7.1
• Python 3.6
• MATLAB

Abstract

This paper presents a new framework for jointly enhancing the resolution and the dynamic range of an image, i.e., simultaneous super-resolution (SR) and high dynamic range imaging (HDRI), based on a convolutional neural network (CNN). From the common trends of both tasks, we train a CNN for the joint HDRI and SR by focusing on the reconstruction of high-frequency details. Specifically, the high-frequency component in our work is the reflectance component according to the Retinex-based image decomposition, and only the reflectance component is manipulated by the CNN while another component (illumination) is processed in a conventional way. In training the CNN, we devise an appropriate loss function that contributes to the naturalness quality of resulting images. Experiments show that our algorithm outperforms the cascade implementation of CNN-based SR and HDRI.

Brief Description of Our Proposed Method

Overall Process of the Proposed Scheme

LDR-LR input is first decomposed into the illumination and the reflectance components. ILL-E and REF-E enhance each of components respectively and finally merged together to generate an HDR-SR image.

Image Decomposition

WLS filtering based decomposition is first applied. The figure shows the luminance, estimated illumination, and reflectance components.

Network Architecture

For REF-E, we adopt the convolutional neural network (CNN), specifically, stacked hourglass structure.

Experimental Results

Quantitative results on the test set (NR-IQAs)

Visualized Results

Guidelines for Codes

Requisites should be installed beforehand.

Clone this repo.

git clone http://github.com/JWSoh/HDRI-SR.git
cd HDRI-SR/

Test

0. Ready for input data (low (standard) dynamic range, low-resolution).

1. Decompose your input data to obtain reflectance components.

Run MATLAB code below.

Decompose_Test.m

2. Pass REF-E (reflectance enhancement).

Run Python code below. Require Tensorflow to run CNN on GPU.

python test_real.py --gpu [GPU_number] --datapath [Input data path] --modelpath [Model_B/Model_C]

--gpu: The number designates the index of GPU which is going to be used. [Default: 0]
--datapath: Path of input reflectances [Default: LR_mat(x2)]
--modelpath: Which model to use? Model_B or Model_C? [Default: Model_B]

3. Recombine or Synthesize to get HDR-SR image.

Run MATLAB code below.

Synthesize_Test.m

4. To visualize the results on your LDR monitor, you need a tone-mapping algorithm.

There are lots of tone-mapping algorithms, you can download some of them from google, or simply use representative algorithms from MATLAB HDR toolbox.

Citation

@article{soh2019joint,
  title={Joint High Dynamic Range Imaging and Super-Resolution From a Single Image},
  author={Soh, Jae Woong and Park, Jae Sung and Cho, Nam Ik},
  journal={IEEE Access},
  volume={7},
  pages={177427--177437},
  year={2019},
  publisher={IEEE}
}