Modular Neural Image Signal Processing

Mahmoud Afifi, Zhongling Wang, Ran Zhang, and Michael S. Brown

_{🎬 Click to watch the video}

We present a modular neural image signal processing (ISP) framework that produces high-quality display-referred images while providing a high degree of modularity with explicit control over multiple intermediate stages of the rendering pipeline. Our ISP is fully differentiable and requires no manual tuning, and its modular structure not only improves rendering accuracy but also enhances scalability, debuggability, generalization to unseen cameras, and flexibility to support different user-preference picture styles within a lightweight and efficient design.

On top of this modular neural ISP, we developed a user-interactive photo-editing tool that supports diverse editing operations, different picture styles, and enables unlimited post-editable re-rendering and re-styling. The tool accepts DNG raw images from any camera as well as sRGB images from third-party sources. Across multiple test sets, our method consistently delivers competitive qualitative and quantitative performance.

🔗 Paper

---

🛠️ Installation

You can set up the environment using Conda or venv:

Option 1: Using Conda (recommended)

# Create and activate a new conda environment
conda create -n modular_neural_isp_env python=3.9 -y
conda activate modular_neural_isp_env

# Install PyTorch (adjust CUDA version if needed)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

# Install all other dependencies
pip install -r requirements.txt

Option 2: Using venv

# Create and activate a virtual environment
python -m venv modular_neural_isp_env
source modular_neural_isp_env/bin/activate        # macOS/Linux
# On Windows:
# .\modular_neural_isp_env\Scripts\activate

# Install PyTorch (adjust CUDA version if needed)
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

# Install all other dependencies
pip install -r requirements.txt

📸 ExifTool (Required for Raw Metadata)

Some components of this project require ExifTool to read raw metadata.

1. Install ExifTool

Linux (Ubuntu/Debian):

sudo apt install libimage-exiftool-perl

Arch Linux:

sudo pacman -S perl-image-exiftool

macOS (Homebrew):

brew install exiftool

Windows:

1. Download the Windows package from: https://exiftool.org
2. Extract the ZIP file.
3. Rename exiftool(-k).exe → exiftool.exe
4. Place it somewhere permanent, e.g. C:\exiftool\exiftool.exe.

2. Configure ExifTool Path

If ExifTool is not detected automatically, update EXIFTOOL_PATH in utils/constants.py:

import platform

if platform.system() == 'Windows':
    EXIFTOOL_PATH = r'C:\exiftool\exiftool.exe'    # Update accordingly
elif platform.system() == 'Darwin':  # macOS
    EXIFTOOL_PATH = '/usr/local/bin/exiftool'   # Update accordingly
else:  # Linux
    EXIFTOOL_PATH = '/usr/bin/exiftool'   # Update accordingly

---

▶️ Get Started

For console-based rendering of a DNG raw file taken by any camera (or any sRGB image produced by third-party software) refer to the main directory.
To launch the interactive photo-editing tool, see the gui directory.

---

⚙️ Training

To retrain our framework, begin by training the denoiser network (see denoising for instructions). Then train the photofinishing module (refer to photofinishing). Finally, train the detail-enhancement network (see enhancement).

---

📊 Testing

To test our method, check the testing example here.
To benchmark other methods using the same evaluation metrics used in our paper, see the evaluation directory.

---

📄 Citation

If you use this code in your research, please cite our paper:

@article{afifi2025modular,
  title={Modular Neural Image Signal Processing},
  author={Afifi, Mahmoud and Wang, Zhongling and Zhang, Ran and Brown, Michael S},
  journal={arXiv preprint arXiv:2512.08564},
  year={2025}
}