MISATO dataset

[preprint][intro][demo:Youtube,Bilibili]

Overview

This repo contains the proposed evaluation dataset MISATO in our paper "Towards Stable and Faithful Inpainting" .

Our proposed Aligned Stable Inpainting with UnKnown Areas Prior (ASUKA) employs a reconstruction-based masked auto-encoder as a stable prior. Aligned with the stable diffusion inpainting model, ASUKA significantly improves inpainting stability. ASUKA further aligns masked and unmasked regions through an inpainting-specialized decoder, ensuring more faithful inpainting.

To validate the inpainting performance across different domains and mask styles, we construct a evaluation dataset, dubbed as MISATO, from Matterport3D, FlickrLandscape, MegaDepth, and COCO 2014for indoor, outdoor landscape, building, and background inpainting.

Disclaimer

The authors do not own the image copyrights. Please follow the original dataset's license. We appreciate the contributions of Matterport3D, FlickrLandscape, MegaDepth, and COCO 2014.

To use Matterport3D, you must indicate that you agree to the terms of use by signing the Terms of Use agreement form, using your institutional email addresses, and sending it to: matterport3d@googlegroups.com.

Download

The MISATO Dataset is available at Google Drive, Baidu Netdisk.

Structure

Unzip the file, and you will get a folder including:

|-image
  |- 0000.png
  ...
  |- 1999.png
|-mask
  |- 0000.png
  ...
  |- 1999.png

The image-mask pairs are sized 512x512. The numbers 0000-0499 represent outdoor landscapes, 0500-0999 represent indoor scenes, 1000-1499 represent buildings, and 1500-1999 represent backgrounds.

Intended Uses

The data are intended for research purposes to advance the progess of image inpainting.

Citation

If you found the provided dataset useful, please cite our work.

@article{wang2023towards,
  title={Towards Stable and Faithful Inpainting},
  author={Wang, Yikai and Cao, Chenjie and Fu, Yanwei},
  journal={arXiv preprint arXiv:2312.04831},
  year={2023}
}