Decoupled attention network for text recognition

This is a pytorch-based implementation for paper Decoupled attention network for text recognition (AAAI-2020).

Requirements

We recommend you to use Anaconda to manage your libraries.

• Python 2.7 (The data augmentation toolkit does not support python3)
• PyTorch (We have tested 0.4.1 and 1.1.0)
• TorchVision
• OpenCV
• PIL (Pillow)
• Colour
• LMDB
• editdistance

Or use pip to install the libraries. (Maybe the torch is different from the anaconda version. Please check carefully and fix the warnings in training stage if necessary.)

    pip install -r requirements.txt

Besides, a data augmentation toolkit is used for handwritten text recognition.

Updates

Nov 28, 2020

Thanks to huizhang0110, we find a bug which results in higher performance on IAM dataset.

The result on IAM dataset should be corrected as (CER 7.0, WER 20.6).

Conclusions are not affected.

Dec 30, 2019

Trained models:

Google Drive

Baidu Netdisk password: sds8

The handwritten models are well trained (IAM-CER 6.4, IAM-WER 19.6). The scene models are single-directional and nearly well trained (IIIT5K 93.3).

Data Preparation

Offline handwritten text

Here we provide the codes for IAM dataset. For RIMES, please prepare it by yourself.

IAM database can be downloaded from here.

For convenience, we provide the processed annotations (in data/IAM/) and the dataloader (dataset_hw.py). You only need to download data/lines and data/words then put the unzipped files into the folder data/IAM/.

Note that all data is loaded into memory at once. Make sure there is enough memory (about 5 GB).

Scene text

Please convert your own dataset to LMDB format by using the tool (run in Python 2.7) provided by @Baoguang Shi.

You can also download the training (NIPS 2014, CVPR 2016) and testing datasets prepared by us.

• BaiduCloud (about 20G training datasets and testing datasets in LMDB format), password: l8em
• Google Drive (testing datasets in LMDB format)
• OneDrive (testing datasets in LMDB format)

The raw pictures of testing datasets can be found here.

Training and Testing

Modify the path in configuration files (cfgs_scene.py for scene, cfgs_hw.py for handwritten). Make sure the import is correct in line 12, main.py. Then:

	python main.py

Citation

@InProceedings{DAN_aaai20,
  author = {Tianwei Wang and Yuanzhi Zhu and Lianwen Jin and Canjie Luo and Xiaoxue Chen and Yaqiang Wu and Qianying Wang and Mingxiang Cai}, 
  title = {Decoupled attention network for text recognition}, 
  booktitle ={AAAI Conference on Artificial Intelligence}, 
  year = {2020}
}

Attention

The project is only free for academic research purposes.