Prerequisites

conda create -n tta python=3.8.1
conda activate tta
conda install -y ipython pip

# install the required packages
pip install -r requirements.txt 

Preparation

Datasets

To run one of the following benchmark tests, you need to download the corresponding dataset.

• CIFAR100 → CIFAR100-C: CIFAR100-C dataset is automatically downloaded when running the experiments or manually download from here 🔗.
• ImageNet → ImageNet-C: Download ImageNet-C 🔗 dataset from here 🔗.
• ImageNet → ImageNet-3DCC: Download ImageNet-3DCC 🔗 dataset from here 🔗.

For non-source-free methods (like RMT, etc.), you need to download the ImageNet 🔗 dataset.

Models

For the TTA benchmarks, pre-trained models from RobustBench, Torchvision, and Timm are used.

Run Experiments

Python scripts are provided to run the experiments. For example, to run the IMAGNET → IMAGNET-C with OURS, run the following command:

python CTTA.py -acfg configs/adapter/cifar100/OURS.yaml -dcfg configs/dataset/cifar100.yaml -ocfg configs/order/cifar100/0.yaml SEED 0

Bash scripts are provided to run the experiments. For example, run the following command:

nohup bash run.sh > run.log 2>&1 &

Competitors

The repository currently supports the following methods: TEA, RMT, BN, Tent, CoTTA, SAR, RoTTA, TRIBE

Acknowledgements

This project is based on the following projects:

• Robustbench official
• CoTTA official
• Tent official
• SAR official
• RoTTA official
• RMT official
• TRIBE official
• TEA official

Contact

If you have any questions about our work, please contact im@xhy.im