(ICML 2025) Adversarial Robustness via Deformable Convolution with Stochasticity

A random adversarial defense method based on stochastic deformable convolution

⭐ Our paper is available at https://openreview.net/pdf?id=vISiVCssVg.

1️⃣ Environment

• python: 3.9.12
• pyTorch: 2.2.0
• torchVision: 0.17.0
• cuda: 12.1
• torchattacks: 3.4.0
• tqdm: 4.65.0
• wandb: 0.17.4
• numpy: 1.24.3
• scipy: 1.9.1

2️⃣ Main Training

View all available options:

python train.py -h

usage: train.py [-h] [--batch_size BATCH_SIZE] [--data_dir DATA_DIR] [--dataset {cifar10,cifar100,tiny-imagenet}] [--epochs EPOCHS] [--network {ResNet18,WideResNet34,ResNet50,Vit}]
                [--worker WORKER] [--lr_schedule {cyclic,multistep,cosine}] [--lr_min LR_MIN] [--lr_max LR_MAX] [--weight_decay WEIGHT_DECAY] [--momentum MOMENTUM]
                [--none_random_training] [--rand_deform_training] [--randpos_deform_training] [--randpos_multi_deform_training] [--is_n_repeat] [--reNum RENUM]
                [--only_adv_randpos_training] [--rand_path_training] [--epsilon EPSILON] [--alpha ALPHA] [--seed SEED] [--attack_iters ATTACK_ITERS] [--restarts RESTARTS]
                [--none_adv_training] [--save_dir SAVE_DIR] [--pretrain PRETRAIN] [--continue_training] [--lb LB] [--pos POS] [--eot] [--hang] [--device DEVICE]

Example: Train a ResNet18 on CIFAR-10 with PGD Gradient-Selective Adversarial Training and DCS:

python train.py --network ResNet18 --dataset cifar10 --batch_size 128  --device 0 --randpos_deform_training --is_n_repeat

3️⃣ Main Evaluation

Evaluate a trained model under multiple types of attacks (ResNet18 on CIFAR-10):

python evaluate_nonrepeated.py --network ResNet18 --dataset cifar10 --batch_size 128  --device 0 --randpos_deform_training --pretrain [path_to_model_ckpt]

Evaluate a trained model under eotbpda attack (ResNet18 on CIFAR-10):

python evaluate_nonrepeated_eotbpda.py --network ResNet18 --dataset cifar10 --batch_size 128  --device 0  --randpos_deform_training --pretrain [path_to_model_ckpt]

Notable Arguments

Argument	Description
--randpos_deform_training	Enable DCS
--is_n_repeat	Enable Gradient-Selective Adversarial Training
--save_dir / --pretrain	Checkpoint save/load

4️⃣ DCS on Transformer (Vit)

Training:

Stage one:

python train_vit_pretrain.py --device 0 --optimizer sgd --network earlyVit --dataset cifar10 --lr_max 0.01 --weight_decay 5e-4 --lr_schedule multistep --batch_size 128 --epochs 200 --fix

Stage two:

python train_vit_pretrain_2.py --device 0 --optimizer sgd --network earlyVit --dataset cifar10 --lr_max 0.01 --weight_decay 5e-4 --lr_schedule cosine --batch_size 128 --epochs 200 --vit_pretrained_path [path_to_model_stage_one] --randpos_deform_training 

Evaluation:

BASE(ViT-t+Conv):

python evaluate_vit_repeat_2.py --device 0 --network earlyVit --dataset cifar10 --batch_size 128 --pretrain [path_to_model_base]

DCS(ViT-t+DCS):

python evaluate_vit_repeat_2.py --device 0 --network earlyVit --dataset cifar10 --batch_size 128 --pretrain [path_to_model_dcs] --randpos_deform_training

5️⃣ DCS on R50 (imagenet)

cd NIPS_rand_adv

Training:

python train_rand_mask_imagenet.py --rd

Evaluation:

python train_rand_mask_imagenet.py --rd -e --eval_model_path [path_to_model_dcs]

6️⃣ ⭐ Pretrained Models

Pretrained models are available on Hugging Face 🤗.

7️⃣ Citation

If you find this work helpful, please consider citing us:

@inproceedings{maadversarial,
  title={Adversarial Robustness via Deformable Convolution with Stochasticity},
  author={Ma, Yanxiang and Huang, Zixuan and Dong, Minjing and You, Shan and Xu, Chang},
  booktitle={Forty-second International Conference on Machine Learning}
}