Any2AnyTryon: Leveraging Adaptive Position Embeddings for Versatile Virtual Clothing Tasks

Environment

conda create -n Any2AnyTryon python=3.11
conda activate Any2AnyTryon
pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124
pip install --upgrade -r requirements.txt

Demo

Using --group_offloading saves memory but slows processing.

python app.py

Inference

Choose a LoRA from here or use --lora_name with the script.

python infer.py --model_image ./asset/images/model/model1.png --garment_image ./asset/images/garment/garment1.jpg \
--prompt="<MODEL> a man <GARMENT> t-shirt with pockets <TARGET> man wearing the t-shirt"

Test

Download the dataset from the VITON-HD. Generate segmentation masks using AutoMasker for repainting. For unpaired data evaluation, execute the following command:

CUDA_VISIBLE_DEVICES=0 python test_vitonhd.py --model_path black-forest-labs/FLUX.1-dev \
--model_dir data/zalando-hd-resized/test/image --garment_dir data/zalando-hd-resized/test/cloth \
--output_dir ./results/vitonhd_test_unpaired_repaint --meta_file data/zalando-hd-resized/test_pairs.txt \
--mask_dir data/zalando-hd-resized/test/mask --source_dir data/zalando-hd-resized/test/image \
--train_double_block_only --repaint 

For paired test, download test set and extract images to local directory.

python src/download_data.py

CUDA_VISIBLE_DEVICES=0 python test_vitonhd.py --model_path black-forest-labs/FLUX.1-dev \
--model_dir data/zalando-hd-resized/test/image_synthesis --garment_dir data/zalando-hd-resized/test/cloth \
--output_dir ./results/vitonhd_test_paired_repaint --meta_file data/zalando-hd-resized/test_pairs.txt \
--mask_dir data/zalando-hd-resized/test/mask --source_dir data/zalando-hd-resized/test/image \
--train_double_block_only --repaint --paired

Datasets

Some data can be downloaded from here. Use the following code to ensure pixel correspondence between the original image and the synthesized image:

from src.utils import crop_to_multiple_of_16
img_src = Image.open(<img_path>)
img_inpaint = Image.open(io.BytesIO(<inpaint_bytes>))

img_src = crop_to_multiple_of_16(img_src)
assert img_src.size==img_inpaint.size

To-Do List

• Demo code and gradio interface
• Inference code
• Tryon checkpoint
• Model generation checkpoint
• Garment reconstruction checkpoint
• Base all tasks checkpoint
• Dataset preparation
• Training code

Citation

@misc{guo2025any2anytryonleveragingadaptiveposition,
    title={Any2AnyTryon: Leveraging Adaptive Position Embeddings for Versatile Virtual Clothing Tasks}, 
    author={Hailong Guo and Bohan Zeng and Yiren Song and Wentao Zhang and Chuang Zhang and Jiaming Liu},
    year={2025},
    eprint={2501.15891},
    archivePrefix={arXiv},
    primaryClass={cs.CV},
    url={https://arxiv.org/abs/2501.15891}, 
}