Skip to content

Commit 6a07800

Browse files
hyesulimjjho-choi
authored and
stes
committed
Initial release of PatchSAE
Co-authored-by: jjho-choi <jjho.choi@gmail.com>
0 parents  commit 6a07800

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

54 files changed

+402999
-0
lines changed

‎.gitignore‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
__pycache__/
2+
data/
3+
.vscode/
4+
out/
5+
.DS_Store
6+
*.zip
7+
wandb/
8+
logs/

‎.pre-commit-config.yaml‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
repos:
2+
- repo: https://github.com/pre-commit/pre-commit-hooks
3+
rev: v5.0.0
4+
hooks:
5+
- id: end-of-file-fixer
6+
- id: trailing-whitespace
7+
- id: check-yaml
8+
- id: check-executables-have-shebangs
9+
- id: check-toml
10+
- id: check-added-large-files
11+
args: [--maxkb=15000]
12+
- id: check-case-conflict
13+
- id: check-merge-conflict
14+
- id: check-symlinks
15+
- id: debug-statements
16+
- id: detect-private-key
17+
- id: mixed-line-ending
18+
args: [--fix=lf]
19+
- id: requirements-txt-fixer
20+
- repo: https://github.com/astral-sh/ruff-pre-commit
21+
rev: v0.11.6
22+
hooks:
23+
- id: ruff
24+
args: [ --fix, --select=I ]
25+
- id: ruff-format

‎LICENSE‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2024-2025 Hyesu Lim, Jinho Choi, Jaegul Choo, Steffen Schneider
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎NOTICE‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
Code from the following third-party repositories was used.
2+
Please include the contents of the LICENSE as well this NOTICE file in all
3+
re-distributions of this code.
4+
5+
=================================================================================
6+
7+
https://github.com/muzairkhattak/multimodal-prompt-learning
8+
9+
MIT License
10+
11+
Copyright (c) 2022 Muhammad Uzair Khattak
12+
Copyright (c) 2021 Kaiyang Zhou
13+
14+
Permission is hereby granted, free of charge, to any person obtaining a copy
15+
of this software and associated documentation files (the "Software"), to deal
16+
in the Software without restriction, including without limitation the rights
17+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
18+
copies of the Software, and to permit persons to whom the Software is
19+
furnished to do so, subject to the following conditions:
20+
21+
The above copyright notice and this permission notice shall be included in all
22+
copies or substantial portions of the Software.
23+
24+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
25+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
26+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
27+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
28+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
29+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
30+
SOFTWARE.
31+
32+
=================================================================================
33+
34+
https://github.com/muzairkhattak/PromptSRC
35+
36+
MIT License
37+
Copyright (c) 2023 Muhammad Uzair Khattak
38+
Copyright (c) 2022 Muhammad Uzair Khattak
39+
Copyright (c) 2021 Kaiyang Zhou
40+
41+
Permission is hereby granted, free of charge, to any person obtaining a copy
42+
of this software and associated documentation files (the "Software"), to deal
43+
in the Software without restriction, including without limitation the rights
44+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
45+
copies of the Software, and to permit persons to whom the Software is
46+
furnished to do so, subject to the following conditions:
47+
48+
The above copyright notice and this permission notice shall be included in all
49+
copies or substantial portions of the Software.
50+
51+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
52+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
53+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
54+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
55+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
56+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
57+
SOFTWARE.
58+
59+
=================================================================================
60+
61+
https://github.com/HugoFry/mats_sae_training_for_ViTs
62+
63+
MIT License
64+
65+
Copyright (c) 2023 Joseph Bloom
66+
67+
Permission is hereby granted, free of charge, to any person obtaining a copy
68+
of this software and associated documentation files (the "Software"), to deal
69+
in the Software without restriction, including without limitation the rights
70+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
71+
copies of the Software, and to permit persons to whom the Software is
72+
furnished to do so, subject to the following conditions:
73+
74+
The above copyright notice and this permission notice shall be included in all
75+
copies or substantial portions of the Software.
76+
77+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
78+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
79+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
80+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
81+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
82+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
83+
SOFTWARE.
84+
85+
=================================================================================
86+
87+
https://github.com/jbloomAus/SAELens
88+
89+
MIT License
90+
91+
Copyright (c) 2023 Joseph Bloom
92+
93+
Permission is hereby granted, free of charge, to any person obtaining a copy
94+
of this software and associated documentation files (the "Software"), to deal
95+
in the Software without restriction, including without limitation the rights
96+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
97+
copies of the Software, and to permit persons to whom the Software is
98+
furnished to do so, subject to the following conditions:
99+
100+
The above copyright notice and this permission notice shall be included in all
101+
copies or substantial portions of the Software.
102+
103+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
104+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
105+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
106+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
107+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
108+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
109+
SOFTWARE.

‎README.md‎

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
# PatchSAE: Sparse Autoencoders Reveal Selective Remapping of Visual Concepts During Adaptation
2+
3+
[![Website & Demo](https://img.shields.io/badge/🔗_Website_&_Demo-blue)](https://dynamical-inference.ai/patchsae/)
4+
[![Paper](https://img.shields.io/badge/📑_Paper-arXiv-red)](https://arxiv.org/abs/2412.05276)
5+
[![OpenReview](https://img.shields.io/badge/OpenReview-ICLR_2025-green)](https://openreview.net/forum?id=imT03YXlG2)
6+
[![Hugging Face Demo](https://img.shields.io/badge/🤗_Hugging_Face-Demo-yellow)](https://huggingface.co/spaces/dynamical-inference/patchsae-demo)
7+
8+
<div align="center">
9+
<img width="800" alt="PatchSAE visualization" src="https://github.com/hyesulim/patchsae/blob/release/assets/sae_arch.gif">
10+
</div>
11+
12+
## 🚀 Quick Navigation
13+
14+
- [Getting Started](#-getting-started)
15+
- [Interactive Demo](#-interactive-demo)
16+
- [Training & Analysis](#-patchsae-training-and-analysis)
17+
- [Status Updates](#-status-updates)
18+
- [License & Credits](#-license--credits)
19+
20+
## 🛠 Getting Started
21+
22+
Set up your environment with these simple steps:
23+
24+
```bash
25+
# Create and activate environment
26+
conda create --name patchsae python=3.12
27+
conda activate patchsae
28+
29+
# Install dependencies
30+
pip install -r requirements.txt
31+
32+
# Always set PYTHONPATH before running any scripts
33+
cd patchsae
34+
PYTHONPATH=./ python src/demo/app.py
35+
```
36+
37+
## 🎮 Interactive Demo
38+
39+
### Online Demo on Hugging Face 🤗 [![Website & Demo](https://img.shields.io/badge/🔗_Website_&_Demo-blue)](https://dynamical-inference.ai/patchsae/)
40+
41+
Explore our pre-computed images and SAE latents without any installation!
42+
> 💡 The demo may experience slowdowns due to network constraints. For optimal performance, consider disabling your VPN if you encounter any delays.
43+
44+
<div align="center">
45+
<img width="390" alt="Demo interface" src="https://github.com/user-attachments/assets/97d3f862-36c9-4a76-80b6-5dbf90a3116a">
46+
</div>
47+
48+
49+
### Local Demo: Try Your Own Images
50+
51+
Want to experiment with your own images? Follow these steps:
52+
53+
#### 1. Setup Local Demo
54+
55+
First, download the necessary files:
56+
57+
- [out.zip](https://drive.google.com/file/d/1NJzF8PriKz_mopBY4l8_44R0FVi2uw2g/edit)
58+
- [data.zip](https://drive.google.com/file/d/1reuDjXsiMkntf1JJPLC5a3CcWuJ6Ji3Z/edit)
59+
60+
You can download the files using `gdown` as follows:
61+
62+
```bash
63+
# Activate environment first (see Getting Started)
64+
65+
# Download necessary files (35MB + 513MB)
66+
gdown --id 1NJzF8PriKz_mopBY4l8_44R0FVi2uw2g # out.zip
67+
gdown --id 1reuDjXsiMkntf1JJPLC5a3CcWuJ6Ji3Z # data.zip
68+
69+
# Extract files
70+
unzip data.zip
71+
unzip out.zip
72+
```
73+
74+
> 💡 Need `gdown`? Install it with: `conda install conda-forge::gdown`
75+
76+
Your folder structure should look like:
77+
78+
```
79+
patchsae/
80+
├── configs/
81+
├── data/ # From data.zip
82+
├── out/ # From out.zip
83+
├── src/
84+
│ └── demo/
85+
│ └── app.py
86+
├── tasks/
87+
├── requirements.txt
88+
└── ... (other files)
89+
```
90+
91+
#### 2. Launch the Demo
92+
93+
```bash
94+
PYTHONPATH=./ python src/demo/app.py
95+
```
96+
97+
⚠️ **Note**:
98+
- First run will download datasets from HuggingFace automatically (About 30GB in total)
99+
- Demo runs on CPU by default
100+
- Access the interface at http://127.0.0.1:7860 (or the URL shown in terminal)
101+
102+
## 📊 PatchSAE Training and Analysis
103+
104+
- **Training Instructions**: See [tasks/README.md](https://github.com/hyesulim/patchsae/blob/release/tasks/README.md)
105+
- **Analysis Notebooks**:
106+
- [demo.ipynb](https://github.com/hyesulim/patchsae/blob/main/demo.ipynb)
107+
- [analysis.ipynb](https://github.com/hyesulim/patchsae/blob/main/analysis/analysis.ipynb)
108+
109+
## 📝 Status Updates
110+
111+
- **Jan 13, 2025**: Training & Analysis code work properly. Minor error in data loading by class when using ImageNet.
112+
- **Jan 09, 2025**: Analysis code works. Updated training with evaluation during training, fixed optimizer bug.
113+
- **Jan 07, 2025**: Added analysis code. Reproducibility tests completed (trained on ImageNet, tested on Oxford-Flowers).
114+
- **Jan 06, 2025**: Training code updated. Reproducibility testing in progress.
115+
- **Jan 02, 2025**: Training code incomplete in this version. Updates coming soon.
116+
117+
## 📜 License & Credits
118+
119+
### Reference Implementations
120+
121+
- [SAE for ViT](https://github.com/HugoFry/mats_sae_training_for_ViTs)
122+
- [SAELens](https://github.com/jbloomAus/SAELens)
123+
- [Differentiable and Fast Geometric Median in NumPy and PyTorch](https://github.com/krishnap25/geom_median)
124+
- [Self-regulating Prompts: Foundational Model Adaptation without Forgetting [ICCV 2023]](https://github.com/muzairkhattak/PromptSRC)
125+
- Used in: `configs/` and `msrc/models/`
126+
- [MaPLe: Multi-modal Prompt Learning CVPR 2023](https://github.com/muzairkhattak/multimodal-prompt-learning)
127+
- Used in: `configs/models/maple/...yaml` and `data/clip/maple/imagenet/model.pth.tar-2`
128+
129+
### License Notice
130+
131+
Our code is distributed under an MIT license, please see the [LICENSE](LICENSE) file for details.
132+
The [NOTICE](NOTICE) file lists license for all third-party code included in this repository.
133+
Please include the contents of the LICENSE and NOTICE files in all re-distributions of this code.
134+
135+
---
136+
137+
### Citation
138+
139+
If you find our code or models useful in your work, please cite our [paper](https://arxiv.org/abs/2412.05276):
140+
141+
```
142+
@inproceedings{
143+
lim2025sparse,
144+
title={Sparse autoencoders reveal selective remapping of visual concepts during adaptation},
145+
author={Hyesu Lim and Jinho Choi and Jaegul Choo and Steffen Schneider},
146+
booktitle={The Thirteenth International Conference on Learning Representations},
147+
year={2025},
148+
url={https://openreview.net/forum?id=imT03YXlG2}
149+
}
150+
```

‎analysis/__init__.py‎

Whitespace-only changes.

0 commit comments

Comments
 (0)