Chemistry Interpretability Pilots — Gemma-2-2B × Neuronpedia

Three small, self-contained pilots asking whether a general language model (Gemma-2-2B) already forms interpretable, causally-relevant chemistry features — and whether the Neuronpedia hosted-API workflow is solid enough to later port onto specialised chemical models (MolGPT / ChemBERTa / MolFormer).

This is de-risking + preliminary evidence for a predoctoral proposal (Predoctorales DGA 2026–2030) on mechanistic interpretability of chemical and materials models. Everything here runs against the hosted Neuronpedia API — no local GPU.

The three questions

Pilot	Question	Method	Claim it can support
1	Do chemistry-meaningful SAE features exist?	SAE feature + activation search	correlational — "a feature that tracks concept X" (hypothesis)
2	Are they causal / load-bearing?	steering strength-sweep vs. random-feature controls	causal — steering moves the output
3	What mechanism answers a chemical fact?	attribution graph (circuit tracing) + node ablation	one annotated circuit + one load-bearing node

The pilots are deliberately staged correlational → causal: Pilot 1 may only produce hypotheses; a feature is not said to "encode" a concept until Pilot 2 or Pilot 3 intervenes on it and the output moves.

Golden rules (enforced in this repo)

1. Every reported number is regenerated by committed, runnable code (scripts/*.py, mirrored by notebooks/*.ipynb — both call the same npchem library, so there is one source of truth).
2. Every correlational claim stays a hypothesis until causally tested.
3. Findings are logged next to the exact script path + git commit that produced them (experiments/log.md, findings/*.md).
4. Small n. It is stated explicitly everywhere and never generalised.
5. The Neuronpedia client API was verified against the installed version before any analysis code was written — see findings/00_api_reference.md.
6. Fixed seeds for anything stochastic, recorded in the output files.

Structure

npchem/                     # the library — one source of truth for all logic
├── probes.py               #   concepts + probe/control text sets (Pilot 1)
├── client.py               #   auth + Neuronpedia client bootstrap
├── features.py             #   semantic search, activation search, specificity test
└── steering.py             #   steering sweep + outcome metrics (Pilot 2)

scripts/                    # thin, headless entry points (canonical regenerators)
├── 00_verify_api.py        #   introspect the installed client (golden rule #5)
├── 01_feature_discovery.py #   -> data/pilot1_features.csv
├── 02_steering.py          #   -> data/pilot2_steering.csv + figures/
└── 03_circuit_tracing.py   #   -> graphs/*.json + Neuronpedia URLs

notebooks/                  # annotated walkthroughs (import npchem, same results)
├── 01_feature_discovery.ipynb
├── 02_steering.ipynb
└── 03_circuit_tracing.ipynb

data/       pilot1_features.csv, pilot2_steering.csv     # committed deliverables
figures/    pilot2_dose_response.png, pilot3_*.png       # incl. Pilot 3 ablation shots
graphs/     chem-*.json                                  # Pilot 3 attribution-graph metadata
docs/       SAE + transcoder/circuit-tracing explainers, pilot3 ablation walkthrough
findings/   00_api_reference.md, 01.md, 02.md, 03.md, report.md, summary.md
experiments/log.md          # dated lab notebook

Setup

python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
pip install -e .                       # puts `npchem` on the path everywhere

cp .env.example .env                   # then paste your Neuronpedia API key
python scripts/00_verify_api.py        # sanity-check auth + client surface

Get a free API key at https://www.neuronpedia.org (account settings). Pilots 1–2 need nothing beyond the hosted API. Pilot 3 uses the same API to generate attribution graphs server-side; the optional local circuit-tracer path is out of scope here.

Results

Full narrative (intro + steps + results) in findings/report.md; findings/summary.md is the proposal one-pager.

• Pilot 1 — features exist? ✅ yes, with nuance. 11/24 candidates are genuine chemistry features (specific vs. matched controls and chemistry-explained), covering 6/8 concepts, all at layer 20 — cleanly for elements/compound classes (gold, iron, carbon compounds, "hydro-", chlorophyll), but not for the carboxylic_acid/amine functional groups. findings/01.md.
• Pilot 2 — features causal? 🔴 clean, pre-registered null. Steering induces zero chemistry across all 90 generations; it is potent but non-specific (random controls perturb the text as much) — a detector-vs-generator dissociation: features that read chemistry don't overtly write it. See findings/02.md (confirmatory sweep + dose-response figure).
• Pilot 3 — mechanism? ✅ circuit mapped + load-bearing node confirmed. For "…gold is → Au", a faithful graph-node ablation (Neuronpedia Circuit Tracer UI) raises P("Au") 0.58 → 0.76 by suppressing a "Gold" feature at the "is" token. The top positive driver is redundant (clean ablation doesn't move "Au") — a cautionary steering ≠ ablation result. See findings/03.md.

References

• Neuronpedia — hosted SAEs, feature dashboards, steering, circuit tracer https://www.neuronpedia.org · docs https://docs.neuronpedia.org
• Lieberum et al. (2024) — Gemma Scope: Open SAEs on Gemma 2
• Ameisen et al. (2025) — Attribution Graphs (circuit tracing methodology)
• Sister project — negation-circuits-gemma (same hosted-API workflow, applied to negation) — the client/auth scaffolding here is reused from it.

Author

Pablo Cabriada Sierra — ML Engineer, INMA