Add ELECTRA model support (#46)

[azrabano23]

Summary

Adds intervention support for ELECTRA (google/electra-*), addressing the BERT-family item in #46 ([P1] Support more huggingface models — "RoBERTa, DeBERTa, ELECTRA"). RoBERTa is already in progress (#237), so this covers ELECTRA.

ELECTRA uses the standard BERT encoder layout, so the intervention mapping mirrors the existing ESM/RoBERTa encoders. Supported model classes:

• ElectraModel
• ElectraForMaskedLM
• ElectraForSequenceClassification

What's included

• pyvene/models/electra/modelings_intervenable_electra.py — module/dimension mappings for all standard streams (block, mlp in/activation/output, attention in/out/value, and per-head query/key/value with split_head_and_permute), plus create_electra / create_electra_mlm / create_electra_classifier helpers.
• Registration of the three classes in pyvene/models/intervenable_modelcard.py (module + dimension mappings).
• tests/integration_tests/InterventionWithElectraTestCase.py.

Testing

Run on a tiny in-memory config (no downloads), mirroring the other model test cases:

python -m unittest tests.integration_tests.InterventionWithElectraTestCase -v
# Ran 3 tests ... OK

• test_clean_run_positive — a no-op VanillaIntervention reproduces the raw forward pass across all 11 non-head streams.
• test_with_position_intervention_positive — copying a source activation changes the output at each non-head stream (confirms each anchor hooks a real module).
• test_with_head_position_intervention_positive — per-head query/key/value/attention-value streams build and run.

The existing suite (e.g. InterventionWithMLPTestCase) still passes, confirming the modelcard change is non-breaking.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4c9b6fc71c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Register the discriminator pretraining head

For the standard google/electra-*-discriminator checkpoints, users commonly load ElectraForPreTraining / AutoModelForPreTraining, but this registration only covers the base, masked-LM, and sequence-classification classes. Passing an ElectraForPreTraining instance to IntervenableModel with a normal component like block_output will not find its type in type_to_dimension_mapping and fails before hooks are installed, even though the encoder path is the same electra.encoder... prefix as these head models. Please add the matching module and dimension mappings for ElectraForPreTraining as well.

Useful? React with 👍 / 👎.