Rank-One Model Editing for Amnesia (ROMnesia)

This repository provides a modification of the original Rank-One Model Editing (ROME) on auto-regressive transformers (GPU-only). The original respository is available here. It has been forked for the purpose of this project and modifications have been made on the main branch. The original ROME version (at the time of the fork) can be found under the original_rome branch.

Table of Contents

1. Installation
2. Rank-One Model Editing for Amnesia (ROMnesia)
3. How to Cite

Installation

We recommend conda for managing Python, CUDA, and PyTorch-related dependencies, and pip for everything else. To get started, simply install conda and run:

./scripts/setup_conda.sh

Rank-One Model Editing for Amnesia (ROMnesia)

This Colab notebook demonstrates ROMnesia. The API is simple; one simply has to specify a requested rewrite of the following form:

request = [
    {
        "prompt": "{} is the capital of",
        "subject": "Paris",
        "target_true": {"str": "France"},
    }
]

Note on Cross-Platform Compatibility

We currently only support methods that edit autoregressive HuggingFace models using the PyTorch backend. We are working on a set of general-purpose methods (usable on e.g. TensorFlow and without HuggingFace) that will be released soon.

How to Cite

The origianl authors of ROME.

@article{meng2022locating,
  title={Locating and Editing Factual Associations in {GPT}},
  author={Kevin Meng and David Bau and Alex Andonian and Yonatan Belinkov},
  journal={Advances in Neural Information Processing Systems},
  volume={35},
  year={2022}
}