README.md

FastEdit ⚡🩹

Editing large language models within 10 seconds.

One-Sentence Summary

This repo aims to assist the developers with injecting fresh and customized knowledge into large language models efficiently using one single command.

Supported Models

• GPT-J (6B)
• LLaMA (7B/13B)

Implemented Algorithms

• Rank-One Model Editing (ROME)

Requirements

• Python 3.8+ and PyTorch 1.13.1+
• 🤗Transformers and Datasets
• sentencepiece

Hardware Requirements

Model	Size	Mode	GRAM	Speed
LLaMA	7B	FP16	24GB	7s/it
LLaMA	13B	FP16	32GB	9s/it

Getting Started

Data Preparation

Please refer to data folder for checking the details about the format of dataset files.

Dependence Installation

git clone https://github.com/hiyouga/FastEdit.git
conda create -n fastedit python=3.10
conda activate fastedit
cd FastEdit
pip install -r requirements.txt

Model Editing

CUDA_VISIBLE_DEVICES=0 python fastedit/editor.py \
    --data data/example.json \
    --model EleutherAI/gpt-j-6b \
    --config hparams/gpt-j-6b.json \
    --template default

Editing LLMs: A Case

We use the samples in data/example.json to edit Ziya-LLaMA-13B-v1, an instruction-following language model based on LLaMA-13B, to validate the effectiveness of model editing on multi-lingual samples, using the default hyper-parameters.

Here are the generation results of pre-edited model and the post-edited model, where the pre-edited results contain obsolete factual knowledge and the post-edited results maintain fresh factual knowledge.

// pre-edit
The prime minister of the United Kingdom is Boris Johnson.
// post-edit
The prime minister of the United Kingdom is Rishi Sunak.

// pre-edit
The name of prime minister of the UK is Boris Johnson.
// post-edit
The name of prime minister of the UK is Rishi Sunak.

// pre-edit
日本的首相叫作现任日本首相是菅义伟（Suga Yoshihide）。
// post-edit
日本的首相叫作岸田文雄。

// pre-edit
日本首相名字是现任日本首相的名字是菅义伟（Suga Yoshihide）。
// post-edit
日本首相名字是岸田文雄

You can run the following command to reproduce above results.

CUDA_VISIBLE_DEVICES=0 python fastedit/editor.py \
    --data data/example.json \
    --model path_to_your_ziya_13b_model \
    --config hparams/llama-13b.json \
    --template ziya

License

This repository is licensed under the Apache-2.0 License.

Citation

If this work is helpful, please kindly cite as:

@Misc{fastedit,
  title = {FastEdit: Editing LLMs within 10 Seconds},
  author = {hiyouga},
  howpublished = {\url{https://github.com/hiyouga/FastEdit}},
  year = {2023}
}

Acknowledgement

The current codebase of this repo largely benefits from Meng et al.'s ROME implementation. Thanks for their wonderful works.

Star History