[Feature Request] Add WMDP Dataset

[ruidazeng]

Tasks

• Benchmark
• Unlearning method
• Evaluation
• Dataset
• None of the above

Feature request

The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning by Li, Nathaniel, et al. Proceedings of the 41st International Conference on Machine Learning (ICML). 2024.

Paper: https://arxiv.org/abs/2403.03218
Site: https://www.wmdp.ai/
GitHub: https://github.com/centerforaisafety/wmdp?tab=readme-ov-file
Hugging Faces: https://huggingface.co/datasets/cais/wmdp

Motivation

It is important to use the WMDP dataset to test unlearning method's effectiveness on potential hazardous knowledge. Note that the WMDP paper also came with the RMU (Representation Misdirection of Unlearning) method, which was raised in Issue #66 and integrated in PR #69.

Another important note: The WMDP dataset itself does not contain hazardous knowledge in the "red area", but only questions related in the "yellow area". The removal of knowledge of the "red area" is inferred from removal of the "yellow area", as claimed by the paper's authors.

[ruidazeng]

Please assign me to this issue. @molereddy

[ruidazeng]

Model Families (Tested by WMDP Original Authors):

• zephyr-7B-beta
• Mixtral-8x7B-Instruct-v0.1
• Yi-34B-Chat

[molereddy]

Done!

[ruidazeng]

Limitation: The dataset bio-forget-corpus is not public, and requires permission to access the dataset, and is not to be distributed beyond the list of collaborators. Therefore, it is likely not possible to be open-sourced.

[molereddy]

Research works reporting results on WMDP have used the Bio Forget corpus, though. E.g. https://arxiv.org/pdf/2410.02760. You should be able to get access to it.

[ruidazeng]

Research works reporting results on WMDP have used the Bio Forget corpus, though. E.g. https://arxiv.org/pdf/2410.02760. You should be able to get access to it.

It is accessible if approved but not open source-able.

[ruidazeng]

Research works reporting results on WMDP have used the Bio Forget corpus, though. E.g. https://arxiv.org/pdf/2410.02760. You should be able to get access to it.

It is accessible if approved but not open source-able.

https://huggingface.co/datasets/cais/wmdp-corpora

[ruidazeng]

Research works reporting results on WMDP have used the Bio Forget corpus, though. E.g. https://arxiv.org/pdf/2410.02760. You should be able to get access to it.

It is accessible if approved but not open source-able.

https://huggingface.co/datasets/cais/wmdp-corpora

From the form: https://docs.google.com/forms/d/e/1FAIpQLSdnQc8Qn0ozSDu3VE8HLoHPvhpukX1t1dIwE5K5rJw9lnOjKw/viewform

[molereddy]

I don't see how this prevents us from supporting WMDP in OpenUnlearning. Users would require to follow the WMDP owners' instructions to get access to the corpus, download it and then use OpenUnlearning's code to perform and evaluate unlearning. What do you think?

This is similar to the Meta Llama models which you need to get permission to access from huggingface.

[ruidazeng]

I don't see how this prevents us from supporting WMDP in OpenUnlearning. Users would require to follow the WMDP owners' instructions to get access to the corpus, download it and then use OpenUnlearning's code to perform and evaluate unlearning. What do you think?

We can include both cyber-forget-corpus and cyber-retain-corpus, for WMDP-Cyber everything is complete.

For WMDP-Bio, the bio-forget-corpus is restricted; for WMDP-Chem, only the Q&A are available, and they have neither the forget-corpus nor the retain-corpus.

This makes sense for Meta Llama, I am not sure how access is granted to users though to know what API key or interfaces the user needs to have to work with this.

[ruidazeng]

Model Families (Tested by WMDP Original Authors):

• zephyr-7B-beta
• Mixtral-8x7B-Instruct-v0.1
• Yi-34B-Chat

I will open another issue in the future that will support more model families (above) and evaluation metrics below:

All Evaluation Metrics (used in WMDP paper)

Forget Performance:

• QA evaluation
• Probing evaluation

Retain Performance:

• MMLU evaluation
• MT-Bench