Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SynthID (watermerking by Google DeepMind) #34350

Merged
merged 52 commits into from
Oct 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
ce6d213
Add SynthIDTextWatermarkLogitsProcessor
sumedhghaisas2 Aug 14, 2024
19b599b
esolving comments.
sumedhghaisas2 Aug 23, 2024
22fcc5b
Resolving comments.
sumedhghaisas2 Aug 23, 2024
4251d6e
esolving commits,
sumedhghaisas2 Aug 23, 2024
9b7b7b1
Improving SynthIDWatermark tests.
sumedhghaisas2 Aug 30, 2024
b65e3fe
switch to PT version
SunMarc Oct 2, 2024
14ad3bb
Merge pull request #2 from sumedhghaisas2/add-pt-version-detector
sumedhghaisas2 Oct 7, 2024
e03312c
detector as pretrained model + style
zucchini-nlp Oct 10, 2024
fc6fcd2
update training + style
SunMarc Oct 14, 2024
b62edf8
rebase
gante Oct 18, 2024
efeadb5
Merge branch 'main' into rebase
gante Oct 18, 2024
bd8d720
Merge pull request #4 from sumedhghaisas2/rebase
gante Oct 18, 2024
7da4040
Update logits_process.py
gante Oct 18, 2024
edf25d4
Improving SynthIDWatermark tests.
sumedhghaisas2 Aug 30, 2024
a0f5fc2
Shift detector training to wikitext negatives and stabilize with lowe…
sumedhghaisas2 Oct 16, 2024
7b23cd7
Clean up.
sumedhghaisas2 Oct 16, 2024
a7ae44a
in for 7B
sumedhghaisas2 Oct 18, 2024
a45dd5a
cleanup
sumedhghaisas2 Oct 20, 2024
996066c
upport python 3.8.
sumedhghaisas2 Oct 21, 2024
9d4f177
README and final cleanup.
sumedhghaisas2 Oct 21, 2024
9b8805f
HF Hub upload and initiaze.
sumedhghaisas2 Oct 21, 2024
1a263a5
Update requirements for synthid_text.
sumedhghaisas2 Oct 22, 2024
ccea5dd
Adding SynthIDTextWatermarkDetector.
sumedhghaisas2 Oct 22, 2024
fb0f770
Detector testing.
sumedhghaisas2 Oct 22, 2024
9d1e1ee
Documentation changes.
sumedhghaisas2 Oct 22, 2024
7fbbaa9
Copyrights fix.
sumedhghaisas2 Oct 23, 2024
78ca99f
Fix detector api.
sumedhghaisas2 Oct 23, 2024
3409dbd
ironing out errors
gante Oct 23, 2024
cc254b1
ironing out errors
gante Oct 23, 2024
97dcb98
training checks
gante Oct 23, 2024
76fc84a
Merge pull request #3 from sumedhghaisas2/synthid_text
gante Oct 23, 2024
a6d9a00
make fixup and make fix-copies
gante Oct 23, 2024
8fea35d
docstrings and add to docs
gante Oct 23, 2024
ee01b7a
copyright
gante Oct 23, 2024
fd1f53f
BC
gante Oct 23, 2024
98b1963
test docstrings
gante Oct 23, 2024
021c685
move import
gante Oct 23, 2024
34bd287
protect type hints
gante Oct 23, 2024
adb3d01
top level imports
gante Oct 23, 2024
9a7abae
watermarking example
gante Oct 23, 2024
62a0b4e
direct imports
gante Oct 23, 2024
7aec58b
tpr fpr meaning
gante Oct 23, 2024
70b9dfb
process_kwargs
gante Oct 23, 2024
ef4e461
SynthIDTextWatermarkingConfig docstring
gante Oct 23, 2024
058bfa1
assert -> exception
gante Oct 23, 2024
2ac3193
example updates
gante Oct 23, 2024
26370a4
no immutable dict (cant be serialized)
gante Oct 23, 2024
ba28b14
pack fn
gante Oct 23, 2024
178607d
einsum equivalent
gante Oct 23, 2024
04c3f1e
import order
gante Oct 23, 2024
4a6e8df
fix test on gpu
gante Oct 23, 2024
613aae2
add detector example
gante Oct 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions docs/source/en/internal/generation_utils.md
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,9 @@ generation.
[[autodoc]] SuppressTokensLogitsProcessor
- __call__

[[autodoc]] SynthIDTextWatermarkLogitsProcessor
- __call__

[[autodoc]] TemperatureLogitsWarper
- __call__

Expand Down Expand Up @@ -418,5 +421,20 @@ A [`Constraint`] can be used to force the generation to include specific tokens

## Watermark Utils

[[autodoc]] WatermarkingConfig
- __call__

[[autodoc]] WatermarkDetector
- __call__

[[autodoc]] BayesianDetectorConfig
- __call__

[[autodoc]] BayesianDetectorModel
- __call__

[[autodoc]] SynthIDTextWatermarkingConfig
- __call__

[[autodoc]] SynthIDTextWatermarkDetector
- __call__
2 changes: 0 additions & 2 deletions docs/source/en/main_classes/text_generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,6 @@ like token streaming.
- validate
- get_generation_mode

[[autodoc]] generation.WatermarkingConfig

## GenerationMixin

[[autodoc]] GenerationMixin
Expand Down
34 changes: 34 additions & 0 deletions examples/research_projects/synthid_text/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# SynthID Text

This project showcases the use of SynthIDText for watermarking LLMs. The code shown in this repo also
demostrates the training of the detector for detecting such watermarked text. This detector can be uploaded onto
a private HF hub repo (private for security reasons) and can be initialized again through pretrained model loading also shown in this script.

See our blog post: https://huggingface.co/blog/synthid-text


## Python version

User would need python 3.9 to run this example.

## Installation and running

Once you install transformers you would need to install requirements for this project through requirements.txt provided in this folder.

```
pip install -r requirements.txt
```

## To run the detector training

```
python detector_training.py --model_name=google/gemma-7b-it
```

Check the script for more parameters are are tunable and check out paper at link
https://www.nature.com/articles/s41586-024-08025-4 for more information on these parameters.

## Caveat

Make sure to run the training of the detector and the detection on the same hardware
CPU, GPU or TPU to get consistent results (we use detecterministic randomness which is hardware dependent).
Loading
Loading