Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add OLMo model family #29890

Merged
merged 71 commits into from
Apr 17, 2024
Merged

Add OLMo model family #29890

Changes from 1 commit
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
0afbd4a
Add OLMo using add-new-model-like with Llama
2015aroras Mar 14, 2024
9842c5c
Fix incorrect tokenizer for OLMo
2015aroras Mar 14, 2024
fc21050
Copy-paste relevant OLMo methods and their imports
2015aroras Mar 15, 2024
3a748b9
Add OLMo config
2015aroras Mar 15, 2024
e15ed3c
Modify OLMo config to follow HF conventions
2015aroras Mar 26, 2024
0090b1a
Remove unneeded Llama code from OLMo model
2015aroras Mar 26, 2024
8719408
Add ability for OLMo model to output attentions
2015aroras Mar 26, 2024
8b1f082
Add OLMoPreTrainedModel and OLMoModel
2015aroras Mar 26, 2024
49bb675
Add OLMoForCausalLM
2015aroras Mar 26, 2024
32ec380
Minor fixes to OLMo model for style and missing functions
2015aroras Mar 26, 2024
cba91d4
Implement OLMo tokenizer
2015aroras Mar 26, 2024
7cc2a9a
Implement OLMo to HF conversion script
2015aroras Mar 26, 2024
a1f9294
Add tests for OLMo model
2015aroras Mar 26, 2024
f4fa804
Add tests for OLMo fast tokenizer
2015aroras Mar 26, 2024
7b287ca
Add auto-generated dummy objects
2015aroras Mar 26, 2024
d1ba9c7
Remove unimplemented OLMo classes from auto and init classes and re-f…
2015aroras Mar 26, 2024
159b124
Add README and associated auto-generated files
2015aroras Mar 26, 2024
8d8baf5
Use OLMo names for common properties
2015aroras Mar 27, 2024
42492b9
Run make fixup
2015aroras Mar 27, 2024
a92d52a
Remove `|` from OLMo typing
2015aroras Mar 27, 2024
f59ea67
Remove unneeded tokenization_olmo.py
2015aroras Mar 27, 2024
4bc9b6b
Revert model, config and converter to add-new-model-like Llama
2015aroras Mar 30, 2024
4fdd1da
Move logic for adding bos/eos token into GPTNeoxTokenizerFast
2015aroras Mar 30, 2024
3e8ceb1
Change OLMoConfig defaults to match OLMo-7B
2015aroras Mar 30, 2024
f68a915
Use GPTNeoXToknizerFast in OLMo tokenizer tests
2015aroras Mar 30, 2024
43eaeff
Modify auto-generated OLMoModelTests to work for OLMo
2015aroras Mar 30, 2024
6dd1852
Add non-parametric layer norm OLMoLayerNorm
2015aroras Mar 30, 2024
7b438cd
Update weight conversion script for OLMo
2015aroras Mar 30, 2024
3346195
Fix __init__ and auto structure for OLMo
2015aroras Mar 30, 2024
7ba083f
Fix errors from make fixup
2015aroras Mar 30, 2024
47affa6
Remove OLMoTokenizerFast from documentation
2015aroras Mar 30, 2024
b468a0a
Merge remote-tracking branch 'upstream/main' into add-olmo-implementa…
2015aroras Apr 5, 2024
98db6a9
Add missing 'Copied from' for OLMoModel._update_causal_mask
2015aroras Apr 5, 2024
dd0f76c
Run make fix-copies
2015aroras Apr 5, 2024
0340778
Rearrange string replacements in OLMoForCausalLM Copied from
2015aroras Apr 5, 2024
edbc4d1
Move OLMo and Llama CausalLM.forward example into global constants
2015aroras Apr 5, 2024
75f5cee
Fix OLMO_GENERATION_EXAMPLE doc string typo
2015aroras Apr 6, 2024
94b31e1
Add option for qkv clipping to OLMo
2015aroras Apr 10, 2024
3805b97
Rearrange OLMoConfig kwargs in convert_olmo_weights_to_hf
2015aroras Apr 10, 2024
a555819
Add clip_qkv to OLMoConfig in convert_olmo_weights_to_hf
2015aroras Apr 10, 2024
aea6b06
Fix OLMo tokenization bug using conversion script
2015aroras Apr 12, 2024
0bbb55a
Keep model in full precision after conversion
2015aroras Apr 12, 2024
21faf45
Do not add eos token automatically
2015aroras Apr 12, 2024
ee00583
Update references to OLMo model in HF Hub
2015aroras Apr 12, 2024
63a6fbd
Do not add eos token during encoding by default
2015aroras Apr 12, 2024
76c7970
Fix Llama generation example
2015aroras Apr 12, 2024
450f038
Run make fixup
2015aroras Apr 12, 2024
4df56a4
OLMo 7B integration test fix
2015aroras Apr 13, 2024
faa188f
Merge branch 'main' into add-olmo-implementation
2015aroras Apr 13, 2024
0b227bb
Remove unneeded special case for OLMoConfig
2015aroras Apr 13, 2024
9ff65a4
OLMo 7B Twin 2T integration test fix
2015aroras Apr 13, 2024
70f593d
Fix test_model_7b_greedy_generation
2015aroras Apr 13, 2024
54f4aa5
Remove test_compile_static_cache
2015aroras Apr 13, 2024
33d3be7
Fix OLMo and Llama generation example
2015aroras Apr 13, 2024
eab18b6
Run make fixup
2015aroras Apr 13, 2024
628f9d5
Revert "OLMo 7B integration test fix"
2015aroras Apr 15, 2024
751195f
Revert "OLMo 7B Twin 2T integration test fix"
2015aroras Apr 15, 2024
68466dd
Ungate 7B integration tests and fix greedy generation test
2015aroras Apr 15, 2024
91f163f
Add retries for flaky test_eager_matches_sdpa_generate
2015aroras Apr 15, 2024
aa4a92f
Fix output of doc example for OLMoForCausalLM.forward
2015aroras Apr 15, 2024
43ea80e
Downsize OLMo doc test for OLMoForCausalLM.forward to 1B model
2015aroras Apr 15, 2024
fcc457c
Try fix incorrect characters in OLMoForCausalLM.forward doct test
2015aroras Apr 15, 2024
e5fdf50
Try fix incorrect characters in OLMoForCausalLM.forward doc test usin…
2015aroras Apr 15, 2024
4f062d7
Remove pretraining_tp from OLMo config and model
2015aroras Apr 16, 2024
3c60cf3
Add missing 'Copied from' instances
2015aroras Apr 16, 2024
a037fca
Remove unneeded causal_mask from OLMoModel
2015aroras Apr 16, 2024
004f80e
Revert Llama changes
2015aroras Apr 16, 2024
826c91b
Ignore copy for OLMoForCausalLM.forward
2015aroras Apr 16, 2024
c7f4e74
Change 'OLMo' to 'Olmo' in classes
2015aroras Apr 16, 2024
ec344c3
Move minimal OLMo tokenization tests to model tests
2015aroras Apr 16, 2024
78148fe
Add missed 'Copied from' for repeat_kv
2015aroras Apr 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix incorrect tokenizer for OLMo
  • Loading branch information
@2015aroras
2015aroras committed Mar 26, 2024
commit 9842c5c5b6a49eb39fde9f189a802003ceef1c49
2 changes: 1 addition & 1 deletion src/transformers/models/auto/tokenization_auto.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -225,7 +225,7 @@
(
"olmo",
(
"LlamaTokenizer" if is_sentencepiece_available() else None,
"OLMoTokenizer" if is_sentencepiece_available() else None,
"OLMoTokenizerFast" if is_tokenizers_available() else None,
),
("llava", ("LlamaTokenizer", "LlamaTokenizerFast" if is_tokenizers_available() else None)),
Expand Down