Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Nystromformer #14659

Merged
merged 62 commits into from
Jan 11, 2022
Merged
Changes from 1 commit
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
4ea35ec
Initial commit
novice03 Dec 7, 2021
c3161cf
Config and modelling changes
novice03 Dec 7, 2021
c585c49
Modelling and test changes
novice03 Dec 7, 2021
b2d4d43
Code quality fixes
novice03 Dec 7, 2021
226069d
Modeling changes and conversion script
novice03 Dec 10, 2021
b4058a0
Minor modeling changes and conversion script
novice03 Dec 30, 2021
a890a24
Modeling changes
novice03 Jan 1, 2022
99b56a5
Correct modeling, add tests and documentation
novice03 Jan 2, 2022
bc4cbec
Code refactor
novice03 Jan 2, 2022
fcab712
Remove tokenizers
novice03 Jan 2, 2022
f39df6c
Merge branch 'add_nystromformer' of https://github.com/novice03/trans…
novice03 Jan 2, 2022
28db895
Code refactor
novice03 Jan 2, 2022
1e5c17d
Update __init__.py
novice03 Jan 2, 2022
6670757
Fix bugs
novice03 Jan 2, 2022
519329f
Update src/transformers/__init__.py
novice03 Jan 2, 2022
e3579da
Update src/transformers/__init__.py
novice03 Jan 2, 2022
d7444f1
Update src/transformers/models/nystromformer/__init__.py
novice03 Jan 2, 2022
8740d6d
Update docs/source/model_doc/nystromformer.mdx
novice03 Jan 2, 2022
4438125
Update src/transformers/models/nystromformer/configuration_nystromfor…
novice03 Jan 2, 2022
577511e
Update src/transformers/models/nystromformer/configuration_nystromfor…
novice03 Jan 2, 2022
b13c5dd
Update src/transformers/models/nystromformer/configuration_nystromfor…
novice03 Jan 2, 2022
bd0da9e
Update src/transformers/models/nystromformer/configuration_nystromfor…
novice03 Jan 2, 2022
51b2a93
Update src/transformers/models/nystromformer/convert_nystromformer_or…
novice03 Jan 2, 2022
a4efb44
Update src/transformers/models/nystromformer/configuration_nystromfor…
novice03 Jan 2, 2022
eac563c
Update modeling and test_modeling
novice03 Jan 2, 2022
163702c
Code refactor
novice03 Jan 2, 2022
ceb83f2
Merge branch 'master' into add_nystromformer
novice03 Jan 2, 2022
089945f
.rst to .mdx
novice03 Jan 2, 2022
091c7ad
doc changes
novice03 Jan 2, 2022
2553d19
Doc changes
novice03 Jan 2, 2022
0129631
Update modeling_nystromformer.py
novice03 Jan 2, 2022
9c74356
Doc changes
novice03 Jan 2, 2022
fd9f168
Fix copies
novice03 Jan 2, 2022
4332b82
Apply suggestions from code review
novice03 Jan 2, 2022
016cb9d
Apply suggestions from code review
novice03 Jan 2, 2022
d208c78
Update configuration_nystromformer.py
novice03 Jan 2, 2022
b394eea
Fix copies
novice03 Jan 2, 2022
05d2d9b
Update tests/test_modeling_nystromformer.py
novice03 Jan 3, 2022
9d648fa
Update test_modeling_nystromformer.py
novice03 Jan 3, 2022
552cc12
Merge branch 'master' into add_nystromformer
novice03 Jan 3, 2022
ca3e934
Merge branch 'huggingface:master' into add_nystromformer
novice03 Jan 5, 2022
c362660
Merge branch 'huggingface:master' into add_nystromformer
novice03 Jan 10, 2022
875a0bc
Apply suggestions from code review
novice03 Jan 10, 2022
748092a
Fix code style
novice03 Jan 10, 2022
55edee2
Update modeling_nystromformer.py
novice03 Jan 10, 2022
6fc762d
Update modeling_nystromformer.py
novice03 Jan 10, 2022
c6a8d35
Fix code style
novice03 Jan 10, 2022
3e45dba
Merge branch 'huggingface:master' into add_nystromformer
novice03 Jan 10, 2022
f971632
Reformat modeling file
novice03 Jan 10, 2022
e891e05
Merge branch 'huggingface:master' into add_nystromformer
novice03 Jan 10, 2022
b678f5c
Update modeling_nystromformer.py
novice03 Jan 10, 2022
7268ec3
Merge branch 'huggingface:master' into add_nystromformer
novice03 Jan 10, 2022
02fe323
Modify NystromformerForMultipleChoice
novice03 Jan 10, 2022
b4770d7
Merge branch 'add_nystromformer' of https://github.com/novice03/trans…
novice03 Jan 10, 2022
5f3a389
Fix code quality
novice03 Jan 10, 2022
e0e4be6
Apply suggestions from code review
novice03 Jan 10, 2022
55cbd32
Code style changes and torch.no_grad()
novice03 Jan 10, 2022
5a0b891
Merge branch 'huggingface:master' into add_nystromformer
novice03 Jan 10, 2022
7e9e50a
make style
novice03 Jan 10, 2022
7da8e1c
Merge branch 'huggingface:master' into add_nystromformer
novice03 Jan 10, 2022
bd77cfe
Merge branch 'add_nystromformer' of https://github.com/novice03/trans…
novice03 Jan 10, 2022
9a5663b
Apply suggestions from code review
novice03 Jan 11, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update src/transformers/models/nystromformer/configuration_nystromfor…
…mer.py

Co-authored-by: NielsRogge <48327001+NielsRogge@users.noreply.github.com>
  • Loading branch information
novice03 and NielsRogge authored Jan 2, 2022
commit b13c5dd8be7e9b15f2ce966071ca88ce25134d81
Original file line number Diff line number Diff line change
Expand Up @@ -76,9 +76,6 @@ class NystromformerConfig(PretrainedConfig):
The standard deviation of the truncated_normal_initializer for initializing all weight matrices.
layer_norm_eps (:obj:`float`, `optional`, defaults to 1e-12):
The epsilon used by the layer normalization layers.
use_cache (:obj:`bool`, `optional`, defaults to :obj:`True`):
Whether or not the model should return the last key/values attentions (not used by all models). Only
relevant if ``config.is_decoder=True``.

Example::

Expand Down