Add magic method to our TF models to convert datasets with column inference#17160

Merged

Rocketknight1 merged 35 commits intomainfrom magic_tf_dataset_method

Jun 6, 2022

+162-1

Commits on May 19, 2022

Add method to call to_tf_dataset() with column inference
Rocketknight1
committed
Add test for dataset creation
Rocketknight1
committed
Add a default arg for data collator
Rocketknight1
committed
Fix test
Rocketknight1
committed
Fix call with non-dev version of datasets
Rocketknight1
committed
Test correct column removal too
Rocketknight1
committed
make fixup
Rocketknight1
committed
More tests to make sure we remove unwanted columns
Rocketknight1
committed
Fix test to avoid predicting on unbuilt models
Rocketknight1
committed
Fix test to avoid predicting on unbuilt models
Rocketknight1
committed
Fix test to remove unwanted head mask columns from inputs
Rocketknight1
committed
Stop pushing your debug breakpoints to the main repo of the $2bn company you work for
Rocketknight1
committed
Skip the test in convnext because no grouped conv support
Rocketknight1
committed
Drop bools from the dataset dict
Rocketknight1
committed
Make style
Rocketknight1
committed
Skip the training test for models whose input dicts don't give us labels
Rocketknight1
committed
Skip transformerXL in the test because it doesn't return a simple loss
Rocketknight1
committed
Skip TFTapas because of some odd NaN losses
Rocketknight1
committed
make style
Rocketknight1
committed
make fixup
Rocketknight1
committed
Add docstring
Rocketknight1
committed
fixup
Rocketknight1
committed
Update src/transformers/modeling_tf_utils.py

Rocketknight1
and
sgugger
committed
Update src/transformers/modeling_tf_utils.py

Rocketknight1
and
sgugger
committed
Update src/transformers/modeling_tf_utils.py

Rocketknight1
and
sgugger
committed
Update src/transformers/modeling_tf_utils.py

Rocketknight1
and
sgugger
committed
Update src/transformers/modeling_tf_utils.py

Rocketknight1
and
sgugger
committed
Remove breakpoint from tests
Rocketknight1
committed
Fix assert, add requires_backends
Rocketknight1
committed
Protect tokenizer import with if TYPE_CHECKING
Rocketknight1
committed
make fixup
Rocketknight1
committed
Add noqa, more fixup
Rocketknight1
committed
More rearranging for ~* aesthetics *~
Rocketknight1
committed
Adding defaults for shuffle and batch_size to match to_tf_dataset()
Rocketknight1
committed
Update src/transformers/modeling_tf_utils.py

Rocketknight1
and
sgugger
authored

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add magic method to our TF models to convert datasets with column inference#17160

Commits on May 19, 2022

Add method to call to_tf_dataset() with column inference

Add test for dataset creation

Add a default arg for data collator

Fix test

Fix call with non-dev version of datasets

Test correct column removal too

make fixup

More tests to make sure we remove unwanted columns

Fix test to avoid predicting on unbuilt models

Fix test to avoid predicting on unbuilt models

Fix test to remove unwanted head mask columns from inputs

Stop pushing your debug breakpoints to the main repo of the $2bn company you work for

Skip the test in convnext because no grouped conv support

Drop bools from the dataset dict

Make style

Skip the training test for models whose input dicts don't give us labels

Skip transformerXL in the test because it doesn't return a simple loss

Skip TFTapas because of some odd NaN losses

make style

make fixup

Add docstring

fixup

Update src/transformers/modeling_tf_utils.py

Update src/transformers/modeling_tf_utils.py

Update src/transformers/modeling_tf_utils.py

Update src/transformers/modeling_tf_utils.py

Update src/transformers/modeling_tf_utils.py

Remove breakpoint from tests

Fix assert, add requires_backends

Protect tokenizer import with if TYPE_CHECKING

make fixup

Add noqa, more fixup

More rearranging for ~* aesthetics *~

Adding defaults for shuffle and batch_size to match to_tf_dataset()

Update src/transformers/modeling_tf_utils.py