add arg padding_free to DataCollatorForCompletionOnlyLM

[RhuiDih]

add argument DataCollatorForCompletionOnlyLM(padding_free=True) to utilize huggingface/transformers#31629

[ArthurZucker]

cc @kashif this is linked to huggingface/transformers#31629 which will be in the next transformers release!

[kashif]

thanks @ArthurZucker

[pcuenca]

Gentle ping and info, this post covers the support in transformers: https://huggingface.co/blog/packing-with-FA2

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[lewtun]

Thanks for adding this neat extension to the collator! It LGTM barring the unit tests using the assert methods from unittest

[lewtun]

Can you please replace these assert statements with assert methods from unittest, e.g. self.assertTrue() for this case

[kashif]

Suggested change

	assert "attention_mask" not in batch_paddingfree
	self.assertNotIn("attention_mask", batch_paddingfree)

[RhuiDih]

@kashif @lewtun thanks for the suggestion, just pushed the update