[LED] fix global_attention_mask not being passed for generation and docs clarification about grad checkpointing

[caesar-one]

What does this PR do?

There are two changes:

1. The global_attention_mask parameter was not passed in .prepare_inputs_for_generation(...). So, when you call the .generate(...) method it never reaches the LEDModel.forward(...) (even though global_attention_mask is passed in kwargs). So local attention is used for all tokens. This problem was mentioned also in this pull request, but not completely fixed.
2. The documentation of LED states:

To fine-tune LED on all 16384, it is necessary to enable gradient checkpointing by executing model.gradient_checkpointing_enable().

This could be misleading since gradient checkpointing can be used if training leads to OOM errors, but it is not actually necessary, as mentioned in this issue. Some hints on the use of the use_cache flag were also added, as suggested in this notebook by @patrickvonplaten (who initializes the model using AutoModelForSeq2SeqLM.from_pretrained("allenai/led-base-16384", gradient_checkpointing=True, use_cache=False).

Fixes # (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@patrickvonplaten @sgugger

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[JohnGiorgi]

Thanks for catching this @caesar-one, looking forward to it getting merged as I am also trying to use LED-based models with a global attention mask!

[patrickvonplaten]

@ydshieh could you also take a quick look? :-)

[ydshieh]

LGTM, but left a question for the documentation. @patrickvonplaten Could you double check?

Thanks for the fix in this PR, @caesar-one !

[ydshieh]

I am not very sure about

`gradient_checkpointing=True` flag in `.from_pretrained(...)` when loading the model.

as I see

transformers/src/transformers/configuration_utils.py

Lines 361 to 367 in b073345

	# Deal with gradient checkpointing
	if kwargs.get("gradient_checkpointing", False):
	warnings.warn(
	"Passing `gradient_checkpointing` to a config initialization is deprecated and will be removed in v5 "
	"Transformers. Using `model.gradient_checkpointing_enable()` instead, or if you are using the "
	"`Trainer` API, pass `gradient_checkpointing=True` in your `TrainingArguments`."
	)

It looks like we would rather use gradient_checkpointing for TrainingArguments.

[patrickvonplaten]

Agree with @ydshieh, it should be passed to the TrainingArguments

[caesar-one]

@patrickvonplaten could this issue be the reason for the lower performances of Primera (LED-based model) in the hf implementation vs the original implementation? After all, global attention was not considered during generation.

Anyway, thanks for your fast feedback :-)

[patrickvonplaten]

Thanks a lot for the PR @caesar-one !