Use ACT2FN to fetch ReLU activation in the T5 model #16874

eldarkurtic · 2022-04-21T13:17:05Z

all activations should be fetched through ACT2FN
it returns ReLU as nn.Module, which allows attaching hooks on the activation function and prints it to stdout when print(model)

- all activations should be fetched through ACT2FN - it returns ReLU as `nn.Module`, which allows attaching hooks on the activation function and prints it to stdout when `print(model)`

LysandreJik

LGTM!

HuggingFaceDocBuilderDev · 2022-04-21T13:33:01Z

The documentation is not available anymore as the PR was closed or merged.

- all activations should be fetched through ACT2FN - it returns ReLU as `nn.Module`, which allows attaching hooks on the activation function and prints it to stdout when `print(model)`

Use ACT2FN to fetch ReLU activation

af10501

- all activations should be fetched through ACT2FN - it returns ReLU as `nn.Module`, which allows attaching hooks on the activation function and prints it to stdout when `print(model)`

LysandreJik approved these changes Apr 21, 2022

View reviewed changes

LysandreJik requested a review from patrickvonplaten April 21, 2022 13:21

LysandreJik merged commit bae9b64 into huggingface:main Apr 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use ACT2FN to fetch ReLU activation in the T5 model #16874

Use ACT2FN to fetch ReLU activation in the T5 model #16874

eldarkurtic commented Apr 21, 2022

LysandreJik left a comment

HuggingFaceDocBuilderDev commented Apr 21, 2022 •

edited

Loading

Use ACT2FN to fetch ReLU activation in the T5 model #16874

Use ACT2FN to fetch ReLU activation in the T5 model #16874

Conversation

eldarkurtic commented Apr 21, 2022

LysandreJik left a comment

Choose a reason for hiding this comment

HuggingFaceDocBuilderDev commented Apr 21, 2022 • edited Loading

HuggingFaceDocBuilderDev commented Apr 21, 2022 •

edited

Loading