Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] The Transformer Engine plugin seems to be incompatible with LayerNorm that has no weights. #3171

Open
2 of 4 tasks
IDKiro opened this issue Oct 16, 2024 · 0 comments
Open
2 of 4 tasks

Comments

@IDKiro
Copy link

IDKiro commented Oct 16, 2024

System Info

When the elementwise_affine attribute of PyTorch’s LayerNorm is set to False, the LayerNorm has no weights or biases.
At this point, the following error will occur:

  File "/workspace/.venv/lib/python3.10/site-packages/accelerate/accelerator.py", line 1342, in prepare
    args = self._prepare_te(*args)
  File "/workspace/.venv/lib/python3.10/site-packages/accelerate/accelerator.py", line 1642, in _prepare_te
    convert_model(model)
  File "/workspace/.venv/lib/python3.10/site-packages/accelerate/utils/transformer_engine.py", line 77, in convert_model
    convert_model(
  File "/workspace/.venv/lib/python3.10/site-packages/accelerate/utils/transformer_engine.py", line 77, in convert_model
    convert_model(
  File "/workspace/.venv/lib/python3.10/site-packages/accelerate/utils/transformer_engine.py", line 77, in convert_model
    convert_model(
  [Previous line repeated 2 more times]
  File "/workspace/.venv/lib/python3.10/site-packages/accelerate/utils/transformer_engine.py", line 55, in convert_model
    te_module = te.LayerNorm(module.normalized_shape[0], eps=module.eps, params_dtype=module.weight.dtype)
AttributeError: 'NoneType' object has no attribute 'dtype'

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • One of the scripts in the examples/ folder of Accelerate or an officially supported no_trainer script in the examples folder of the transformers repo (such as run_no_trainer_glue.py)
  • My own task or dataset (give details below)

Reproduction

  1. when finetuning the flux model using FP8

Expected behavior

  1. Just add a judgment
@IDKiro IDKiro changed the title The Transformer Engine plugin seems to be incompatible with LayerNorm that has no weights. [Bug] The Transformer Engine plugin seems to be incompatible with LayerNorm that has no weights. Oct 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant