Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GPT2 torch.compile fix #1434

Merged
merged 1 commit into from
Oct 18, 2024
Merged

Conversation

dsmertin
Copy link
Contributor

What does this PR do?

Quick fix for this exception:

PT_HPU_LAZY_MODE=0 python run_generation.py --model_name_or_path gpt2 --batch_size 2 --max_new_tokens 100 --use_hpu_graphs --use_kv_cache --bf16 --bucket_size 50 --torch_compile

File "/root/optimum-habana/examples/text-generation/utils.py", line 187, in get_torch_compiled_model

    model.model = torch.compile(model.model, backend="hpu_backend", options={"keep_input_mutations": True})

  File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1732, in __getattr__

    raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")

AttributeError: 'GaudiGPT2LMHeadModel' object has no attribute 'model'

I'm also preparing better solution, but with more changes which can affect other models.

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests?

@jiminha jiminha requested a review from libinta October 17, 2024 16:51
@jiminha
Copy link
Collaborator

jiminha commented Oct 17, 2024

@regisss this also needs to pull in 1.14 release.

@regisss regisss merged commit 8b61e8b into huggingface:main Oct 18, 2024
regisss pushed a commit that referenced this pull request Oct 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants