Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Qwen2 models #746

Merged
merged 11 commits into from
Dec 5, 2024
Merged

Add support for Qwen2 models #746

Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
4457558
feat(export): allow to set fuse_qkv=False for decoders
dacorvo Dec 2, 2024
bb05b89
feat(decoder): add Qwen2 modeling code
dacorvo Dec 2, 2024
b7adfe0
feat(decoder): allow export from local class
dacorvo Dec 2, 2024
2afc69d
refactor(exporters): isolate decoder export configs
dacorvo Dec 3, 2024
d0f1ebc
feat(export): add support for Qwen2
dacorvo Dec 2, 2024
bd9dd97
test(decoder): add Qwen2
dacorvo Dec 3, 2024
c61cadd
test(tgi): add Qwen2 tests
dacorvo Dec 3, 2024
ddc084b
perf: add QwenMath accuracy example results
dacorvo Dec 3, 2024
1c831a7
perf(tgi): add Qwen2.5-7b performances
dacorvo Dec 3, 2024
113810e
review: add type annotations
dacorvo Dec 4, 2024
6012a37
review: add comments in Qwen2ForSampling
dacorvo Dec 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat(decoder): allow export from local class
  • Loading branch information
@dacorvo
dacorvo committed Dec 2, 2024
commit b7adfe085b049d1bc17b2bd13692596560a08bd1
15 changes: 10 additions & 5 deletions optimum/exporters/neuron/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -452,11 +452,16 @@ def __init__(self, task: str):
raise ModuleNotFoundError(
"The mandatory transformers-neuronx package is missing. Please install optimum[neuronx]."
)
module_name, class_name = self.NEURONX_CLASS.rsplit(".", maxsplit=1)
module = importlib.import_module(f"transformers_neuronx.{module_name}")
self._neuronx_class = getattr(module, class_name, None)
if self._neuronx_class is None:
raise ImportError(f"{class_name} not found in {module_name}. Please check transformers-neuronx version.")
if isinstance(self.NEURONX_CLASS, type):
self._neuronx_class = self.NEURONX_CLASS
else:
module_name, class_name = self.NEURONX_CLASS.rsplit(".", maxsplit=1)
module = importlib.import_module(f"transformers_neuronx.{module_name}")
self._neuronx_class = getattr(module, class_name, None)
if self._neuronx_class is None:
raise ImportError(
f"{class_name} not found in {module_name}. Please check transformers-neuronx version."
)

@property
def neuronx_class(self):
Expand Down