Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix MLPSpeculator handling of num_speculative_tokens #5876

Merged
merged 1 commit into from
Jun 27, 2024
Merged

[BugFix] Fix MLPSpeculator handling of num_speculative_tokens #5876

merged 1 commit into from
Jun 27, 2024

Commits on Jun 26, 2024

  1. [BugFix] Fix MLPSpeculator handling of num_speculative_tokens 

    This fix avoids loading more layers of the speculator model than necessary based on the configured num_speculative_tokens value.

The bug didn't affect the number of speculative tokens actually generated at runtime.
    @njhill
    njhill committed Jun 26, 2024
    Configuration menu
    Copy the full SHA
    d6158e9 View commit details
    Browse the repository at this point in the history