Skip to content

Slow Generation #13

@lmangani

Description

@lmangani

PR #11 fixed the blocker, but generation is now running so slow its unusable - most likely falling back to CPU only.
Lets review the code and make sure its taking full advantage of Apple Meta GPU features and MPS.

If there are blockers, lets explore them together and find an elegant solution to achieve native speed generation on Apple Metal GPUs

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions