-
Notifications
You must be signed in to change notification settings - Fork 68
Open
Labels
enhancementNew feature or requestNew feature or request
Description
### Features
- [x] https://github.com/microsoft/T-MAC/pull/46 for metter multi-threading performance on big.LITTLE architecture and support more models such as qwen2
- [x] Provide Ubuntu docker image for quick evaluation
- [ ] Auto detect matmul shapes for models in GGUF format.
### Documents
- [ ] New DEMO gif with phi-3.5 on Surface Laptop 7
- [ ] Add Android results
### Optional (maybe delayed for next release)
- [ ] Optimization for AVX-512 CPUs
- [ ] Optimization for ARM-v9.2 CPUs with SME2 LUTI4
- [ ] Optimization for multi-threading scenarios
scarlett2018, BarfingLemurs, bkaruman and GreenShadowsmassudy
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request