Description
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
The GenAI
packages will provide torchsharp implementation for a series of popular GenAI models. The goal is to load the same weight from the corresponding python regular model.
- Add design doc (add document for GenAI #7170)
- Add
Microsoft.ML.GenAI.Core
(Add GenAI core package #7177)
The following models will be added in the first wave
- Phi-3 (
Microsoft.ML.GenAI.Phi
) Add Microsoft.ML.GenAI.Phi, test package and sample project. #7184- Add README to
Microsoft.ML.GenAI.Phi
project [GenAI] Add readme to Microsoft.ML.GenAI.Phi #7206
- Add README to
- LLaMA (
Microsoft.ML.GenAI.LLaMA
) [GenAI] Add LLaMA support #7220 - Mistral (
Microsoft.ML.GenAI.Mistral
)- Mistral-7b-instruct v3
- Mistral-nemo
- Generate Embedding from CausalLMModel [GenAI] Add generateEmbedding API to CausalLMPipeline #7227
- Stable Diffusion (
Microsoft.ML.GenAI.StableDiffusion
)
MEAI intergration
- Add CausalLMPipelineChatClient [GenAI] Introduce CausalLMPipelineChatClient for MEAI.IChatClient #7270
Along with the benchmark
-
Benchmark for Phi-3
-
Flash Attention support [GenAI] Add FlashAttention #7238
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.