Description
Describe the bug
When you use IKernel.InvokeStreamingAsync and have a FunctionFilter. The FunctionInvocationContext.Result.ValueType does not match the type passed into the generic on the InvokeStreamingAsync method.
To Reproduce
Steps to reproduce the behavior:
- Create a FunctionFilter
- Invoke the next parameter to generate the results
- Use context.Result.ValueType to check what the value type is
- Invoke a KernelFunction by using kernel.InvokeStreamingAsync by either passing in a type for the generic, such as string, or using the default of StreamingKernelContent.
Expected behavior
The type in context.Result.ValueType should match the generic type passed into kernel.InvokeStreamingAsync, or at least be IAsyncEnumerable where T is the generic type.
Screenshots
Using InvokeStreamingAsync and checking typeof(string), or typeof(IAsyncEnumerable)
**************************** Streaming String ******************************
################################# Function Filter #################################
ValueType: Microsoft.SemanticKernel.KernelFunctionFromPrompt+d__71[System.String] Unable to handle result type:Microsoft.SemanticKernel.KernelFunctionFromPrompt+<InvokeStreamingCoreAsync>d__7
1[System.String]
It looks like you started to ask a question, but didn't finish it. Please go ahead and complete your question, and I'll be happy to help!
Using InvokeStreamingAsync and checking typeof(StreamingKernelContent), or typeof(IAsyncEnumerable)
**************************** Streaming KernelContent ******************************
################################# Function Filter #################################
ValueType: Microsoft.SemanticKernel.KernelFunctionFromPrompt+d__71[Microsoft.SemanticKernel.StreamingKernelContent] Unable to handle result type:Microsoft.SemanticKernel.KernelFunctionFromPrompt+<InvokeStreamingCoreAsync>d__7
1[Microsoft.SemanticKernel.StreamingKernelContent]
Would you like to know the answer about what "New York" refers to?
Platform
- Language: C#
- Source: Microsoft.SemanticKernel 1.54.0
- AI model: llama3.2
- IDE: VS Code
- OS: Windows
Additional context
Full source code to reproduce the problem
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Microsoft.SemanticKernel;
// Create a kernel with Azure OpenAI chat completion
IKernelBuilder builder = Kernel.CreateBuilder();
builder.AddOpenAIChatCompletion(
modelId: "llama3.2",
apiKey: "YOUR_API_KEY",
httpClient: new HttpClient
{
Timeout = TimeSpan.FromMinutes(2),
BaseAddress = new Uri("http://localhost:7869/v1")
}
);
// Build the kernel
Kernel kernel = builder.Build();
kernel.FunctionInvocationFilters.Add(new FunctionFilter());
var questionAnswerFunction = kernel.CreateFunctionFromPrompt("Question: {{$input}}; Answer:");
Console.WriteLine("**************************** Streaming String ******************************");
await foreach (string text in kernel.InvokeStreamingAsync<string>(questionAnswerFunction, new() { ["input"] = "What is New York?" }))
{
Console.Write(text);
}
Console.WriteLine();
Console.WriteLine("**************************** Streaming KernelContent ******************************");
await foreach (StreamingKernelContent kernelContent in kernel.InvokeStreamingAsync(questionAnswerFunction, new() { ["input"] = "What is New York?" }))
{
Console.Write(kernelContent.ToString());
}
public class FunctionFilter : IFunctionInvocationFilter
{
public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next)
{
Console.WriteLine("################################# Function Filter #################################");
await next(context);
Console.WriteLine($"ValueType: {context.Result.ValueType}");
if (context.IsStreaming)
{
if (context.Result.ValueType == typeof(string))
{
var enuemerable = context.Result.GetValue<IAsyncEnumerable<string>>()!;
await foreach (var text in enuemerable)
{
Console.Write(text);
}
}
else if (context.Result.ValueType == typeof(IAsyncEnumerable<string>))
{
var enuemerable = context.Result.GetValue<IAsyncEnumerable<string>>()!;
await foreach (var text in enuemerable)
{
Console.Write(text);
}
}
else if (context.Result.ValueType == typeof(StreamingKernelContent))
{
var enuemerable = context.Result.GetValue<IAsyncEnumerable<StreamingKernelContent>>()!;
await foreach (var kernelContent in enuemerable)
{
Console.Write(kernelContent);
}
}
else if (context.Result.ValueType == typeof(IAsyncEnumerable<StreamingKernelContent>))
{
var enuemerable = context.Result.GetValue<IAsyncEnumerable<StreamingKernelContent>>()!;
await foreach (var kernelContent in enuemerable)
{
Console.Write(kernelContent);
}
}
else
{
Console.WriteLine($"Unable to handle result type:{context.Result.ValueType}");
}
}
}
}
Metadata
Metadata
Assignees
Type
Projects
Status