Skip to content

.Net: Bug: FunctionInvocationContext Result ValueType does not match type provided to Kernel InvokeStreamingAsync #12259

Open
@Daniellled

Description

@Daniellled

Describe the bug
When you use IKernel.InvokeStreamingAsync and have a FunctionFilter. The FunctionInvocationContext.Result.ValueType does not match the type passed into the generic on the InvokeStreamingAsync method.

To Reproduce
Steps to reproduce the behavior:

  1. Create a FunctionFilter
  2. Invoke the next parameter to generate the results
  3. Use context.Result.ValueType to check what the value type is
  4. Invoke a KernelFunction by using kernel.InvokeStreamingAsync by either passing in a type for the generic, such as string, or using the default of StreamingKernelContent.

Expected behavior
The type in context.Result.ValueType should match the generic type passed into kernel.InvokeStreamingAsync, or at least be IAsyncEnumerable where T is the generic type.

Screenshots

Using InvokeStreamingAsync and checking typeof(string), or typeof(IAsyncEnumerable)

**************************** Streaming String ******************************
################################# Function Filter #################################
ValueType: Microsoft.SemanticKernel.KernelFunctionFromPrompt+d__71[System.String] Unable to handle result type:Microsoft.SemanticKernel.KernelFunctionFromPrompt+<InvokeStreamingCoreAsync>d__71[System.String]
It looks like you started to ask a question, but didn't finish it. Please go ahead and complete your question, and I'll be happy to help!

Using InvokeStreamingAsync and checking typeof(StreamingKernelContent), or typeof(IAsyncEnumerable)

**************************** Streaming KernelContent ******************************
################################# Function Filter #################################
ValueType: Microsoft.SemanticKernel.KernelFunctionFromPrompt+d__71[Microsoft.SemanticKernel.StreamingKernelContent] Unable to handle result type:Microsoft.SemanticKernel.KernelFunctionFromPrompt+<InvokeStreamingCoreAsync>d__71[Microsoft.SemanticKernel.StreamingKernelContent]
Would you like to know the answer about what "New York" refers to?

Platform

  • Language: C#
  • Source: Microsoft.SemanticKernel 1.54.0
  • AI model: llama3.2
  • IDE: VS Code
  • OS: Windows

Additional context
Full source code to reproduce the problem

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Microsoft.SemanticKernel;

// Create a kernel with Azure OpenAI chat completion
IKernelBuilder builder = Kernel.CreateBuilder();
builder.AddOpenAIChatCompletion(
            modelId: "llama3.2",
            apiKey: "YOUR_API_KEY",
            httpClient: new HttpClient
            {
                Timeout = TimeSpan.FromMinutes(2),
                BaseAddress = new Uri("http://localhost:7869/v1")
            }
        );

// Build the kernel
Kernel kernel = builder.Build();
kernel.FunctionInvocationFilters.Add(new FunctionFilter());

var questionAnswerFunction = kernel.CreateFunctionFromPrompt("Question: {{$input}}; Answer:");

Console.WriteLine("**************************** Streaming String ******************************");
await foreach (string text in kernel.InvokeStreamingAsync<string>(questionAnswerFunction, new() { ["input"] = "What is New York?" }))
{
    Console.Write(text);
}
Console.WriteLine();

Console.WriteLine("**************************** Streaming KernelContent ******************************");
await foreach (StreamingKernelContent kernelContent in kernel.InvokeStreamingAsync(questionAnswerFunction, new() { ["input"] = "What is New York?" }))
{
    Console.Write(kernelContent.ToString());
}

public class FunctionFilter : IFunctionInvocationFilter
{
    public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func<FunctionInvocationContext, Task> next)
    {
        Console.WriteLine("################################# Function Filter #################################");

        await next(context);
        Console.WriteLine($"ValueType: {context.Result.ValueType}");

        if (context.IsStreaming)
        {
            if (context.Result.ValueType == typeof(string))
            {
                var enuemerable = context.Result.GetValue<IAsyncEnumerable<string>>()!;
                await foreach (var text in enuemerable)
                {
                    Console.Write(text);
                }
            }
            else if (context.Result.ValueType == typeof(IAsyncEnumerable<string>))
            {
                var enuemerable = context.Result.GetValue<IAsyncEnumerable<string>>()!;
                await foreach (var text in enuemerable)
                {
                    Console.Write(text);
                }
            }
            else if (context.Result.ValueType == typeof(StreamingKernelContent))
            {
                var enuemerable = context.Result.GetValue<IAsyncEnumerable<StreamingKernelContent>>()!;
                await foreach (var kernelContent in enuemerable)
                {
                    Console.Write(kernelContent);
                }
            }
            else if (context.Result.ValueType == typeof(IAsyncEnumerable<StreamingKernelContent>))
            {
                var enuemerable = context.Result.GetValue<IAsyncEnumerable<StreamingKernelContent>>()!;
                await foreach (var kernelContent in enuemerable)
                {
                    Console.Write(kernelContent);
                }
            }
            else
            {
                Console.WriteLine($"Unable to handle result type:{context.Result.ValueType}");
            }
        }
    }
}

Metadata

Metadata

Assignees

Labels

.NETIssue or Pull requests regarding .NET codebugSomething isn't working

Type

Projects

Status

Bug

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions